AI and computer vision drive a growing shop-and-go platform

AI and computer vision weren’t necessarily top-of-mind for Sodexo, a food and facilities management company that manages more than 400 university food programs, which was looking for a forward-looking, seamless experience to offer students rather than the usual buffet meals .

All the company knew was that they wanted something like Amazon Go’s checkout-free, shop-and-go stores. That is, where shoppers can walk in, take items off the shelves, and leave without queuing at the register or suffering from swiping codes at self-checkout.

“Students today want things that they can partially or fully prepare in their room or apartment, with organic, very local options,” said Kevin Rettle, global vice president of product development and digital innovation at Sodexo. “We also wanted to remove friction, but many solutions still require the guest’s interaction with a cashier — this generation really doesn’t want to talk to a lot of people during their service interactions.”

For the University of Denver, Sodexo chose the San Jose-based AiFi, which provides a frictionless and cashierless AI-powered retail solution. Its flexibility (the company says it can deploy two stores a week) and diverse locations (sports stadiums, music festivals, supermarket chains, college campuses and more) make it unique, explains Steve Gu, who co-founded AiFi in 2016 with his wife. Ying Zheng. Both Gu and Zheng have PhDs in computer vision and have spent time at Apple and Google.

Powered solely by cameras and computer vision technology, AiFi today announced that it now has a total of 80 checkout-free stores worldwide, along with retailers such as Carrefour, Aldi, Loop and Verizon. It has also opened 53 Zabka stores in Poland and 2 NFL stores. Gu states that this is an industry benchmark for how to scale this technology in a way that: Amazon Gothat has more than 42 stores, can’t.

Cameras and computer vision, not sensors

Amazon Go’s stores are equipped with specialized cameras, sensors and weighted shelves, Gu explained. “That makes the solution very expensive and difficult to scale,” he said. Instead, AiFi uses the “cheapest possible out-of-the-box cameras,” combined with what it believes is its real strength: computer vision.

AiFi uses advanced AI models via a host of cameras placed across the ceiling, Gu said, to understand everything that happens in the store. Cameras track customers throughout their shopping journey, while computer vision recognizes products and detects various activities, including putting items on or taking off the shelves.

Under the hood of the platform are neural network models developed specifically for people tracking and for activity and product recognition. AiFi also developed advanced calibration algorithms that allow the company to recreate the retail environment in 3D.

AiFi also uses simulated data sets. “We spend quite a bit of effort building those simulated environments so that we can train the AI ​​algorithms and the models in them,” Gu said. “That really helps us develop those models faster and make them more scalable.”

In a simulated world, he explains, you can easily customize human shapes and features, as well as shelf layout and product appearance. You can create a cluttered, overcrowded retail environment or one that is neat and orderly. “Things that can’t be done in the real world can easily be done in a simulated world,” he said. “The AI ​​can learn about those scenarios and then perform or outperform in a real-life environment.”

Computer vision that is constantly evolving

AiFi’s system is evolving and will improve over time, Gu continued, citing current challenges, including the platform’s ability to recognize small items like chewing gum or lipstick.

“If they’re not placed in the right place, it’s very difficult for the computer vision to discern what it is,” he said. There are also issues with items with similar looks and textures. “If they are placed together in adjacent rooms, it sometimes causes confusion for the cameras and computer vision to recognize these products,” he said. “But the good thing is it’s not purely based on the visual texture — you also have the 3D scene geometry, the location, the context.”

There are also current restrictions on the size of the store and the number of people it can track. “The question is, is the solution also scalable to 100,000-square-foot supercenters?” he said. “In addition, the system is able to track hundreds of people shopping at the same time in a retail environment. But to scale that up further, to track thousands of people, with very complex shopping behavior, that is something that is still being worked on.”

To enter an AiFi-powered store, shoppers don’t need a biometric scan or an AiFi app — they can use a credit card or use the retailer’s app. For example, at the University of Denver, Sodexo wanted a partner who was agnostic to the front. “We were able to use our wallet and payment processing and link the AiFi technology, the cameras and the AI ​​in our system,” said Rettle.

Consumer acceptance is key

“From a product ownership perspective, you always hold your breath a bit. Will it work?” he said. But in the end, the students at the University of Denver immediately switched to the AiFi concept.

“We didn’t have to teach any of the students what to do,” he said. “They get it without a lot of prompts.”

Critics in the retail space also predicted that the AiFi technology would be a “loss prevention nightmare — that the students will figure out how to play the system,” Rettle said. Instead, the current accuracy rate for the AiFi solution is 98.3%, and the shrink rate (where shoppers walk out without paying) has actually dropped, he said.

Some products don’t quite work with AiFi’s solution yet, Rettle admits, including student and fan swag. “The platform has therefore yet to understand consumer behavior, which will certainly evolve with the technology,” he said.

Rettle also said he doesn’t envision a campus or stadium that could shift to 100% autonomous retail. “For us, it’s something that complements,” he said. “But I see a strong future in terms of being able to continue to implement and drive ubiquity with the consumer adoption solution.”

For Gu, the potential of AiFi is “huge,” with more than a dozen new stores in the works and a growing partnership with Microsoft as an independent software vendor (AiFi runs its solution on Azure). “You’ll see a lot of autonomous retail in different industries – not just stadiums, festivals and universities, but also offices, cinemas and other spaces,” he said.