A Tech Walkthrough of Amazon Fresh

Last week I experienced my first taste of an Amazon Fresh and surprisingly made huge savings on my grocery shopping when I visited the first recently opened store in Southern California. Luckily for me was only a 5-minute drive away. Excited by the thought of ‘Just Walk Out’ shopping, deep learning, biometrics, and being stalked, hmm, I proceeded to enter the store with my partner.

As you can see by the picture, there are three ways to enter the Amazon Fresh store. A little incorrect, there are 4 ways to enter.

1. Via the Amazon App – Checking with the Store Code and then Scan to Enter
2. Scan your palm – You need to link your palm to your Amazon account
3. Use your Credit or debit card which is linked to your Amazon account
4. Go through the traditional shopping gate – What I call legacy shopping

I hadn’t had a chance to link my palm to my amazon account, and to save time, I decided to go for option 2 for this visit. Use my debit card that is linked to my Amazon account to enter, I head to the gates, somewhat excited about what’s to come.

Error! No Amazon account found. Sometimes I have had issues with my Starling debit card before, and perhaps for some reason it doesn’t like debit cards, so I decided to go for the next best option and use my AMEX credit card which is linked to my Amazon account.

As a side note, it seems a lot of Americans use ‘credit card’ as a blanket term for paying with a swiped/contactless card, a conversation that I had with my partner when I explained I was using my debit card.

Two cards and no success, an Amazon rep approaches (they are never more than 5 steps away), and we discuss the problem. Apparently, this has happened a few times before when the credit card is linked to an Amazon account outside of the US, after asking a few more questions, we find out that it has happened every time in this store. I feel a little unworthy now.

This is odd, I don’t have any problem ordering goods from amazon.fr or amazon.com, but it is apparently a known issue, fix it Amazon! So now we are limited to legacy shopping, I’m really feeling underwhelmed at this point, and considering walking back out. Perhaps this blog will end here…

To the rescue, my partner proposes that we use her account to enter the store, save on queuing (getting in line as Americans say) and experience Amazon Fresh as it’s designed. She even suggests scanning my palm and linking it to her account, I declined this offer, it’s a grocery store, and I might have thought twice if they had electronics for sale. I do wonder about how Amazon Fresh stores biometrics; I will investigate further.

Amazon One is the technology that Amazon uses for palm scanning, using proprietary imaging and computer vision algorithms, recognising your palm in seconds. You can learn more about how Amazon One works here.

We enter the store, smoothly by inserting her credit card into the machine at the gate, it swings open, and we are in.

The store itself has a warehouse feel to it, looking around and the first thing that I am drawn to is the sheer number of cameras dotted about the place, there are hundreds of them on every row, thousands in total.

As soon as you enter those gates you are being watched, those thousands of cameras are tracking your every move around the store, as well as proximity weight sensors, which all come together to provide a seamless shopping experience, using what Amazon calls “Just Walk Out” shopping.

The cameras are hooked up to custom boards that perform computer vision functionalities, which I believe are for object identification and motion detection. There are also separate depth-sensing cameras if you look close enough. The images taken from the cameras are then sent back to some sort of centralised processing area, where deep learning tech identifies the people shopping in the store, and the objects they are picking up from the shelves, and putting them back. All this information is then stored on a virtual shopping cart linked to each shopper’s Amazon account. No product scanning is needed, just pick up what you want, put it in a shopping cart, or directly in your bag for life. Or pile everything up on your arms, lean it against your body and juggle all the items until you walk out to your car, we have all done that right?

It wouldn’t be an Amazon store without Alexa, and there are a number of these stations dotted around the store.

Useful if you don’t know where an item is located in-store, or you want to pair a nice bottle of red with some cheese, or perhaps you are feeling lonely and you want Alexa to read you a story.

No matter what questions you have, Alexa is on hand to help out.

The store itself is a typical grocery-type store branded in Amazon colours, with the most noticeable difference being a lot of wasted open space, this could be designed that way on purpose though for the tech to work. There is a salad bar, homemade pizzas on demand, and then your standard food aisles.

The prices are reasonable enough. Apparently, you receive special offers direct to your device when inside of any Amazon Fresh stores, something that I didn’t get to experience, unfortunately.

There is a small tech-related area in-store, so it feels more like an Amazon, a full aisle dedicated to tech would have been much better and would certainly increase the average value of a shopping cart.

I also liked the inclusion of a returns and customer pickup desk, an easy way to send back any items that are faulty or pick up an order from a secure drop box.

Food shopping + returns + pickup in one trip.

When you have finished your shopping, you just scan to walk out of the gates and just walk out. No need to queue, no need to pack your bags at the checkout, no need to self-scan any items, just walk out. It felt strange the first time, we had to ask an Amazon rep if we needed to do anything else.

As soon as you get out of the building, payment is taken from your linked credit card, and a receipt is delivered to your phone. Pretty awesome really.

So how did I make huge savings on my grocery shopping? I didn’t pay for any of it that’s how, none of my Amazon linked cards would work at the entrance gates, so my partner used her account to pay for all the shopping, hopefully, Amazon will fix this issue moving forward, as we see the store being more for convenience purposes rather than being value for money, but I must point out that it wasn’t expensive either, but for a traditional grocery shopping experience I would go to a larger store that provides more value and options, so it really is the technology that makes Amazon Fresh worth shopping at.

At the time of writing, there are 33 Amazon Fresh stores located across the USA, as well as 15 stores in London and other international cities.

Want to Establish a data modeling strategy for your business?

Contact Nexus Technology to hear how.