UPDATED 22:24 EDT / SEPTEMBER 23 2019

AI

Amazon’s Echo can now help describe grocery items to the visually impaired

Amazon.com’s virtual assistant just got more intelligent after the company revealed Monday that the visually impaired can now ask it what they are holding.

The new feature, called “Show and Tell,” enables customers who are blind or people with low vision to hold an item in front of first- and second-generation Echo Show devices and ask, “Alexa, what am I holding?” or “Alexa, what’s in my hand?” The object will then be identified using Amazon’s machine learning technology.

That could be especially helpful for grocery items such as tins and boxes that might be hard to identify properly. As Amazon demonstrated in a video, a blind person asked Alexa what she’s holding and Alexa replied, “It looks like tea.”

“The whole idea for Show and Tell came about from feedback from blind and low vision customers,” said Sarah Caplener, who leads Amazon’s Alexa for Everyone team. “We heard that product identification can be a challenge and something customers wanted Alexa’s help with. Whether a customer is sorting through a bag of groceries, or trying to determine what item was left out on the counter, we want to make those moments simpler by helping identify these items and giving customers the information they need in that moment.”

Partnering with California-based Vista Center for the Blind and Visually Impaired, Amazon worked with visually impaired people to understand the problems they have at home and how Alexa could help.

One volunteer who is completely blind called the technology “revolutionary,” saying with such devices acting as his eyes, he is much more independent. He said he likes to cook, but without the help of a personal assistant, it’s impossible to know what spice he is holding. In another example, Alexa identifies for him a box of macaroni and cheese.

“It’s essential to work with our customers, not just for them, to create something that’s truly helpful,” said Caplener.

The downside: The feature is available at the moment only in the U.S. Amazon didn’t say when it will roll out to other countries. To use the feature, customers just have to hold a product in front of the Echo Show and ask the magic question.

Photo: Amazon

A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU