How Amazon teaches Alexa, and what it hopes the virtual assistant will learn someday

Credit: geekwire.com

Amazon senior principal scientist Nikko Strom at the AI NEXT tech conference. (GeekWire Photo / Geof Wheelwright)

Days after Amazon announced that it was bringing its Alexa to its iOS Amazon app, Amazon senior principal scientist Nikko Strom spoke at the AI NEXT tech conference in Bellevue, Wash., this weekend to share behind-the-scenes details of the company’s voice-enabled assistant and its broader artificial intelligence initiatives.


Strom, a founding member of the team that built Amazon Echo and Alexa, told the audience of AI scientists that the growing number of Alexa-based devices (not publicly disclosed by Amazon but estimated by Consumer Intelligence Research Partners at more than 8 million) has provided Amazon with a significant amount of data to use in improving and refining Alexa-powered devices.


“All of these things that make Alexa great and expanding all the time means that we get lots of data,” he said. “One of the things about this era is that people actually like using these (devices) — I’ve been in the industry for a long time and I worked on telephony systems and people didn’t really like to use them.”


Strom compared the amount of data that Amazon received from the millions of Alexa devices to what a 16-year-old might have heard during their young life. He said that in 16 years, a person might hear — and have “training data” about — as much as 14,016 hours of speech (based on the assumption that about 10 percent of what a person hears in a day is speech).


Amazon uses “large-scale distributed training” to analyze the voice data it gets from users Alexa-enabled devices in order to improve their speed and accuracy.

“We have all this data – we have thousands of hours of stored data from our customers in Amazon S3 (Amazon Simple Storage Service) and we train these models on AWS EC2 (Amazon Web Services Elastic Compute Cloud) instances,” he said, explaining that the company has to use “distributed training” across 80 GPU (graphical processing unit) instances in order to crunch the massive amount of data it receives.


This large-scale distributed training of the voice recognition model in Alexa allows Amazon to constantly make updates to accuracy and quality.


Strom also took time to address concerns about what, when and how Amazon collects voice data — and stressed that the company is only interested in the voice data necessary to run its services, not in the content of anyone’s conversations.


The issue recently came to public attention in an Arkansas murder case where Bentonville police issued a warrant demanding records for an Echo device belonging to a charged murder suspect. The case has prompted debate about how First Amendment rights should be protected when speech is stored on digital devices. In reply to a general question about the Alexa technology and privacy, Strom suggested that Alexa’s handling of voice data is not always entirely understood from people who read about it in the press.


“What people don’t always get in these articles is that it (Alexa) is listening for a wake word all the time. It is only listening for the wake word,” explained Strom. “It’s only when the blue ring starts spinning — that’s when Alexa has heard the wake word and starts recording you. Only the thing you say after that wake word is ever recorded.”


Alexa has branched out from Amazon’s Echo and Fire TV devices into a growing number of third-party products. (Amazon Photo)

The range of Alexa devices is also growing, with the technology now in use on smartphones, cars and refrigerators. In addition, Strom said that the number of third party “skills” (voice-activated apps enabled for use with Alexa) is growing so fast that it’s hard to keep up with. Amazon recently said Alexa surpassed 10,000 skills.


“Skills are super-exciting, but they are also a big challenge for us because there’s many of them and we don’t build them ourselves,” he said, as he frankly discussed the need to maintain strong communications with skills developers.


Finally, Strom hinted about what it might take to make Alexa a little smarter — so that it understands what someone means, not just what they say. To do that, Alexa would have to tackle emotions and intonation.


“Alexa cannot capture the emotion in your speech right now, but it can do something indirectly by capturing the meaning of what you say — which can be emotional,” he said. “It will recognize your curse words, for example. We have over 100 scientists working on Alexa on speech in general.”


Strom said the company didn’t yet have anything to announce on emotion recognition, but that it would continue to be an area of interest.


Amazon is enjoying strong sales for its Alexa-enabled devices – and a recent analysts from RBC Capital Markets recently estimated that sales of Alexa devices could hit $5 billion by 2020 (with another $5 billion in annual revenues predicted to come from shopping done via the voice assistant). But it is not alone in the market. Microsoft’s Cortana (included with Windows 10 and available on iOS and Android devices), Google’s Google Home and Google Assistant (which is pre-loaded on Android smartphones) are still strong competitors.


Top Stories

Doctor Who review: Bill and the Timelord must grin and bear it in Smile

“There’s a giant smiley abattoir, and I’m having this childish impulse to blow it up.”
  • 1 day ago
  • 13

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago
  • 4

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago
  • 4

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago
  • 4

Watch ‘Science Guy’ Bill Nye’s epic defense of science at ‘March for Science’ in Washington, D.C.

Bill Nye “the Science Guy,” the television personality and former Boeing engineer who got his start in Seattle TV sketch comedy, gave a rousing speech in defense of science at the March for Science today in Washington, D.C. — to the traditional chant of “Bill, Bill, Bill, Bill!” from the crowd. Watch his address above and continue reading for a transcript. Bill Nye, Planetary Society CEO and host of “Bill Nye Saves The World.” Greetings, greetings, follow citizens! We are marching today to remind people everywhere — our lawmakers, especially — of the significance of science for our health and prosperity. The process of science… Read More
  • 1 day ago
  • 3
farawayyachtingcharters - Easy Branches
botoxfillerveintheraphyinphuket - Easy Branches

Latest in Technology

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago

Week in Review: Most popular stories on GeekWire for the week of April 16, 2017

See the technology stories that people were reading on GeekWire for the week of April 16, 2017.… Read More
  • 16 hours ago

Doctor Who review: Bill and the Timelord must grin and bear it in Smile

“There’s a giant smiley abattoir, and I’m having this childish impulse to blow it up.”
  • 1 day ago

Scenes from the ‘other Washington’: D.C.’s March for Science sparks a chain reaction

WASHINGTON, D.C. – The scenery for the world’s most watched March for Science may have featured the political icons of the nation’s capital, but the weather was pure Seattle. Today’s rally brought thousands of people to the National Mall for hours’ worth of teach-ins and speeches, despite a Seattle-style drizzle. Bill Nye the Science Guy, a co-chair for the national March for Science movement, struck on the theme of the day when he noted that many policymakers in Washington and around the world were “deliberately and actively suppressing science.” He also noted the stereotypical view of scientists as detached nerds who dwell… Read More
  • 1 day ago

‘The oceans are rising and so are we.’ Photos, tweets, and more from Seattle’s March for Science

It’s a soggy Seattle Saturday but that didn’t deter thousands of activists from marching for climate research and sound science policy today. A huge assembly marched from Cal Anderson park in Capitol Hill to the International Fountain at Seattle Center, many chanting “the oceans are rising and so are we.” Washington Gov. Jay Inslee and Seattle Mayor Ed Murray came out to show their support. Seattle’s March for Science is one of countless events taking place across the country in honor of Earth Day. Many of the people bracing the fickle April elements are protesting President Trump’s actions related to climate change… Read More
  • 1 day ago

Week in Geek: Facebook tries telepathic typing, inside an Amazon greenhouse, and more

Are you ready to type with just your brain? Facebook revealed this week that it’s working with a team of more than 60 scientists around the world to develop what it calls a “silent speech” interface to decode human thoughts and convert them to text. We debate the potential benefits and risks of this futuristic technology on this episode of the Week in Geek, our weekly tech news podcast. In other news, Google parent Alphabet is looking for 10,000 volunteers around the world to share personal medical information over the course of four years. The goal is to get a big-picture look at human health and track possible… Read More
  • 2 days ago

This Week in Seattle: Mayoral race heats up with income tax proposals and new contenders

This week in Seattle: The mayoral race heats up; Scoop partners with the city to promote carpooling; Earth Day celebrations kick off; and Gov. Inslee calls for a special session. Seattle celebrates Earth Day like the nature freaks we are Happy Earth Day! There are events all around town today celebrating this gorgeous blue planet we call home. Champions of government-sponsored science research and policies that support those programs will March for Science at Cal Anderson Park. Local, sustainable food advocates should head over to MOHAI for the Edible City Science Fair. The Student Conservation Association is planning a day… Read More
  • 2 days ago

Russian man gets longest-ever US hacking sentence, 27 years in prison

Roman Seleznev bankrupted businesses, did $170 million in damage.
  • 2 days ago

Seahawks star Russell Wilson, astronaut Buzz Aldrin share inspiration with students at WE Day

Write down your goals, listen more, and dream big. These are good pieces of advice for anyone, really, but they held special importance for more than 15,000 students who gathered inside an energetic Key Arena on Friday in Seattle for WE Day, an annual event that celebrates youth empowerment. WE Day events, hosted by non-profit WE, are attended annually by more than 200,000 students from more than 10,000 schools in the U.S., U.K., and Canada. Students earn tickets by participating in local WE Schools programs. There were several celebrities who spoke on stage Friday in Seattle, including space icon Buzz Aldrin. The Apollo… Read More
  • 2 days ago