How Amazon teaches Alexa, and what it hopes the virtual assistant will learn someday

Credit: geekwire.com

Amazon senior principal scientist Nikko Strom at the AI NEXT tech conference. (GeekWire Photo / Geof Wheelwright)

Days after Amazon announced that it was bringing its Alexa to its iOS Amazon app, Amazon senior principal scientist Nikko Strom spoke at the AI NEXT tech conference in Bellevue, Wash., this weekend to share behind-the-scenes details of the company’s voice-enabled assistant and its broader artificial intelligence initiatives.


Strom, a founding member of the team that built Amazon Echo and Alexa, told the audience of AI scientists that the growing number of Alexa-based devices (not publicly disclosed by Amazon but estimated by Consumer Intelligence Research Partners at more than 8 million) has provided Amazon with a significant amount of data to use in improving and refining Alexa-powered devices.


“All of these things that make Alexa great and expanding all the time means that we get lots of data,” he said. “One of the things about this era is that people actually like using these (devices) — I’ve been in the industry for a long time and I worked on telephony systems and people didn’t really like to use them.”


Strom compared the amount of data that Amazon received from the millions of Alexa devices to what a 16-year-old might have heard during their young life. He said that in 16 years, a person might hear — and have “training data” about — as much as 14,016 hours of speech (based on the assumption that about 10 percent of what a person hears in a day is speech).


Amazon uses “large-scale distributed training” to analyze the voice data it gets from users Alexa-enabled devices in order to improve their speed and accuracy.

“We have all this data – we have thousands of hours of stored data from our customers in Amazon S3 (Amazon Simple Storage Service) and we train these models on AWS EC2 (Amazon Web Services Elastic Compute Cloud) instances,” he said, explaining that the company has to use “distributed training” across 80 GPU (graphical processing unit) instances in order to crunch the massive amount of data it receives.


This large-scale distributed training of the voice recognition model in Alexa allows Amazon to constantly make updates to accuracy and quality.


Strom also took time to address concerns about what, when and how Amazon collects voice data — and stressed that the company is only interested in the voice data necessary to run its services, not in the content of anyone’s conversations.


The issue recently came to public attention in an Arkansas murder case where Bentonville police issued a warrant demanding records for an Echo device belonging to a charged murder suspect. The case has prompted debate about how First Amendment rights should be protected when speech is stored on digital devices. In reply to a general question about the Alexa technology and privacy, Strom suggested that Alexa’s handling of voice data is not always entirely understood from people who read about it in the press.


“What people don’t always get in these articles is that it (Alexa) is listening for a wake word all the time. It is only listening for the wake word,” explained Strom. “It’s only when the blue ring starts spinning — that’s when Alexa has heard the wake word and starts recording you. Only the thing you say after that wake word is ever recorded.”


Alexa has branched out from Amazon’s Echo and Fire TV devices into a growing number of third-party products. (Amazon Photo)

The range of Alexa devices is also growing, with the technology now in use on smartphones, cars and refrigerators. In addition, Strom said that the number of third party “skills” (voice-activated apps enabled for use with Alexa) is growing so fast that it’s hard to keep up with. Amazon recently said Alexa surpassed 10,000 skills.


“Skills are super-exciting, but they are also a big challenge for us because there’s many of them and we don’t build them ourselves,” he said, as he frankly discussed the need to maintain strong communications with skills developers.


Finally, Strom hinted about what it might take to make Alexa a little smarter — so that it understands what someone means, not just what they say. To do that, Alexa would have to tackle emotions and intonation.


“Alexa cannot capture the emotion in your speech right now, but it can do something indirectly by capturing the meaning of what you say — which can be emotional,” he said. “It will recognize your curse words, for example. We have over 100 scientists working on Alexa on speech in general.”


Strom said the company didn’t yet have anything to announce on emotion recognition, but that it would continue to be an area of interest.


Amazon is enjoying strong sales for its Alexa-enabled devices – and a recent analysts from RBC Capital Markets recently estimated that sales of Alexa devices could hit $5 billion by 2020 (with another $5 billion in annual revenues predicted to come from shopping done via the voice assistant). But it is not alone in the market. Microsoft’s Cortana (included with Windows 10 and available on iOS and Android devices), Google’s Google Home and Google Assistant (which is pre-loaded on Android smartphones) are still strong competitors.


Top Stories

farawayyachtingcharters - Easy Branches
botoxfillerveintheraphyinphuket - Easy Branches

Latest in Technology

GeekWire Deals: Sleep soundly with this crazy-soft sheet set

Is there anything greater than an amazing night of sleep? When you wake refreshed, you can take on anything. Enhance your sleep sanctuary with today’s GeekWire Deals offer. Slip into the Premium Collection 1800 Thread Count Sheet Set. The finely woven bamboo derived microfibers come together for ultimate softness. Cover all your cozy bases with a flat sheet, fitted sheet, and four pillowcases. You’ll be sleeping so deeply, you’ll be dreaming in million-dollar ideas. These plush and durable sheets are normally $299.99, but you can tuck in for $41.99.
  • 9 hours ago

Bill Gates and Paul Allen had a business before Microsoft, and this engineer was their partner

Microsoft is one of the most successful and influential companies in the world, but did you know that Bill Gates and Paul Allen launched an earlier venture as high school students in Seattle? It was called Traf-O-Data, a startup in the 1970s that aspired to provide technology to count traffic. Their partner in Traf-O-Data was Paul Gilbert, who was a University of Washington electrical engineering student during the era that Gates and Allen were (now famously) using computer labs on the college campus at night, to hone their programming skills and develop their new business. Gilbert, now 65 years old, recalls seeing Gates and Allen walking down a University… Read More
  • 10 hours ago

This Week in Seattle: Uber’s uncertain future, South Lake Union upzone, and more stuff you should know

  This Week in Seattle: Lawmakers promote equality and development, Uber’s future in the city looks uncertain, and unemployment continues its steady decline. Continue reading for the week’s top regional stories. Uber exec says future uncertain if Seattle drivers unionize They’re speeding toward an impasse. Uber is fighting a landmark law that allows Seattle drivers to bargain collectively. Northwest GM Brooke Steger said she’s “unsure of the future of Uber in Seattle,” at an event this week, adding “we don’t know if we will be able to continue to operate here,” if drivers unionize. The law is scheduled to begin rolling… Read More
  • 12 hours ago

Amazon will collect sales tax in four additional states starting April 1

Amazon will start collecting sales tax in four additional states starting next month. Customers in Hawaii, Idaho, Maine, and New Mexico — the four remaining states that did not require sales tax payments on Amazon purchases — will start paying tax on April 1, CNBC reported on Friday. We’ve reached out to Amazon to confirm the new policy and update this story when we hear back. In February, Amazon added 10 more states where it now collects sales tax. Amazon doesn’t collect tax in Alaska, Delaware, Oregon, Montana, and New Hampshire, as those states do not have sales tax. Amazon has battled several… Read More
  • 1 day ago

GeekWire 200 March update: Startups spring up the list thanks to cash infusions

March Madness isn’t just about bracket pools, wild upsets and the Boss Button. It’s also a busy time for startups, if the GeekWire 200 list of privately-held Pacific Northwest startups is any indication. A whopping 14 companies made double-digit moves up the charts this week, as local startups continue to hire and grow. What would that be in March Madness terms, Fabulous 14? Funded 14? The top five remained static again this month with DocuSign at the top, followed by Redfin, Avalara, Blue Origin and Puppet. Bellevue, Wash.-based project management company Smartsheet continued its quest toward the top five, moving up another spot to number… Read More
  • 1 day ago

Outcry rises when Trump’s treasury chief, Steve Mnuchin, scoffs at AI impact on jobs

Experts say the potential impact of automation and artificial intelligence could be one of the biggest economic issues of the 21st century, but Treasury Secretary Steve Mnuchin says it’s not on his radar screen. Mnuchin made his comments during a “News Shapers” sitdown with Axios’ Mike Allen. His observations are pointed enough, and brief enough, that they’re worth an extended quote: Mnuchin: “In terms of artificial intelligence taking over American jobs, I think we’re like so far away from that, not even on my radar screen.” Allen: “How far away?” Mnuchin: “Far enough that it’s …” Allen: “Seven more years?” Mnuchin:… Read More
  • 1 day ago

Working Geek: FlowPlay CEO Derrick Morton relieves stress by going totally off the grid

Derrick Morton knows how to keep it all in balance. He’s spent the past decade as CEO of multi-platform game developer FlowPlay while still making time for his interests — like backpacking, live music, and travel. “My friend, co-founder, and CTO Doug Pearson and I started our journey with FlowPlay in 2007 and recently celebrated 10-years of building virtual worlds for audiences from every walk of life,” Morton said. While Pearson handles the technology side of the business, Morton focuses on managing teams, marketing initiatives, business development, and operations. Before FlowPlay, he spent three years at RealNetworks, where he served… Read More
  • 1 day ago

Spacewalkers help get station ready for space taxis during first of three outings

Spacewalkers made progress today on preparations at the International Space Station for the arrival of the first commerclal space taxis, which could happen as early as this year. During today’s operation, which lasted just over six and a half hours, NASA astronaut Shane Kimbrough and French astronaut Thomas Pesquet disconnected cables and electrical connections on a big piece of equipment known as the Pressurized Mating Adapter-3, or PMA-3. NASA said the astronauts also lubricated parts on the Dextre manipulator that’s at the end of the station’s Canadian-built robotic arm, inspected a radiator valve and replaced some external cameras. PMA-3 serves as… Read More
  • 1 day ago

The Week in Geek: How Starbucks and Amazon are steadily becoming more alike

Starbucks and Amazon, two of Seattle’s iconic corporate giants, started off very different from each other. At heart, Starbucks is a chain of coffee stores, and Amazon is an e-commerce company. But they’re slowly and steadily becoming … well, more alike. In this episode of the Week in Geek podcast, we recap the big news of the week from Starbucks and Amazon, and end up discussing how the two companies are moving toward the same model: a hybrid of digital sales and brick-and-mortar retail operations. For example, Starbucks announced several new tech-driven features at its annual shareholders meeting this week, including an Alexa-powered ordering… Read More
  • 1 day ago

Charter promises Trump a broadband push, but no extra Internet connections

Charter's $25 billion promise is vague and includes stuff it already planned.
  • 1 day ago

Geek of the Week: Data-driven CEO Omri Kohl aims to transform businesses at Pyramid Analytics

Omri Kohl started his first business — making and delivering sandwiches — while he was in college. Within a year, the business funded his education. For more than 20 years, Kohl, who calls himself “an entrepreneur at heart,” has been building startups and “turning them into successful, industry leading companies.” He’s now the co-founder and CEO of Pyramid Analytics, a Bellevue, Wash.-based business-intelligence technology company — and he’s GeekWire’s latest Geek of the Week. Seeing how pervasive data was across every organization, Kohl said he and two colleagues formed what became Pyramid in 2008. Their vision was to create a fully-featured… Read More
  • 1 day ago

Mobile ad developer claims in lawsuit it was cut out of Amazon smartphone deal by partner

An expansion of Amazon’s discounted smartphone program for Prime members is at the center of a lawsuit filed in Los Angeles last week. Pay Per Swipe, a Los Angeles-based mobile advertising company that builds apps to put ads on smartphone lock screens, sued TCT Mobile, an entity that shares an Irvine, Calif., address with phone maker Alcatel, for breach of contract and unfair business practices, among other charges. Pay Per Swipe alleges that TCT broke a non-disclosure agreement and used confidential information about lock screen ad products the two firms worked on together for its own gain. TCT allegedly used that information… Read More
  • 1 day ago