How Amazon teaches Alexa, and what it hopes the virtual assistant will learn someday


Amazon senior principal scientist Nikko Strom at the AI NEXT tech conference. (GeekWire Photo / Geof Wheelwright)

Days after Amazon announced that it was bringing its Alexa to its iOS Amazon app, Amazon senior principal scientist Nikko Strom spoke at the AI NEXT tech conference in Bellevue, Wash., this weekend to share behind-the-scenes details of the company’s voice-enabled assistant and its broader artificial intelligence initiatives.

Strom, a founding member of the team that built Amazon Echo and Alexa, told the audience of AI scientists that the growing number of Alexa-based devices (not publicly disclosed by Amazon but estimated by Consumer Intelligence Research Partners at more than 8 million) has provided Amazon with a significant amount of data to use in improving and refining Alexa-powered devices.

“All of these things that make Alexa great and expanding all the time means that we get lots of data,” he said. “One of the things about this era is that people actually like using these (devices) — I’ve been in the industry for a long time and I worked on telephony systems and people didn’t really like to use them.”

Strom compared the amount of data that Amazon received from the millions of Alexa devices to what a 16-year-old might have heard during their young life. He said that in 16 years, a person might hear — and have “training data” about — as much as 14,016 hours of speech (based on the assumption that about 10 percent of what a person hears in a day is speech).

Amazon uses “large-scale distributed training” to analyze the voice data it gets from users Alexa-enabled devices in order to improve their speed and accuracy.

“We have all this data – we have thousands of hours of stored data from our customers in Amazon S3 (Amazon Simple Storage Service) and we train these models on AWS EC2 (Amazon Web Services Elastic Compute Cloud) instances,” he said, explaining that the company has to use “distributed training” across 80 GPU (graphical processing unit) instances in order to crunch the massive amount of data it receives.

This large-scale distributed training of the voice recognition model in Alexa allows Amazon to constantly make updates to accuracy and quality.

Strom also took time to address concerns about what, when and how Amazon collects voice data — and stressed that the company is only interested in the voice data necessary to run its services, not in the content of anyone’s conversations.

The issue recently came to public attention in an Arkansas murder case where Bentonville police issued a warrant demanding records for an Echo device belonging to a charged murder suspect. The case has prompted debate about how First Amendment rights should be protected when speech is stored on digital devices. In reply to a general question about the Alexa technology and privacy, Strom suggested that Alexa’s handling of voice data is not always entirely understood from people who read about it in the press.

“What people don’t always get in these articles is that it (Alexa) is listening for a wake word all the time. It is only listening for the wake word,” explained Strom. “It’s only when the blue ring starts spinning — that’s when Alexa has heard the wake word and starts recording you. Only the thing you say after that wake word is ever recorded.”

Alexa has branched out from Amazon’s Echo and Fire TV devices into a growing number of third-party products. (Amazon Photo)

The range of Alexa devices is also growing, with the technology now in use on smartphones, cars and refrigerators. In addition, Strom said that the number of third party “skills” (voice-activated apps enabled for use with Alexa) is growing so fast that it’s hard to keep up with. Amazon recently said Alexa surpassed 10,000 skills.

“Skills are super-exciting, but they are also a big challenge for us because there’s many of them and we don’t build them ourselves,” he said, as he frankly discussed the need to maintain strong communications with skills developers.

Finally, Strom hinted about what it might take to make Alexa a little smarter — so that it understands what someone means, not just what they say. To do that, Alexa would have to tackle emotions and intonation.

“Alexa cannot capture the emotion in your speech right now, but it can do something indirectly by capturing the meaning of what you say — which can be emotional,” he said. “It will recognize your curse words, for example. We have over 100 scientists working on Alexa on speech in general.”

Strom said the company didn’t yet have anything to announce on emotion recognition, but that it would continue to be an area of interest.

Amazon is enjoying strong sales for its Alexa-enabled devices – and a recent analysts from RBC Capital Markets recently estimated that sales of Alexa devices could hit $5 billion by 2020 (with another $5 billion in annual revenues predicted to come from shopping done via the voice assistant). But it is not alone in the market. Microsoft’s Cortana (included with Windows 10 and available on iOS and Android devices), Google’s Google Home and Google Assistant (which is pre-loaded on Android smartphones) are still strong competitors.

Top Stories

Q&A with Topgolf’s CEO about the popular sports entertainment company and a possible Seattle location

Golf isn’t the most popular sport, but Topgolf is certainly doing its part to change that perception. The growing sports entertainment company operates more than 30 driving-range-meets-bowling-alley entertainment facilities — with 11 more soon to open — that feature 6-person hitting bays with microchipped trackable golf balls, along with food, drink, TV screens, free WiFi, and much more. Technology is a key part of the Topgolf experience — from the balls with RFID chips to the Protracer cameras — and that’s why we’re excited to hear from Topgolf co-chairman and CEO Erik Anderson at the GeekWire Sports Tech Summit on June 21-22 in Seattle at CenturyLink Field. Anderson, who… Read More
  • 3 days ago
  • 14

GeekWire Deals: Tackle your business plan with the simple, visual Roadmap Planner

Running a business is complicated. When you’re at the helm of strategy, finances, and vision, it can feel like you’re spending more time planning than actually doing. Tasks like this require serious organization, and that’s exactly what today’s GeekWire Deals offer will provide. It’s time to upgrade from a mess of Post-it notes and recruit Roadmap Planner. This project management mastermind brings everything into one easy-to-navigate digital calendar, with all the tagging and color-coding you need to stay on-track. Visualize your current state, your future goals, and all the progress you make along the way. Share easily with team members… Read More
  • 3 days ago
  • 12

Echodyne gets a $29M boost from NEA, Bill Gates and others for low-cost radar

Microsoft co-founders Bill Gates and Paul Allen are among the investors putting another $29 million into Echodyne, the Intellectual Ventures spin-out that’s developing low-cost, miniaturized radar systems for drones and self-driving cars. Echodyne founder and CEO Eben Frankenberg said the Series B funding round was led by New Enterprise Associates, or NEA, a Silicon Valley venture capital firm. Gates, Seattle’s Madrona Venture Group, the Kresge Foundation and Allen’s Vulcan Capital are among the investors following up on their participation in 2014’s $15 million Series A round, Frankenberg told GeekWire. He declined to say how the new investment affects the valuation of the… Read More
  • 2 days ago
  • 11

How Jeff Bezos’ passion for space is inspiring the next generation of entrepreneurs and explorers

When Amazon CEO Jeff Bezos visited the Museum of Flight this weekend to answer questions from students, the kids did not hold back. “That’s one of the great things about kids,” Bezos said on Saturday. “There are always questions.” Scores of elementary-school and middle-school students came from the Seattle area as well as from Deer Park, a city just north of Spokane on the other side of the state, to cram into the museum’s “Apollo” exhibit and meet America’s second-richest person (after Bill Gates). The kids asked about Bezos’ successful expedition to recover sunken rocket engines from the Apollo moon missions, about his… Read More
  • 3 days ago
  • 11

HP gives Envy laptops an edgier look, updates Spectre x2 with Kaby Lake CPUs

Sharper edges, new kickstands, Intel and AMD processors, and more.
  • 2 days ago
  • 10
farawayyachtingcharters - Easy Branches
botoxfillerveintheraphyinphuket - Easy Branches

Latest in Technology

Nerd Nite meets NASA, sparking memories for an astronaut with deep Northwest roots

The High Dive is known for drawing crowds into the local music scene, but this week, music wasn’t what drew nearly 200 people to the venue – it was NASA. Nerd Nite Seattle is a monthly gathering at the bar in the city’s Fremont neighborhood, featuring beer, tasty Mexican food, science talks and, of course, nerdy Seattleites. Tuesday night’s event drew in plenty of the regulars, plus an assortment of first-timers. One of the first-timers was NASA astronaut Anne McClain. She mingled with the crowd, and then got on stage to speak about her rigorous astronaut training, share hilarious stories about… Read More
  • 30 minutes ago

What’s it take to become a Seattle local? As city changes, website questions natives and transplants

A week or so ago, before Seattle decided to play summer for a few days, I grilled dinner in a torrential downpour, and it felt … normal. There was no attempt to prove a point about being a seasoned Northwesterner — there was just a plan in place to grill that night and people needed to eat. I don’t know, after 21 years in Seattle, whether that anecdote makes me more of a “local,” but it does illustrate, I think, an acceptance of what it means to live here. And it’s a point among many that are raised in a fun… Read More
  • 1 hour ago

Appeals court sides with Amazon in dispute over counterfeit pillowcases

A federal appeals court held up a 2015 decision in favor of Amazon this week in a dispute over counterfeit goods sold by a third-party retailer on the e-commerce giant’s site. It is a big win for Amazon as its third-party marketplace has become an important driver in the company’s bottom line in recent years. Third-party sellers on Amazon accounted for 49 percent of all paid units in 2016, the company said at its annual shareholder meeting this week. The potentially landmark case was filed by Seattle-based novelty pillowcase maker Milo & Gabby in 2013. The boutique business said a Chinese manufacturer copied their designs,… Read More
  • 2 hours ago

Geared Up: Microsoft’s new Surface Pro explained, and Nintendo takes a summer road trip

Microsoft continued its stream of Surface announcements this week with the new Surface Pro — notably not called the Surface Pro 5, even though it’s a successor to the Surface Pro 4. The company is billing it as a fast and powerful alternative to devices like the iPad Pro and is avoiding the “tablet” label by calling it a “versatile laptop.” Microsoft said the new device will feature up to 13.5 hours of battery life and the “fastest digital pen on the planet,” although it’s sold separately. It’s also shunning USB C ports, which Apple has been using on many of its… Read More
  • 2 hours ago

Ikea’s smart light bulbs will work with Amazon Alexa, Apple Siri and Google Assistant

 Ikea’s smart light bulbs will answer to voice commands starting this summer. The company announced in a Swedish-language press release that its smart lighting product line, called Trådfri, will work with Amazon Alexa, Google Assistant, and Apple Siri and HomeKit. Read More
  • 3 hours ago

Amazon adds shoppable stickers to its iOS app

 Using stickers to decorate your photos is a common feature found in social apps like Snapchat, Instagram, Facebook, Twitter, and others. But today, stickers have started showing up in a more surprising place: Amazon’s mobile app. Following a recent update, Amazon has added support for stickers within its iOS app’s camera feed – the feature that allows you to search for… Read More
  • 3 hours ago

Zillow announces $1M prize for anyone who can improve the algorithm for its Zestimate

The Zillow Zestimate, that much-debated, computer-generated home valuation tool, has been a fixture of the Seattle-based real estate media company for 11 years. And now Zillow is putting a price tag on whether the algorithm that assigns a price tag can be improved upon. Zillow announced Wednesday that it was launching a competition aimed at awarding $1 million to the first person or team who could most improve the Zestimate algorithm. “Zillow Prize” hopes to attract data scientists everywhere to try their hand at tinkering with “one of the highest-profile, most accurate and sophisticated examples of machine learning.” Zillow consistently… Read More
  • 3 hours ago

IBM, Google, and Lyft team up on open-source project for more control over microservices

After years of evangelizing microservices as the path to a more nimble development organization, companies are realizing they need to make it easier to keep track of all those moving parts. IBM, Google, and Lyft released a new open-source project Wednesday to give developers better control over their microservices projects. Istio is the name of the joint project, and it’s designed to give “developers and devops fine-grained visibility and control over traffic without requiring any changes to application code,” Google’s Valrun Talwar wrote in a blog post. It runs on Kubernetes, but will eventually reach other platforms such as Cloud… Read More
  • 3 hours ago

Microsoft is worth more than Dallas: Analyst warns of destabilization as tech giants grow more valuable than cities

The world’s largest tech companies are worth more than the gross domestic product (GDP) of most U.S. cities, according to new research from Bank of America Merrill Lynch. The report, titled “Occupy Silicon Valley,” compares the market caps of tech giants to the GDPs of the top U.S. cities. By that metric, Google and Apple are worth more than all U.S. cities except Los Angeles and New York (those with the largest economies). Microsoft is outpaced by just L.A., New York, and Chicago. Amazon’s value is greater than the GDP of Washington, D.C. What’s more, the combined market cap of Google… Read More
  • 3 hours ago

How technology could change the DMV: mobile driver’s licenses, cars that issue speeding tickets and more

Technology has disrupted and changed a variety of industries and experiences, but most wouldn’t put a trip to the DMV in the category of modernized experiences. But the industry is working on a variety of changes, focused on big data, mobile technology and bringing documents online that could speed up the process of getting or renewing a driver’s license. This week, experts focused on everything from law enforcement, to licensing to vehicle titles gathered in Seattle for a conference put on by the American Association of Motor Vehicle Association. At the event, there were several panels focused on improving technology in the… Read More
  • 3 hours ago

Boeing wins DARPA’s go-ahead to build and test XS-1 military space plane by 2020

The Pentagon’s Defense Advanced Research Projects Agency says it has selected the Boeing Co. to develop and test its XS-1 military space plane, using a design now known as the Phantom Express. Boeing won out over Northrop Grumman and Masten Space Systems to take the long-running XS-1 project beyond Phase 1 into Phase 2 and 3, culminating with test flights in 2020. The reusable space plane will be designed to carry and deploy a small, expendable upper stage, then return to Earth for a runway landing. The upper stage would be capable of putting satellites weighing up to 3,000 pounds in… Read More
  • 3 hours ago

Report: ARM-owner Softbank takes $4B stake in Nvidia as AI research heats up

Softbank is shaping up to be a major force behind the chip designs of the next decade, if a report concerning its stake in Nvidia is correct. A little less than a year after it bought mobile chip designer ARM for $35 billion, Softbank has slowly amassed a $4 billion stake in Nvidia, according to Bloomberg. The investment is part of a $100 billion fund that Softbank founder Masayoshi Son wants to invest in technology companies, and is just shy of the threshold that would require Softbank to disclose the stake. Nvidia is currently considered the leading maker of chips… Read More
  • 3 hours ago