AWS Machine Learning Game Gets A Boost With New Services, Custom-Designed ML Chip

Amazon wants to help more businesses inject machine-learning technology into their processes, regardless of where their abilities fall on the machine learning spectrum.

The Seattle-based cloud giant announced a series of new machine learning capabilities and services, as well as a custom-designed chip specifically designed for all layers in the machine learning stack on Wednesday at AWS re:Invent 2018.

"Necessity is the mother of invention," AWS CEO Andy Jassy told a packed audience during his keynote. "There is a lot of machine learning being done on the cloud, and most of it is happening on AWS. There is real thirst and hunger to enable customers to adopt machine learning more easily and that’s just what we want to do."

[Related: AWS CEO Andy Jassy To Partners: 'There Are So Many Customers That Need Your Help']

First out of the gate, Jassy unveiled Amazon Elastic Inference, a new service that allows developers to dramatically decrease inference costs with up to 75 percent savings when compared to the cost of using a dedicated GPU instance.

Inference, not training, is the most costly part of running machine learning in production, Jassy said. Rather than run on a whole Amazon EC2 P2 or P3 instance with low utilization, developers can run on a smaller, general-purpose Amazon EC2 instance and provision the right amount of GPU performance from Amazon Elastic Inference and only pay for what they use, he stressed. Elastic Inference, available now, is integrated with Amazon SageMaker and the Amazon EC2 Deep Learning Amazon Machine Image (AMI).

AWS Inferentia is AWS' brand-new , high performance machine learning inference chip. Inferentia has been custom-designed by Amazon for larger workloads that consume entire GPUs or require lower latency. The chip, available in 2019, can provide hundreds of teraflops per chip and thousands of teraflops per Amazon EC2 instance for multiple frameworks, including TensorFlow, Apache MXNet, and PyTorch, and multiple data types, according to Amazon.

The new chip sends the message that if the market doesn't have what Amazon needs, the cloud giant will build it themselves, said Gene Villeneuve, senior vice president of Tehama.io, a business unit of Pythian, an Advanced Consulting Partner with AWS.

"I think it is fascinating to see AWS getting into the chip/ processor business with the AWS Inferentia ML optimized chip," he said. "I think this announcement will get a lot of buzz going at IBM and Intel."

Amazon is building on its machine learning platform it revealed last year during re:Invent, SageMaker, with the announcement of SageMaker Ground Truth. Available now, the latest service lets developers produce high-quality labeled training data, no human annotation necessary. The service learns from annotations in real-time and can automatically apply labels to much of the remaining dataset, which then reduces the need for human review, according to Amazon. Jassy said that the new offering will not only save developers time, but it can reduce costs by up to up to 70 percent when compared to human annotation.

Also launched was SageMaker RL, a fully managed service for reinforcement learning algorithms and simulators. Reinforcement learning is a technology that can train models without large amounts of training data. Available today, Jassy revealed Amazon's new Machine Learning Marketplace offers more than 150 new models and algorithms to developers through Amazon SageMaker.

Higher up the stack, Amazon is launching three new services, in preview today, that will inject more intelligence into applications, including Amazon textract, Amazon Personalize, and Amazon Forecast.

Amazon Textract tackles the problem of extracting data from documents and forms through manual data entry or using simple optical character recognition (OCR) software, both of which are slow an expensive process, Jassy said. Textract uses machine learning to instantly read virtually any type of document to accurately extract text and data without the need for any manual review or custom code. The service can process millions of document pages in several hours, Amazon said.

Based on the same technology that powers Amazon.com, Amazon Personalize is a real-time recommendation and personalization managed service that businesses can use to build, train, and deploy custom, private personalization and recommendation models for any use case, such as personalized search results or email marketing and push notifications.

Similar to Personalize, Amazon Forecast is also based on technology developed by Amazon.com. The service is geared toward helping customers predict future trends in supply chain, inventory levels, and product demand, based on historical and causal data, such as seasonal demands, Jassy said. Amazon Forecast can generate and deploy custom, private machine learning forecasting models up to 50 percent more accurate at one-tenth of the cost of supply chain software, he said.

Logicworks, a Premier Amazon consulting partner, will be taking advantage of Amazon's new Machine learning features and functionality.

"There's a lot of customer demand in the machine learning arena, so it's great to see that innovation. The new enhancements will be great for us and our customers," said Marilyn Daly, vice president of marketing for Logicworks, a Premier Amazon consulting partner.

The ability to build machine learning functions into services, no machine learning experience necessary, is a huge opportunity for partners, Terry Wise, global vice president of channels and alliances, told CRN.

"The more packaged machine learning is and the more they can drive business outcomes, the higher the value is that partners can provide to their customers," Wise said. Think about a partner building a new forecasting tool for an insurance company -- that's pretty cool and something that's repeatable."

Jassy said that within the next decade, he believes that virtually every application will have AI and machine learning infused into it, so Amazon has no plans of slowing down in this arena.

"I think this is going to be a gigantic area," he said. "You can expect that we're not close to being done."