Google Showcases On-Device Artificial Intelligence Breakthroughs At I/O

Partners expect Google's advances in local processing of machine-learning models, first coming to consumer products, to ultimately present a boon to enterprise Internet of Things practices.

Google's I/O developer conference often introduces cutting-edge technologies that find their way into the enterprise, and partners saw that potential Tuesday in demonstrations of breakthroughs in localized artificial intelligence.

A major focus of the opening-day keynote was embedding the ability to train and execute machine-learning models directly on remote devices, which reduces latency and protects data privacy.

That capability stems from advances in exponentially shrinking the size of models that only recently were achieved by Google's AI researchers, Google CEO Sundar Pichai told I/O attendees gathered near the company's headquarters in Mountain View, Calif.

[Related: Google I/O 2019: Four Big Mobile Announcements]

Bringing machine-learning models directly into a smartphone creates "a data center in your pocket," Pichai said.

While realizing such technology poses "an incredibly challenging computer science project," Pichai said, Google has now hit major milestones, having shrunk 100-GB models to half a gigabyte—small enough to run on a mobile device.

Google showcased several use cases for the technology soon to be integrated across its spectrum of Android and Nest consumer products, delivering capabilities like live-captioning of spoken text, voice navigation of apps, and image recognition with almost zero delay.

Reveals like those are why Google I/O has become a destination for so many of Google's enterprise partners, Kevin McMahon, executive director of the technology innovation group at SPR, a Google Cloud consultancy based in Chicago, told CRN.

"From an enterprise perspective, a lot of this stuff is developed for Google, but they're essentially dogfooding for bringing it out to enterprises at some point on their cloud platform and helping share the technology with others as they learn and develop and iterate around it," McMahon said. That "helps partners succeed."

I/O is a conference that allows enterprise developers to directly "talk to the people who wrote the code your livelihood essentially depends on," he said.

The AI-on-device breakthroughs that took center stage bode well for Google's channel, especially those partners delivering Internet of Things solutions to the enterprise.

"There are many use cases that require pushing AI to the edge," said Alex Gorbachev, Cloud CTO of Pythian, an Ottawa-based Google partner specializing in data analytics.

Pythian is working on such a solution for an industrial client that requires real-time processing of billions of data points per second on the factory floor, then crunching machine-learning models at least 20 times per second.

"It's just not feasible to transfer such volume into the cloud for processing," Gorbachev said. That's why Google's "on-prem cloud" advances like Edge TPU and Edge IoT, and now condensed models for local execution, present a unique path forward.

Many new industrial AI applications have such requirements, Gorbachev said.

Another advantage of Google shrinking machine-learning models to the size of devices is the ability to avoid moving data off them.

"In the age of increased demand for user privacy, effective application of ML models on devices is a great way to ensure data privacy," Gorbachev said.

That can mean classifying images on photos snapped on Android devices, or transcribing audio to text, or translating between languages—all without sending personal or corporate data to the cloud.

"More advanced implementations even allow distributed model training across many devices and centralizing training iterations into one model while never sharing the private training data," Gorbachev told CRN.

Stephanie Cuthbertson, Google's senior director for Android, unveiled Android Q during the keynote, the 10th version of the leading mobile OS, which will include "live caption" capability.

Live captioning comes from "a huge breakthrough" in speech recognition made earlier this year that for the first time makes it possible to convert voice to text without streaming to the cloud.

Pichai also discussed progress on the development of Federated Learning—a technology that allows Google's products to run AI locally but still update and benefit from a global model.

The bleeding-edge technology ships machine-learning models directly to a device, then sends only the model updates to the cloud—not the data itself, Google's CEO explained.

It's "still very early," Pichai said, but Google believes the progress it has already achieved creates potential for Federated Learning to be deployed in many of its coming products.

By focusing at I/O on technologies it's been researching for years like Federated Learning, Google made very clear that AI and machine learning is "where they're placing their bets," SPR’s McMahon said. "It's got its stamp everywhere here."

The true test of a mobile app is how it works in an offline state—able to execute a workflow that relies on processing without bouncing back to the cloud.

In improving that capability, Google "opens up a whole set of use cases here," McMahon said, solving concerns around latency and privacy in an era where tech companies are increasingly under fire for their data practices.

"It's a tough market for trust right now. That has enterprise implications too," McMahon said. "To unlock a lot of these cool features, enable businesses to leverage these capabilities, they’ve got to have a good answer to that or else the market is going to reject this because of fears of the unknown."

Google executives said a lot of the right things on the data privacy front Tuesday, though "proof is always in the pudding, so we'll see how it plays out over the next year," McMahon told CRN.

Google also used I/O as a forum to unveil upgrades to its Maps product, including an Incognito mode similar to the Chrome browser capability that protects user privacy, and an augmented reality feature that gives directions within a streetscape view.

For Los Angeles-based SADA Systems, Google's largest Maps reseller, those features carry the potential to enhance its booming geospatial enterprise practice.

"Any major upgrades to consumer Google Maps services, such as augmented reality support, will likely mean similar feature enhancements are on the horizon for the Google Maps APIs," said Jacob Youshia, SADA's director of business development for geospatial solutions.

Upgrades like those will enable SADA and other Google partners to deliver to customers "a wider array of forward-looking use cases and services," Youshia told CRN.