Google’s advances in Augmented Reality (“AR”) and AI/ Machine Learning/ Deep Learning not only enable user productivity gains, but they facilitate Google’s ability to collect personal data. These “convenience tools” in most cases require that we trade data privacy for productivity gains. To borrow a phrase from one of Google’s execs – “Google initially indexed the Internet, Google is now indexing the real world.” Further, many of these features carry an extremely small footprint. Only 100 KB for the advanced translation feature covered in this video. Perhaps most impressively from a footprint standpoint is the fact that the next generation Google Assistant will see the 100 GB deep learning models shrink to 0.5 GB. Yes, less than a single GB enabling Assistant to run on-device. In many cases these advanced features will fit on very inexpensive phones ($35) commonly found in low income, densely populated areas of the world, effectively arming users with the world’s most sophisticated data collection software platform (Good news for Google). This particular video segment covers 1.) the integration of 3D models into Search; 2.) the integration of Augmented Reality into Search and real-world use cases; 3.) the integration of text-to-speech, computer vision, and natural language processing to address everyday use cases.

Android Q: Live Caption (on-device speech recognition). Live caption operates entirely on-device. The demo in this video clip was executed while the phone was in airplane mode. Google reduced the footprint of its recurrent neural net (“RNN”) NLP model from 2GB to 80MB and therefore is able to execute this workflow entirely on-device.