On-device AI critical to expanding generative AI

By Kwon Oh-hyung

Artificial intelligence (AI) is coming closer to our everyday lives. If you're using your smartphone to read this article, chances are it is using AI technology to make your on-device experience better, from connectivity to graphics and extending further into other features such as improved photography, automatic subtitles and more. It makes our experiences on smart connected devices more intuitive and personalized and our days more productive.

Kwon Oh-hyung, senior vice president and president of Asia-Pacific at Qualcomm Technologies

With generative AI, your smartphone can also become a true digital assistant. PC users can use it to read or compose emails, draft documents and create presentations automatically. AI-driven cockpits will allow the driver and passengers to more naturally interact with and control entertainment, navigation and vehicle functions to create more a personalized and intuitive experience.

Up until now, the AI discussion has centered on the cloud. ChatGPT uses huge amounts of processing, storage and memory. Much of the focus so far has been on training and refining new models, which generally take place in the cloud.

As processing requirements, use cases and users of generative AI increase, on-device AI will play an essential role in allowing generative AI to scale across applications and industries, and expand to more devices at the edge. Enabling generative AI on a device will help accelerate adoption everywhere, minimizing the rising costs of cloud infrastructure.

Also, the difference in energy costs is significant as well as helping cloud providers offload data center energy consumption to meet their environmental and sustainability goals from environmental, social and corporate governance (ESG) perspective.

Stable Diffusion is able to generate realistic images of concepts based on text inputs. Demonstrations at MWC 2023 have shown that it can run on devices, rather than only in the cloud. Integrating intelligence on devices is critical to realizing the full potential of mobile experiences, providing greater immediacy, reliability, personalization, privacy and affordability. On-device AI inherently helps protect users' privacy since queries and personal information remain solely on the device.

To address the given challenges, future generative AI needs to take the on-device AI approach to scale. Such architecture provides benefits with regard to cost, energy, performance, privacy and security. It is applicable to virtually all generative AI applications and device segments ― including phones, laptops, XR headsets, vehicles and the Internet of Things ― enabling them to reach their full potential and meet enterprise and consumer needs globally.

Kwon Oh-hyung is senior vice president and president of Asia-Pacific at Qualcomm Technologies.