AI Smart glasses - the new wearable assistant for daily life

The second innovation is the development of AI agents that are rapidly improving their ability to engage in natural-language interactions with users, and to operate intelligently in the context of the user's life (for instance, understanding instructions such as 'Message the friends who I'm meeting tonight to say that I will be wearing the red dress that I bought on Vinted last week').

Manufacturers are learning that smart glasses offer a convenient and enjoyable way to interact with these agents in everyday life. No longer does the user have to take a phone out of their pocket, wake it up, and tap buttons to access the agent: smart glasses are ready and available all the time and, crucially, can see what the user sees.

Smart glasses, then, are emerging as a valuable complement to AI agents, providing a way for them to be incorporated into daily life in a seamless way. With advanced audio, video and sensor capabilities built in, smart glasses can also help to make core human activities - conversing, exploring, learning - more rewarding and enjoyable.

To succeed in this new market, manufacturers will need to draw on optical technologies for illuminating and sensing the visible and non-visible worlds - technologies in which ams OSRAM is continually advancing the state of the art.

Future Smart Glasses: Extending Human Senses

To understand how this works in practice, we can break down smart glass operations into three core categories: Input, Context Sensing and Output.

Input: Smart glasses respond naturally to voice commands, hand gestures, and tactile interaction through sound, motion, and touch sensors. Input can also come from companion devices like wristbands or rings that enable gesture control without touching the glasses.
Context Sensing: Sensors continuously read information about your environment and yourself. By seeing what you see, hearing your surroundings, tracking your gaze and gestures, and even understanding your facial expression, the glasses can better interpret your input and respond naturally to your life. Additional sensors improve camera quality, detect indoor/outdoor location, and monitor how environmental light affects health and mood.

Context Sensing: Sensors continuously read information about your environment and yourself. By seeing what you see, hearing your surroundings, tracking your gaze and gestures, and even understanding your facial expression, the glasses can better interpret your input and respond naturally to your life. Additional sensors improve camera quality, detect indoor/outdoor location, and monitor how environmental light affects health and mood.

Output: Smart AI glasses will provide responses to the wearer in a non-intrusive way. Outputs can be provided via miniature speakers embedded in the frames, LEDs in the peripheral vision for status cues, and advanced display systems - like micro-projectors that display on transparent lenses - that overlay visuals and 3D objects onto the real world for augmented reality experiences.

Together, these elements provide powerful capabilities that enable compelling use cases that will drive further adoption.

• Content Capture: Taking pictures, shooting video, streaming audio. A tamper-proof indicator light provides critical privacy protection by showing when the camera is recording.
• Real-time Translation: Provides real-time translation of foreign language text or speech.
• Hearing Augmentation: Amplification of the voice of the person that the wearer is looking at, while muffling other ambient noise - helping people to have conversations in a noisy environment.
• Health & Wellness: Monitoring the wearer's health, and the effect of the environment on their well-being - for instance, detecting excessive exposure to sunlight that could pose a risk of sunburn, or measuring the effect of ambient light on the wearer's circadian rhythm.
• AI Assistant: The glasses can understand the situation you are in and can help you to retrieve information when you need it, for example "Where did I leave my keys?", "How many eggs are left in the fridge?" or "Reserve a table in my favorite restaurant". Over time it will learn to adapt to your needs to provide more nuanced assistance.

AI data processing is at the core of all these AI smart glass functionalities, since it enables glasses to 'understand' text, video and audio inputs, and to put them in context with previous inputs. Optical sensors serve as the AI's interface to the world.

The challenge for these sensors is to provide high responsiveness to light while meeting the strict size and industrial design constraints of the smart glasses form factor. The components selected for integration into smart glasses will have to fit into thin, light frames. Since optical sensors often need to face the outside, they need to be obscured to not disturb the design language of the glasses. Ultra-low power consumption is another key requirement, as smart glasses will only include a small battery, and users will expect a minimum of a full day's wear between battery charges.

Glasses which look where the eye looks

Dr. Tim Böscke

Dr. Tim Böscke is Director at ams OSRAM International in Regensburg, leading the Mobile, Wearable & Computing segment within Application Marketing & Architecture. He drives cross-divisional consumer strategies focused on AR/VR, smart glasses, wearables, computing, and other devices. With extensive semiconductor and optoelectronics experience, he previously led System Architecture and Sensor Development at OSRAM Opto Semiconductors, where he was instrumental in defining the company's activities in vital sign monitoring and AR/VR sensing. Dr. Böscke holds a PhD in Electrical Engineering from Technical University of Hamburg and is a prolific inventor with more than 60 patent families.