Northwestern tech helps power Apple Vision Pro wheelchair-control feature

• For Journalists
• News Releases
• Latest Releases
• News Release

Northwestern tech helps power Apple Vision Pro wheelchair-control feature

New accessibility feature enables users to drive powered wheelchairs with their eyes

• Software developed by Northwestern-led project included in Vision Pro's new accessibility features
• Software helps translate eye movements into signals that powered wheelchair control systems can understand
• Technology could help users with severe mobility impairments who cannot use hand-operated controls like joysticks

EVANSTON, Ill. - A simple glance now can help drive a wheelchair.

Apple recently unveiled a new Vision Pro accessibility feature that allows users to control powered wheelchairs using only their eyes. Northwestern University technology helps make it possible.

The feature uses the Vision Pro headset's advanced eye-tracking system to send driving commands to compatible wheelchairs. Software developed by Northwestern's Project DRIVE acts as a bridge between the headset and wheelchair's control system, translating eye movements into signals the wheelchair can understand.

Together, the system and software enable users to bypass traditional hand-operated controls to navigate, steer and stop their wheelchair with their eyes.

The innovation is part of a broader suite of accessibility updates, which Apple will launch in September.

"Often, as a person's motor impairment increases in severity, their ability to operate the very machines designed to assist them decreases," said Northwestern's Brenna Argall, the principal investigator of Project DRIVE. "Many people with severe mobility impairments cannot reliably use traditional manual controls like joysticks. This technology creates another pathway for independent mobility by allowing users to interact with and control their wheelchair through natural eye movements."

An expert in assistive machines, Argall is a professor of computer science and mechanical engineering at Northwestern's McCormick School of Engineering and a professor of physical medicine & rehabilitation at Northwestern University Feinberg School of Medicine. She also is a research scientist at Shirley Ryan AbilityLab, where she directs the Assistive & Rehabilitation Robotics Lab (argallab).

Supported with funding by the U.S. National Science Foundation's Directorate for Technology, Innovation and Partnerships (NSF TIP), Project DRIVE aims to make powered wheelchairs safer and more accessible for people with severe mobility impairments. With Project DRIVE, Argall and her team develop software and interfaces that allow alternative input systems - such as eye tracking, head movements, sip-and-puff controls, switches, brain-computer interfaces or other assistive technologies - to communicate with wheelchair drive systems.

Since its launch in 2023, Project DRIVE aimed to develop an open Wheelchair Digital Interface (WDI), so new input methods could communicate with wheelchair control systems. Rather than developing a wheelchair-control system from scratch, Apple leveraged this technology.

A multidisciplinary effort, Project DRIVE unites academic, clinical, nonprofit and industry partners. LUCI, which brings smart technologies to powered wheelchairs, is among those partners. LifeDrive Mobility, LLC, an alternative controls partner, helped expand and validate the WDI. Apple's Vision Pro wheelchair-control feature also integrates with LUCI-enabled wheelchairs, which use sensors to detect dynamic obstacles and sudden drop-offs.

"For years, Project DRIVE focused on developing the technology infrastructure needed to connect emerging assistive technologies with powered wheelchairs," Argall said. "Seeing that work become part of Apple's platform demonstrates how academic research, stakeholder collaborations, and industry partnerships can come together to expand independence for people with disabilities."

Brenna Argall

Professor of Computer Science, Mechanical Engineering, Physical Medicine and Rehabilitation

• [email protected]