Research Spotlight: The “Body Brain Connection” at the CSU Research Competition

For some individuals with severe to profound hearing loss, cochlear implants offer sensory support that can improve hearing and communication skills. Small electronic devices that must be inserted surgically into the skull, cochlear implants, can offer some relief, though they come with a significant catch: Not all people with them can undergo magnetic resonance imaging (MRI) due to the internal magnetic components. MRI is a valuable diagnostic tool used to detect a variety of health conditions and measure attention.

That's why audiology doctoral student Allison Tran has partnered with Jonathan Chiu, '27 MS Human Factors, and Ayush Sunil Gawai, '27 MS Applied Data Science, on "Body Brain Connect: Visual Attention in Human-Computer Interaction via Eye and Brain Activity," an interdisciplinary research project that they are presenting at the 40th Annual California State University Student Research Competition . This prestigious, three-day statewide event showcases outstanding research and other scholarly, and creative work by undergraduate and graduate students from all academic disciplines across the CSU's 22 campuses.

The three graduate students were brought together by their mentors, Assistant Professor of Applied Data Science Guannan Liu, Assistant Professor of Industrial and Systems Engineering Yue Luo and Assistant Professor of Audiology Yi Yuan - all experts who recognized the value of interdisciplinary research. It didn't take long for Tran, Chiu and Gawai to brainstorm a research question that appealed to all of their interests.

"The main purpose of our project is to see if we can use eye tracking as an objective way to measure levels of attention," says Chiu. "This extends to a wide variety of applications from cochlear implants in the audiology sphere to the development of human centered AI in the engineering world."

Tran adds that most measures of attention rely on self-reported surveys and questionnaires, which can be subjective to how someone experiences an event, situation or an assignment. Instead, the Brain Body Connect team is focusing on biological physical reactions by combining eye-tracking with fNIRS, or functional near-infrared spectroscopy, a noninvasive tool that measures brain activity.

"The fNIRS is a cap that goes over your head and uses infrared light to measure oxygenation levels in the brain," says Tran, '28 AuD. "The theory is that regions of the brain that are activated by a task will have more, or need more oxygen. That's why we're looking at hemoglobin levels at rest versus during an activity."

Chiu explains that their tests, which last between 30-45 minutes, ask participants to complete simple visual tasks based on attention. Some exercises are global attention tasks that involve scanning an entire image, while others are selective attention tasks that isolate different features in a scene. The third task is sustained attention, where participants must use their short-term memory to recall a former image and compare that to a present one. All of these tasks are completed while the participants wear eye trackers and head cap to measure activity.

As the team data analyst, Gawai gathers, analyzes and creates visuals using data from the fNIRS and eyewear. He creates heatmaps of gazing views and attention trajectories, performs quantitative analysis to evaluate differences across tasks,interprets attention behavior from the data and has built an AI pipeline that analyzes the entire process.

"This project has worked out so well so far that we keep improving it," Gawai says. "We're planning to add more subjects to our study, and we're going to make the AI pipeline work more autonomously to support the data analysis process. "

The research team first presented a ten-minute explanation of their work at the SJSU Research Competition earlier this semester. They were selected as finalists, qualifying them for the CSU Student Research Competition, where they'll compete against finalists from across the 22-campus California State University system. Throughout this process, they have relied on support and feedback from the Division of Research and Innovation at SJSU.

"We're really thankful for the research [and innovation] team, who has given us a lot of support," says Tran. "We've met with them for mentorship, advice and feedback, and that's been super helpful. Hopefully next week goes well and we're able to represent SJSU."

When asked about the long-term impact of their research, the team is hopeful that their findings could help clinicians and medical professionals find better ways to measure attention objectively - information that can inform how patients perceive, communicate and understand the world.

"Communication is a big part of everyday life, and not everybody has the skills or the means to get there naturally on their own," says Tran. "I really like being the cheerleader for my patients, and giving them all the resources to help them achieve their communication goals. Audiology is a great way for me to connect with that passion, while also being in the medical field and getting involved in the technology, whether that's hearing aids, cochlear implants or diagnostic batteries. It's a good mix of creativity, patient care and tech."

Gawai describes himself as a "builder" who sees data analysis and AI as important vehicles to streamline and support the development of new technology.

Chiu, whose background is in design, adds that the research project has been the highlight of his graduate experience thus far.

"I've always been really interested in how people can do things intuitively, and how we can design things to be usable, easy and accessible," he says. "Human factors and usability in general combine that sense of design with the rigor of research and really carefully thinking these things through, to make sure we're designing things that are safe and effective for people to use."