Tiny explosions, soft materials make onscreen braille more robust

From texting on a smart phone to ordering train tickets at a kiosk, touch screens are ubiquitous and, in most cases, relatively reliable. But for people who are blind or visually impaired and use electronic braille devices, the technology can be vulnerable to the elements, easily broken or clogged by dirt, and difficult to repair.

By combining the design principles and materials of soft robotics with microscale combustions, Cornell researchers have now created a high-resolution electronic tactile display that is more robust than other haptic braille systems and can operate in messy, unpredictable environments.

The technology also has potential applications in teleoperation, automation and could bring more tactile experiences to virtual reality.

The research was publishedAug. 27 in Science Robotics. The paper's co-first authors are Ronald Heisser, Ph.D. '23 and postdoctoral researcher Khoi Ly.

"The central premise of this work is two-fold: using energy stored in fluid to reduce the complexity of mass transport, and then thermal control of pressure to remove the requirements of complex valving," said Rob Shepherd, the John F. Carr Professor of Mechanical Engineering in Cornell Engineering and the paper's senior author.

"Very small amounts of combustible fuel allows us to create high-pressure actuation for tactile feedback wherever we like using small fluid channels, and cooling the gas during the reaction means this pressure stays localized and does not create pressure where we do not want it," he said. "This chemical and thermal approach to tactile feedback solves the long-standing 'Holy Braille' challenge."

The majority of refreshable electronic tactile displays contain dozens of tiny, intricate components in a single braille cell, which has six raised dots. Considering that a page of braille can hold upwards of 6,000 dots, that adds up to a lot of moving parts, all at risk of being jostled or damaged. Also, most refreshable displays only have a single line of braille, with a maximum of roughly 40 characters, which can be extremely limiting for readers, according to Heisser.

"Now people want to have multi-line displays so you can show pictures, or if you want to edit a spreadsheet or write computer code and read it back in braille," he said.

Rather than relying on electromechanical systems - such as motors, hydraulics or tethered pumps - to power their tactile displays, Shepherd's Organic Robotics Labhas taken a more explosive approach: micro combustion. In 2021, they unveiled a systemin which liquid metal electrodes caused a spark to ignite a microscale volume of premixed methane and oxygen. The rapid combustion forced a haptic array of densely packed, 3-millimeter-wide actuators to cause molded silicone membrane dots - their form determined by a magnetic latching system - to pop up.

For the new iteration, the researchers created a 10-by-10-dot array of 2-millimeter-wide soft actuators, which are eversible - i.e., able to be turned inside out. When triggered by a mini combustion of oxygen and butane, the dots pop up in 0.24 milliseconds and remain fixed in place by virtue of their domed shape until a vacuum sucks them down. The untethered system maintains the elegance of soft robotics, Heisser said, resulting in something that is less bulky, less expensive and more resilient - "far beyond what typical braille displays are like."

"We opted to have this rubber format where we're molding separate components together, but because we're kind of molding it all in one go and adhering everything, you have sheets of rubber," said Heisser, currently a postdoctoral researcher at Massachusetts Institute of Technology. "So now, instead of having 1,000 moving parts, we just have a few parts, and these parts aren't sliding against each other. They're integrated in this way that makes it simpler from a manufacturing and use standpoint."

The silicone sheets would be replaceable, extending the lifespan of the device, and could be scaled up to include a larger number of braille characters while still being relatively portable. The hermetically sealed design also keeps out dirt and troublesome liquids.

"From a maintenance standpoint, if you want to give someone the ability to read braille in a public setting, like a museum or restaurant or sports game, we think this sort of display would be much more appropriate, more reliable," Heisser said. "So someone spills beer on the braille display, is it going to survive? We think, in our case, yes, you can just wipe it down."

This type of technology has numerous medical and industrial applications in which the sense of touch is important, from mimicking muscle to providing high-resolution haptic feedback during surgery or from automated machines, in addition to increasing accessibility and literacy for people who are blind or visually impaired.

"As technologies become more and more digitized, as we rely more and more on computer access, human-computer interaction becomes essential," Heisser said. "Reading braille is equivalent to literacy. The workaround has been screen-reading technologies that allow you to interact with the computer, but don't encourage your cognitive fluency."

Co-authors include Nikolaos Bouklas, associate professor of mechanical and aerospace engineering; Sadaf Sobhani, assistant professor of mechanical and aerospace engineering; postdoctoral researcher Ofek Peretz; doctoral student Young Kim; Carlos Diaz-Ruiz, Ph.D. '23, Rachel Miller Ph.D. '23 and Cameron Aubin, M.S., Ph.D. '22.

The research was supported by the U.S. Department of Energy's Advanced Research Project Agency-Energy, the Office of Naval Research and the National Science Foundation.