Mixed reality research could revolutionize training of first responders

High-tech training for first responders

Neil Nachbar

From left: URI graduate students Jiangong Chen, Matthew Constant and Noah Johnson wearing Google Daydream headsets in Bin Li’s research laboratory

When disasters strike, first responders bravely spring into action and risk their lives to save others. What if those risks could be decreased with the use of technology in training sessions?

By combining images of actual buildings with computer-generated graphics, simulations can be created in which multiple people can train together in real time. This technology is known as wireless collaborative mixed reality.

Bin Li, assistant professor of computer engineering at the University of Rhode Island, has been awarded a five-year, $505,900 National Science Foundation grant to develop technology for such training purposes.

“Wireless Collaborative Mixed Reality could potentially revolutionize existing collaborative mission-critical training exercises, such as firefighter drills and disaster response training,” said Li, who received the grant from the NSF’s CAREER Award program.

“Mixed reality is a new paradigm that merges both real and virtual worlds to create new environments and visualizations,” said Li. 

“With the rapid growth of wireless augmented reality and virtual reality devices, such as smartphones and HoloLens, wireless collaborative mixed reality applications can provide an interactive and immersive experience for a group of people.”

In the case of firefighters, central servers would run fire accident simulations over a 3D virtual model of a real building and deliver timely 360-degree virtual reality images to each firefighter via wireless communication, according to each firefighter’s location and perspective in the real building.

“Each firefighter’s AR/VR headset would recognize real-world objects, render fire-spreading VR images on top of them, and then the firefighter would take the corresponding actions, such as extinguishing the virtual fires,” said Li.

The actions of the firefighter are then fed back to the central servers for further simulations.

“This approach to firefighter training is not currently available,” stated Li. “It would significantly improve the performance and adaptability of firefighters in real fire scenarios.”

Li’s research will take place in the Smart Networking and Computing Laboratory in URI’s Fascitelli Center for Advanced Engineering. During the research process, firefighters will be asked to give their feedback on the technology.

The NSF grant will enable the professor to add a graduate student to the project.

The Faculty Early Career Development (CAREER) Program offers the NSF’s most prestigious awards in support of early-career faculty who have the potential to serve as academic role models in research and education and to lead advances in the mission of their department or organization.