A new grant will fund the use of extended reality — technologies that merge the physical and virtual worlds — as rehabilitation for patients with cardiovascular and Parkinson’s diseases.

BY KATHARINE GAMMON

USING VIRTUAL REALITY TO HELP REHABILITATE PATIENTS has long been a goal for physical therapists.

After all, virtual worlds can offer immersive experiences in a safe, controlled environment that is tailored to individual patient needs. But making systems that work in all settings — and creating systems that can connect users around the globe, without glitching — comes with challenges.

Associate Professor James Finley has earned a new National Science Foundation grant to better explore the use of extended reality in rehabilitation for patients with cardiovascular diseases and Parkinson’s. The two-year, multi-institution grant will study the networks needed to enable extended-reality rehabilitation as well as the creation of immersive environments.

Using Optical Flow to Address Gait Asymmetry

Finley originally became interested in extended reality when he was a postdoctoral researcher. Trained as an engineer, Finley studied the influence of optic flow on how people walk — basically, the pattern of apparent motion of objects, surfaces and edges in a visual scene.

Using optic flow, Finley was able to modify people’s gait asymmetry while they were walking. “That was the initial spark that really led me to think about how we can use visual information in other applications,” he said. “It just so happened that when I started my position here, VR became consumer grade. In 2010 if you wanted a VR headset, it cost about $10,000, but by 2014 if you wanted one, you could get one for $300.”

In one of his previous studies, Finley teamed up with researchers from the USC School of Cinematic Arts to design Wordplay, a puzzle similar to Wheel of Fortune, where patients had to move around to grab letters and place them in the puzzle’s virtual environment. They built a prototype and got feedback — but then the COVID-19 pandemic took hold, putting the project on hiatus. But researchers around the country saw the publication of the prototype and reached out about this new project.

Now, Finley — a study co-investigator — has teamed up with a larger group of researchers, including scientists from the University of Michigan and game developers from Duke University to study “ways to basically push the limits of our current communication protocols.”

They have created a game where a patient can interact with multiple people in a virtual environment — from multiple locations. A key goal is to make the experience of rehabilitation more enjoyable and effective by incorporating social interactions between remote users, the researchers explained. The technology also allows clinicians to guide and interact with their patients remotely. To create a virtual environment that users experience as fast and seamless, the project develops new approaches to the underlying networking infrastructure needed to run the application.

Into the Virtual Realm for Treatment

Finley imagines a clinic of the future, where someone recovering from Parkinson’s or a stroke could enter a game environment. “Your therapist is there to guide you,” he said. “But once you go into the virtual environment, you see one of your loved ones. Friends who have the same system at home, and you can play together. We’ve been brainstorming different ways that we could add a social component to this training application.”

Data gathered during the game can also inform treatment — Finley says the plan is that as the person moves around, sensors collect their actions and give back quantitative data about their performance. One research question to answer is how physical therapists might change their treatment protocols if they have access to systems that track their progress.

Physical therapists can also be seen in the environment with the patient — which helps them understand how the exercises are working. It’s not about replacing the therapist but offering them more data and insight into what is happening with the patient.

Finley is excited about the potential for extended reality to bolster therapy in the future. “There are always two sides to technology,” he said. “It has to be very user-friendly and robust. If we can address that challenge, and we have a system that clinicians think is useful, then I can imagine that this is something that is not only beneficial from a therapeutic perspective but also draws people into the clinic.”