Researchers in Japan Develop AI That Can Read Brain Activity From MRI Images

Using MRI Images, a group of researchers out of Osaka University have developed an AI system that can reconstruct what you see. Using Stable Diffusion in conjunction with their model, Neuroscientist Yu Takagi and his partner Shinji Nishimoto were able to produce images by converting brain activity from individuals in an MRI machine.

As you might know, Stable Diffusion, similar to MidJourney and DALL-E 2, takes text input and generates images. In this case, the AI was trained to scan existing images with corresponding captions. Over time, the AI was able to make connections between specific images and words.

So the way it works is that Takagi and his team used this training with two different AI models. One could connect images with functional magnetic resonance imaging, or fMRI, data and the other could like the fMRI data to text descriptions of the images.

Speaking to Al Jazeera, Yu Takagi said of his first reactions to the model, “I still remember when I saw the first images. I went into the bathroom and looked at myself in the mirror and saw my face and thought, ‘Okay, that’s normal. Maybe I’m not going crazy’.”

So far, the AI has an accuracy rating of around 80%. It uses the first AI model to create a vague and indistinct image of what the participants within the MRI machine see. Then it uses the second model to recognize and clarify the images using previously recorded brain pattern associations.

For Takagi, the accuracy rate was surprising. He said in part, “We really didn’t expect this kind of result.” Though the research has made some significant progress, Takagi is concerned that too many people are already taking it to the science fiction realm. “Unfortunately there are many misunderstandings with our research…We can’t decode imaginations or dreams; we think this is too optimistic. But, of course, there is potential in the future.”

As for privacy and possible misuse of the technology if it can advance further, Takagi acknowledged the need for greater discussions, “For us, privacy issues are the most important thing…If a government or institution can read people’s minds, it’s a very sensitive issue. There needs to be high-level discussions to make sure this can’t happen.”

Editor’s Note: ODSC APAC is only a few weeks away! Join the leading minds in Data Science of the APAC region in a two-day event virtual event. Enjoy expert-led sessions with cutting-edge technology. Register now.