Researchers Use AI to Recreate Images From Brain Activity

(Credit: DALL-E AI Image) Science fiction is filled with machines that can read the human mind, but you might be able to drop the “fiction” part soon. Researchers from Osaka University have used an advanced AI model to process human brain activity and recreate images seen by the human test subject. Unlike past attempts to read human brain waves, the diffusion model utilized in this study didn’t require extensive training or fine-tuning, and the results are shockingly accurate.

To create the reconstructed images, the researchers started with functional Magnetic Resonance Imaging (fMRI) scans. Study participants were shown various images while the medical equipment collected brain activity data. Some past experiments have done similar work with fMRI data, for example, to recreate a face that someone had been shown. However, these models are hamstrung by generative AI algorithms that need to be trained with large data sets. There simply isn’t very much neuroscience training data available. The key to the new research was to use a diffusion model.

In diffusion models, an AI introduces random noise to the data and then learns now to remove it. The model can then apply the denoising process to random seeds to create a realistic image. The more stimuli in the brain, the more noise in the data. And with more noise, you get more resolution in the final image. This study used the Stable Diffusion image generator, which Qualcomm recently downsized to run on a smartphone.

The examples above show the presented images on the top row and the AI-generated versions below. Even without being told what the original image depicted, the AI is able to generate something relatively close just from the fMRI data. It might not nail the exact shape or scale of the image, but the Stable Diffusion 512 x 512 images are certainly in the ballpark.

Currently, gathering the necessary fMRI data to generate these images requires the subject to stick their head inside a giant magnet. That might not be necessary for long, though. Companies like Elon Musk’s Neuralink are working toward brain-computer interface implants that would be able to record brain data using tiny electrodes. At that point, we’re getting very, very close to Black Mirror territory. On the other hand, this technology could be hugely useful. It could allow instant recall of important visual details or give non-verbal people another way to communicate.

Now read:

• Elon Musk’s Neuralink Expects to Install Brain Chip in Humans in 6 Months
• Biocomputers Using Human Brain Cells Could Overpower AI
• Nvidia CEO Calls ChatGPT the ‘iPhone Moment’ for AI