A new artificial intelligence (AI) algorithm can transform your brain activity into images. AI was able to recognize a thousand pictures with a match accuracy of about 80%.
In the picture below, in the top row, are the images that people have looked at. In the bottom row are images created by an algorithm that analyzed the mental activity of these people.
The DALL-E 2 neural network algorithm from OpenAI was taken as the basis, which, using stable diffusion (LDM), creates images according to a text description.
A team of scientists from Japan distributed a set of pictures to the experimental group, and when people looked at the pictures, they were given an MRI (magnetic resonance imaging). Then the MRI images were given to artificial intelligence for interpretation.
“We show that our method can reconstruct high-resolution images with high semantic accuracy from human brain activity,” the team shared in a study published in bioRxiv.
“Unlike previous image reconstruction studies, our method does not require training or fine-tuning of complex deep learning models.”
The algorithm extracts information from parts of the brain involved in image perception, such as the occipital and temporal lobes, according to Yu Takagi, who led the study.
The team used MRI because it captures changes in blood flow in active areas of the brain. MRI can even detect oxygen molecules, so scanners can see where in the brain our neurons — the nerve cells in the brain — are working the most (and consuming the most oxygen) when we think or feel emotions.
There were only four participants in this study, each viewing a set of 1,000 images.
“We demonstrate that our simple framework can reconstruct high-resolution images from brain activity with high semantic fidelity,” the study says.
“We quantify each component of the LDM in terms of neuroscience, mapping specific components to individual brain regions. We present an objective interpretation of how the text-to-image process implemented by LDM incorporates the semantic information expressed by the conditional text while preserving the look and feel of the original image.”
Combining artificial intelligence with brain scanners has been a longstanding challenge for the scientific community, which they believe holds the new key to unlocking our inner world.
