Reconstruction of visual imagery from human brain activity measured by fMRI

To reconstruct visual images, we first decoded (translated) measured brain activity patterns into deep neural network (DNN) features, then fed those decoded features to a reconstruction algorithm. Our reconstruction algorithm starts from a given initial image and iteratively optimizes the pixel values so that the DNN features of the current image become similar to those decoded from brain activity. We applied our algorithm to brain activity collected during mental imagery.

Left: Imagined images.
Right: Reconstructed images. The iterative optimization process is shown.