Dietrich College News

From left to right, Professor Tom Mitchell, PBS NOVA Host David Pogue and Professor Marcel Just

November 2012

PBS NOVA Profiles Marcel Just's Research

Reporter Challenge Leads To Mind Reading 2.0

In 2008, Carnegie Mellon University Professors Marcel Just and Tom Mitchell shocked the world with their thought identification research, using a brain scanner to demonstrate, for the first time, the ability to read people’s minds.

They did this by applying machine-learning techniques to brain imaging data to identify the contents of an individual’s thoughts of concrete objects. A demonstration of how the process works – from one computer displaying a labeled picture of a concrete object (e.g. hammer) to a person in the MRI scanner, to a separate computer analyzing the resulting brain image to determine which object was shown – was featured on 60 Minutes.

Since the initial discovery, Just and Mitchell have been working to identify other types of thoughts besides concrete objects – from numbers to emotions to social interactions - all involving a visual stimulus (usually a printed word) for the person inside the brain scanner to think about.

That was, until recently.

In August, a PBS NOVA scienceNOW media crew visited CMU’s Scientific Imaging and Brain Research (SIBR) center to learn more about the thought identification research for an upcoming episode on the future. Host David Pogue went through the experiment and was shown ten pairs of objects two times while in the fMRI. With the cameras rolling, the brain analysis computer then correctly identified each object that Pogue had been thinking about.

Pogue was impressed but unsatisfied. He explained that as an amateur magician, he found it disquieting that the computer that presented the words “knew” what he would be thinking about. He was concerned that the presentation computer and the brain analysis computer could have been in cahoots with each other. He challenged the research team to find some way to cut the presentation computer out of the loop, so that only he, Pogue, knew what he had been thinking about at any given time.

With some trepidation, the team accepted the challenge. “We had never tried anything like this before in the lab, let alone on national TV,” said Just, the D.O. Hebb Professor of Psychology within the Dietrich College and director of CMU’s Center for Cognitive Brain Imaging. “But the theory said it should be possible.”

Mitchell, the E. Fredkin University Professor of Machine Learning within the School of Computer Science, added, “I always tell my students not to mess with a computer demonstration that works.”

So, the team constructed a new experiment on the fly, and Pogue went back into the scanner, but this it was up to him to decide on the spot which of two objects – either a skyscraper or strawberry - to think about. There was nothing presented on the screen that told him what to think about. He did this ten times. He was armed with a notepad and pen to jot down which object he had been thinking about on each of the ten trials. When the scan was complete, Just, Mitchell, Pogue and the video crew all anxiously waited to see whether the brain analysis computer could correctly identify Pogue’s thoughts, despite no explicit instruction of which word he should think of, the fact that the computer had never trained on his brain before and that the computer had never seen a person’s fMRI data for the two words, “strawberry” and “skyscraper.”

With the cameras rolling, the analysis computer issued its identification for the first item, and everyone held their breath and turned to Pogue. He paused dramatically, and then finally said “Correct!” He again said “Correct” for the second item. Just and Mitchell started to relax. And so it went until all ten items were correctly identified. It was a major scientific experiment producing a fascinating new finding, done in the context of a demo for a science documentary.

“We were all nervous about trying this experiment for the first time with the cameras rolling,” said Mitchell. “But, we had seen before that our computer model had successfully predicted neural representations for new words, and that these neural representations are remarkably similar across different people.”

Just added, “It’s exciting to know that it is possible to identify internally-generated thoughts, and something we will follow up on more formally in our future studies.”

To watch the PBS NOVA scienceNOW episode featuring Just and Mitchell’s latest mind reading discovery, click here.

While combining machine learning with neuroscience may seem like a new phenomenon to some people, Carnegie Mellon has been taking an interdisciplinary approach to brain sciences for decades, using it to decipher and improve learning, perception and thinking; deal with aging or injured brains; and treat and understand disorders such as autism, dyslexia and Alzheimer’s. To continue to use psychology, computer science and computation to solve real world problems, the university recently launched a Brain Mind and Learning initiative.

Stay connected with CMU's Dietrich College of Humanities and Social Sciences on Twitter and Facebook.

Other sources of Carnegie Mellon news include the university news service website and the Carnegie Mellon Today magazine.

Contact Shilo Rea, Director of Public Relations at shilo@cmu.edu or (412) 268-6094.

About the Quick Links