Carnegie Mellon University Home

Dietrich College News

From left to right, Professor Tom Mitchell, PBS NOVA Host David Pogue and Professor Marcel Just

November 2012

PBS NOVA Profiles Marcel Just's Research

Reporter Challenge Leads To Mind Reading 2.0

In 2008, Carnegie Mellon University Professors Marcel Just and Tom Mitchell shocked the world with their thought identification research, using a brain scanner to demonstrate, for the first time, the ability to read people’s minds.

They did this by applying machine-learning techniques to brain imaging data to identify the contents of an individual’s thoughts of concrete objects. A demonstration of how the process works – from one computer displaying a labeled picture of a concrete object (e.g. hammer) to a person in the MRI scanner, to a separate computer analyzing the resulting brain image to determine which object was shown – was featured on 60 Minutes.

Since the initial discovery, Just and Mitchell have been working to identify other types of thoughts besides concrete objects – from numbers to emotions to social interactions - all involving a visual stimulus (usually a printed word) for the person inside the brain scanner to think about.

That was, until recently.

In August, a PBS NOVA scienceNOW media crew visited CMU’s Scientific Imaging and Brain Research (SIBR) center to learn more about the thought identification research for an upcoming episode on the future. Host David Pogue went through the experiment and was shown ten pairs of objects two times while in the fMRI. With the cameras rolling, the brain analysis computer then correctly identified each object that Pogue had been thinking about.

Pogue was impressed but unsatisfied. He explained that as an amateur magician, he found it disquieting that the computer that presented the words “knew” what he would be thinking about. He was concerned that the presentation computer and the brain analysis computer could have been in cahoots with each other. He challenged the research team to find some way to cut the presentation computer out of the loop, so that only he, Pogue, knew what he had been thinking about at any given time.

With some trepidation, the team accepted the challenge. “We had never tried anything like this before in the lab, let alone on national TV,” said Just, the D.O. Hebb Professor of Psychology within the Dietrich College and director of CMU’s Center for Cognitive Brain Imaging. “But the theory said it should be possible.”

Mitchell, the E. Fredkin University Professor of Machine Learning within the School of Computer Science, added, “I always tell my students not to mess with a computer demonstration that works.”

So, the team constructed a new experiment on the fly, and Pogue went back into the scanner, but this it was up to him to decide on the spot which of two objects – either a skyscraper or strawberry - to think about. There was nothing presented on the screen that told him what to think about. He did this ten times. He was armed with a notepad and pen to jot down which object he had been thinking about on each of the ten trials. When the scan was complete, Just, Mitchell, Pogue and the video crew all anxiously waited to see whether the brain analysis computer could correctly identify Pogue’s thoughts, despite no explicit instruction of which word he should think of, the fact that the computer had never trained on his brain before and that the computer had never seen a person’s fMRI data for the two words, “strawberry” and “skyscraper.”

With the cameras rolling, the analysis computer issued its identification for the first item, and everyone held their breath and turned to Pogue. He paused dramatically, and then finally said “Correct!” He again said “Correct” for the second item. Just and Mitchell started to relax. And so it went until all ten items were correctly identified. It was a major scientific experiment producing a fascinating new finding, done in the context of a demo for a science documentary.

“We were all nervous about trying this experiment for the first time with the cameras rolling,” said Mitchell. “But, we had seen before that our computer model had successfully predicted neural representations for new words, and that these neural representations are remarkably similar across different people.”

Just added, “It’s exciting to know that it is possible to identify internally-generated thoughts, and something we will follow up on more formally in our future studies.”

To watch the PBS NOVA scienceNOW episode featuring Just and Mitchell’s latest mind reading discovery, click here.

While combining machine learning with neuroscience may seem like a new phenomenon to some people, Carnegie Mellon has been taking an interdisciplinary approach to brain sciences for decades, using it to decipher and improve learning, perception and thinking; deal with aging or injured brains; and treat and understand disorders such as autism, dyslexia and Alzheimer’s. To continue to use psychology, computer science and computation to solve real world problems, the university recently launched a Brain Mind and Learning initiative.

Stay connected with CMU's Dietrich College of Humanities and Social Sciences on Twitter and Facebook.

Other sources of Carnegie Mellon news include the university news service website and the Carnegie Mellon Today magazine.

Contact Shilo Rea, Director of Public Relations at shilo@cmu.edu or (412) 268-6094.

 

About the Quick Links

Follow the Dietrich College on Twitter and Facebook.
Sign up to receive Dietrich College News, the college's monthly e-newsletter.

  Scott Sandage Helps "Modern Family's" Jesse Tyler Ferguson Learn His Family History on "Who Do You Think You Are?"
Premiering at 9 p.m. EDT, Wednesday, July 30, the episode will show Sandage helping Ferguson uncover the life of his great-grandfather whom trouble seemed to follow at every turn. Read more.
  Save the Date: Celebrating the Work of Steven Klepper
The Dietrich College and Department of Social and Decision Sciences will hold an academic conference and community memorial event on October 17, 2014 to celebrate the work of Steven Klepper. Read more (pdf).
  Dietrich College News: May 2014
Features this month include new psychology research, "Lucky After Dark," the 2014 Adamson Student Writing Awards, alumni news, commencement photos and much more. Read Dietrich College News.
  Heavily Decorated Classrooms Disrupt Attention and Learning In Young Children
Published in Psychological Science, CMU's Anna V. Fisher, Karrie E. Godwin and Howard Seltman looked at whether classroom displays affected children's ability to maintain focus during instruction and to learn the lesson content. They found that children in highly decorated classrooms were more distracted, spent more time off-task and demonstrated smaller learning gains than when the decorations were removed. Read more.   Watch a video.
  Four Students Begin Dietrich Honors Fellowship Program
The program provides summer funding to rising seniors in the Dietrich College's senior honors program as they undertake early-stage research and development of their thesis topics. In this video, the four students participating in program's inaugural year discuss their projects which range from relationship research to anthropology and ethnography studies. Watch the videoFind out more about the program.
  Mapping the Future of the Humanities and Social Sciences
Carnegie Mellon University — which has a long history of the humanities and social sciences collaborating with other fields to solve problems — and the American Academy of Arts and Sciences held the first Pittsburgh-area discussion of the "Heart of the Matter" report and its implications for improving education and creating a sustainable global society. Read more.   Watch the video.  View photos.
  New Dietrich Honors Fellowship Program Announced
The program will provide summer funding support to rising seniors in the Dietrich College senior honors program as they undertake early-stage research and development of their thesis topics. Read more.
  Richard Scheines Appointed Dean of Dietrich College
Scheines, professor and head of the Department of Philosophy, has been selected to lead the Dietrich College of Humanities and Social Sciences as dean, effective July 1, 2014. He will succeed John Lehoczky, who has served as dean since 2000 and will return to the Department of Statistics’ faculty. Read more.
  Students, Alumni Attend Under Construction
Nearly 60 alumni returned to support 150 current students and help them build their careers. The second annual “Under Construction: Building Your Future” event encouraged students and alumni to explore the diverse fields available after graduation. Read more.
  Video: The Humanities at Carnegie Mellon University
The Dietrich College of Humanities and Social Sciences is proud to have distinguished faculty and talented students who are leading the humanities into the 21st century. In this short video, Dietrich College faculty, staff and alumni talk about the college’s excellence in the humanities. Watch the video.

 

H&SS Home | Admissions | Advising & Careers | Departments & Programs | Research | Computing & Libraries | News | Alumni

Site Index | About H&SS | Message from the Dean

 

Carnegie Mellon University
College of Humanities & Social Sciences | 5000 Forbes Avenue | Baker Hall 154 | Pittsburgh, PA 15213 | (412) 268-2830