Algorithm helps make faces more memorable

Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) have developed an algorithm which makes subtle changes to various points on the face to make it more memorable without changing a person’s overall appearance.

“We want to modify the extent to which people will actually remember a face,” says lead author Aditya Khosla, a graduate student in the Computer Vision group within CSAIL. “This is a very subtle quality, because we don’t want to take your face and replace it with the most memorable one in our database, we want your face to still look like you.”

The system could ultimately be used in a smartphone app to allow people to modify a digital image of their face before uploading it to their social networking pages. It could also be used for job applications, to create a digital version of an applicant’s face that will more readily stick in the minds of potential employers, says Khosla, who developed the algorithm with CSAIL principal research scientist Aude Oliva, the senior author of the paper, Antonio Torralba, an associate professor of electrical engineering and computer science, and graduate student Wilma Bainbridge.

Conversely, it could also be used to make faces appear less memorable, so that actors in the background of a television program or film do not distract viewers’ attention from the main actors, for example.

To develop the memorability algorithm, the team first fed the software a database of more than 2000 images. Each of these images had been awarded a ‘memorability score’, based on the ability of human volunteers to remember the pictures. In this way the software was able to analyse the information to detect subtle trends in the features of these faces that made them more or less memorable to people.

The researchers then programmed the algorithm with a set of objectives - to make the face as memorable as possible, but without changing the identity of the person or altering their facial attributes, such as their age, gender or overall attractiveness. Changing the width of a nose may make a face look much more distinctive, for example, but it could also completely alter how attractive the person is, and so would fail to meet the algorithm’s objectives.

When the system has a new face to modify, it first takes the image and generates thousands of copies, known as samples. Each of these samples contains tiny modifications to different parts of the face. The algorithm then analyses how well each of these samples meets its objectives.

Once the algorithm finds a sample that succeeds in making the face look more memorable without significantly altering the person’s appearance, it makes yet more copies of this new image, with each containing further alterations. It then keeps repeating this process until it finds a version that best meets its objectives.

“It’s really like applying an elastic mesh onto the photograph that slightly modifies the face,” Oliva says. “So the face still looks like you, but maybe with a bit of lifting.”

The team then selected photographs of 500 people and modified them to produce both a memorable and forgettable version of each. When they tested these images on a group of volunteers, they found that the algorithm succeeded in making the faces more or less memorable, as required, in around 75% of cases.

Making a face appear familiar can also make it seem more likable, Oliva says. She and Bainbridge have published a complementary paper in the journal Cognitive Science and Social Psychology on the attributes that make a face memorable. The first time we see a face, we tend to “tag” it with attributes based on appearance, such as intelligence, kindness or coldness. “If we tag a person with familiarity, because we think this is a face we have seen before, we have a tendency to like it more and, for instance, to think the person is more trustworthy,” she says.

The team is now investigating the possibility of adding other attributes to their model, so that it could modify faces to be both more memorable and to appear more intelligent or trustworthy, for example. “So you could imagine having a system that would be able to change the features of your face to make you whatever you would wish for, but always in a very subtle way,” Oliva says.

The research was funded by grants from Xerox, Google, Facebook and the Office of Naval Research.