Reel Bias Test
Diversity & Inclusion Research Study
Reel Women Facing Real Bias: Analyzing Gender Asymmetry in Hollywood Movies Using Data Visualization
Motivation
The society is undergoing a social reckoning in the form of movements like #MeToo and #TimesUp. Gender inequality roots itself in both the subconscious of individuals and the societal culture as a whole which is highly influenced by the cinema. Discussing the underrepresentation of women in one of the biggest entertainment markets in the world — Hollywood has become so important in today’s world.
Role: Quantitative UX Researcher, Designer
Team: Kausalya Ganesh, Naishi Jain
Methods & Tools: Data Visualization, Tableau, Microsoft Excel
Duration: 10 weeks
Challenge
The Bechdel-Wallace test was one of the starting points of this conversation in the 80s. This test was meant to be a litmus test to demonstrate the lack of screen-space allotted to female characters in movies, but it doesn’t elaborate on the nature of female interaction beyond its independence from male-centric topics.
We aimed to uncover the gaps in the existing gender bias detection tests and find new ways to discuss and quantify the gender asymmetry in Hollywood.
Research
Background Research
Gender Inequality behind the scenes
As seen for the top 250 grossing films of 2017...
There is a 5:1 ratio of men to women working behind the scenes on films.
Percentage of female decision makers ranges from 11% to 25%, whereas that of male decision makers ranges from 75% to 89%.
Women work more documentaries than narratives by 12%.
Gender Inequality in Scripts
The USC Viterbi School of Engineering used AI to study nearly 1000 films in 2017, concluding that in the majority of films, female roles are not central to plot.
Dialogues
Men had more than 37,000 dialogues; women had just over 15,000
Characters
Men portrayed almost 4,900 characters; women portrayed just over 2,000
Writers
Scripts had 7x more male writers than female writers
Sources: USC Annenberg Media, Diversity & Social Change Initiative's Inequality in 900 Popular Films Report, USC Viterbi School of Engineering & The 2017 Celluloid Ceiling Report
Quantitative Research & Analysis
We used lists from various sources to identify the 50 most popular movies of 2017. This dataset consists of movies that did very well in the opinion of the general audience and famous critics on IMDb and Rotten Tomatoes. We assumed that the popularity of these movies can be translated to their influence and impact.
33 out of the 50 movies in the list pass the Bechdel-Wallace Test.
Gaps in the Bechdel-Wallace Test
Many movies barely passed the test on a technicality, such as female-female conversations not being specifically about men, but might still be centered around gender-stereotypical topics.
Thor: Ragnarok failed the test
The antagonist is a female character with a significant role in the movie, but the movie failed the test on account of having no two female characters interacting with each other about something other than the leading male characters.
Beauty & the Beast passed the test
Beauty discusses her wardrobe and the castle rules (something other than a man, but still irrelevant) with Mrs. Potts, Plummete, and Madame Gaderobe.
Need for a new set of criteria
A surprising number of films fail to do basic character development work with women. Whether female interactions go beyond male-centric topics determines how much the supporting female cast matters to the overall storyline.
Female characters are pigeonholed into predefined roles such the action chick, romantic interest, or the middle-aged mother. According to a global film study by the Geena Davis Institute on Gender in Media, female characters are at least twice as likely as men to be shown in sexually explicit scenes and are five times more likely to be referenced as attractive.
Women frequently serve little purpose beyond causing plot problems for the protagonist or having a baby/dying to raise the stakes for a male character. Film-makers claim that they unintentionally make movies about themselves. The most powerful producers, writers, and directors are men, so male-themes permeate into Hollywood’s output.
The imbalance in the sex ratio of the crew is stark. We can’t reasonably determine the sexual orientation of the entire crew, but it is possible to highlight the gender imbalance in key roles.
“In the next two years, I’m making a vow that everything I produce will be 50 percent women. And I challenge everyone out there who is in a position of power, in all industries, to challenge yourself and stand with us in solidarity and do the same.” — Regina King
Solution
In an attempt to add to the ongoing dialogue, we analyzed films and existing literature on gender inequality in Hollywood. Based on the insights gained through our research, we created six criteria to test and highlight gender imbalance. This is called the ‘Reel Bias Test’ (RBT).
Reel Bias Test is a points-based test that evaluates movies based on six criteria. If a movie scores more than 3 out of 7 points, it passes the Reel Bias Test.
Results
We tested the 50 movies from the list against the Reel Bias Test criteria to compare the results. We designed a Sankey chart data visualization to communicate the results of the Reel Bias Test.
21 out of the 50 movies in the list pass the Reel Bias Test.
An interactive visualization of Reel Bias Test is available on Tableau
Only 28% of the movies have a female decision-maker (Director, Producer or Screenwriter) in their crew.
Lady Bird won ‘Best Motion Picture’ at the Golden Globes 2018, but the Director, Greta Gerwig did not even get nominated for Best Director. The category had five male nominees.
Going in Style, Dunkirk, and Pirates of the Caribbean - Dead Men Tell No Tales don’t pass any of the criteria.
Dunkirk is a period drama set in the 40s, so failing this test was inevitable. Despite being set in the year 2017, 'Going in Style' fails as there were no significant female characters in this movie.
Reel Bias Test v/s Bechdel-Wallace Test
In order to compare the results of the two tests directly, we created a Waffle Chart on Tableau and placed filters to analyze the results by production houses, budget, decision maker gender and box office numbers provided on IMDb and Wikipedia.
An interactive visualization of the comparison is available on Tableau
Only 9 movies with all male-decision makers pass the Reel Bias Test.
23 movies with the same criteria pass the Bechdel-Wallace Test.
While Thor: Ragnarok fails the Bechdel-Wallace test, it manages to pass the Reel Bias Test on the basis of having a strong female antagonist and more than two supporting female characters.
Guardians of Galaxy V2 passes the Bechdel-Wallace test but fails the Reel Bias Test because the only criteria it passes is that of two female characters talking to each other about something other than a man.
Limitations: One Test is Not Enough
The Reel Bias Test identifies some of the problematic scenarios to inform the decision-makers and the general audience of the gender asymmetries in Hollywood. However, this test does not take into account the total screen time, the total number of lines spoken by each gender or the inequality in pay, yet… The test also takes only two genders into account.
One test can’t change an entire industry that has an inherent problem with giving women equal opportunities and equal pay. Any test has a serious real-world limitation and the only way to reduce the errors is by revamping the criteria regularly.
Applications
Online Streaming Platforms
With this new era of internet-enabled entertainment, consumers have the greatest control over the type of content they consume. Once automated, an interesting implementation of the Reel Bias Test would be on platforms like Netflix, Hulu, and Amazon Prime. Original content and other movies that pass this test could inform audiences of the gender parity in their production by referring to the test in some way.
Other Film Industries
We also plan on conducting a similar research on movies produced in Bollywood and other film industries. We hope to learn if the criteria of Reel Bias Test apply to movies in other industries, or if with changing cultures, the criteria change.
Project Outcome
Based on the research conducted in the first phase of the project, we framed some research questions in order to explore further in this direction.
What is considered gender bias in Hollywood?
To generate a definition of gender bias from analysis.
How do we frame criteria for a gender bias test?
To understand the various attributes in order to construct a gender bias test.
How does the test show if there is gender bias in Hollywood?
To validate the criteria designed to test gender bias in Hollywood.
Won an award for this project in the Diversity & Inclusion Category at the iSchool Symposium 2019.
Learnings
Quantitative Research & Analysis
Since we were dealing with numerical data, we used quantitative research and correlational analysis to understand and assess the relationship between variables such as box office numbers, gender ratio, etc. Such analysis helped is identify patterns and uncover insights from the data.
Data Visualization
We used the principles of data visualization to create a Sanky and Waffle chart using Tableau in order to effectively and engagingly communicate the results of the analysis.
Selected Works
Beyond the Kitchen TableQualitative User Research & Interaction Design
Spin: A Smart Way of PresentingInclusive Design & Co-Design
Aging in Place with ADRDQualitative User Research & Inclusive Design
Let's Connect
I am available for full-time opportunities starting June 2020.
If you are interested in working together, I'd love to connect with you!