Facebook Ad's Massive Design Bug

In mid-September ProPublica published an article proving Facebook’s advertising system helped them market to people who expressed interest in radical and racial topics:

“Want to market Nazi memorabilia, or recruit marchers for a far-right rally? Facebook’s self-service ad-buying platform had the right audience for you.

Until this week, when we asked Facebook about it, the world’s largest social network enabled advertisers to direct their pitches to the news feeds of almost 2,300 people who expressed interest in the topics of “Jew hater,” “How to burn jews,” or, “History of ‘why jews ruin the world.’” Published September 14th, 2017

Turns out it was technically possible to use Facebook ads to target users claiming to be anti-semitic. This same system could perhaps be used by Russians to target American Voters but that’s a whole other thing. Facebook’s ad system allows advertisers to type categories of users they’d like to target as an input field and then an automated matching system will check those user supplied categories (full or parital) with ones Facebook already had like this:

Aside from the moral and ethical problems of targeting users and groups this way, from a software perspective we might classify this as a design bug. Unlike a coding bug where the program behaves in a way the programmer didn’t intend, a design bug is when the program behaves in a way the programmer did intend but stakeholders don’t like it. In this example Facebook’s ad system was behaving as it was programmed but in a way that was HIGHLY questionable to many stakeholders like advertisers, advertisees, journalists and now federal authorities.

Design Bugs

Design bugs are some of the most common types of bugs developers (testers) find. They are are often caught by questioning the system, approach, design, requirements, et .al in a holistic way. In cross functional and agile teams I’ve found this is easier because you can take a step back to look at the bigger picture with many of the original decision makers. Going through the implications of those decisions as a mental exercise will often expose a bug or two that can be then addressed in the near future.

To help in questioning design bugs, try to:

• Find a set of scenarios or circumstances that showcase your concerns
• Make those concerns as costly or annoying as possible
• Find places outside of the program where users will be impacted

For those familiar with the RIMGEN bug reporting mnemonic, you might notice some similarities, although with a different emphasis.

We (all software developers) introduce design bugs all of the time with the justification of it “works as designed”. Most times the design of a system is’t so damaging. Sometimes it is incredibly.[highlight]This lesson seems worth keeping in mind. [/highlight]

Other Things:

• You can see Facebook’s response here.
• Benedict Evans suggested this was more of an exploit and programmers (I say all software developers) should consider what happens when users are evil. This is probably a valuable approach. I often take the approach of “anything you allow me to do I will do”. In other words, give me enough room to hang myself and I will.