Changing the Future of Big Data: Examining and Modifying Algorithms

eAIR recently spoke with Cathy O'Neil, closing keynote speaker for the 2018 AIR Forum in Orlando, Florida. Cathy  is the author of the New York Times bestseller, Weapons of Math Destruction. She earned a Ph.D. in math from Harvard, was a postdoctoral fellow in the MIT math department, and a professor at Barnard College. In the private sector, she was a quantitative analyst for the hedge fund D. E. Shaw during the credit crisis. In 2011 she began working as a data scientist in the New York start-up scene, building models to predict people’s purchases/clicks. Cathy wrote Doing Data Science in 2013 and is a columnist for Bloomberg View

Keynote-CATHY-ONEIL.jpgeAIR: Part 2 of the title of your book, Weapons of Math Destruction, is How Big Data Increases Inequality and Threatens Democracy. With Big Data becoming so important to higher education, especially to IR practitioners, how can these professionals avoid the “Dark Side” of Big Data?

The short answer is, we have to put the science into data science. Specifically, we have to anticipate what might go wrong with our models and set up tests beforehand and ongoing monitors once an algorithm is deployed to measure whether something bad is happening, whether that is bias, discrimination, unfairness, or simply inaccuracy or inconsistency.

eAIR: In your Bloomberg View article, “Congress Is Missing the Point on Facebook,” you assert that what America really needs is a data bill of rights. How can data professionals advocate for ethical collection and use of data and analytics?

If they are empowered to, they should supply the targets of their algorithms with basic information about what data is being used, how it’s being interpreted, and how their score (if it’s a scoring system, which it usually is) could get better. Note that I am only advocating this for powerful and influential scoring systems that could ruin someone’s life - or at least have an actual negative impact on it - if their score is calculated in an unfair way. Many algorithms do not rise to this level of potential impact, so their requirements for transparency might not be as high. But in cases where someone might be restricted in what classes they take, what major they choose, or where they attend college, I think this is warranted.

eAIR: Chapter 3 of Weapons of Math Destruction focuses on data and analytics related to college, with which IR professionals are intimately engaged. How can professionals in this area ensure that their work does not perpetuate the inequalities you examine in your book?

I think the most important thing to keep in mind is that naïve algorithms don’t cause things to be fair, but instead repeat the past. If we think the past deserves repeating, that’s fine. But if we actively wish to evolve past some bad mistakes we’ve made in the past, then we’ll need to do more than simply make a naïve algorithm. That could mean examining and modifying a naïve algorithm to make it better, or it could mean taking a new approach, like orchestras did when they started “blind auditions” to remove the implicit bias from choosing new orchestra members.

eAIR: What are your thoughts on the EU’s GDPR? Does it take the right steps to protect individuals’ data?

I’m not an expert, but I worry that it’s 1) too broad, not concentrating on truly worrisome algorithms enough (and therefore not asking enough of those truly worrisome algorithms) and 2) asking for things like “the right to explanation” that might sound good but might end up being meaningless.

eAIR: Without giving too much away, can you offer a few highlights from your closing keynote at the 2018 AIR Forum?

I want to end on a cautiously optimistic note. Although the promise of big data – that is, to remove human bias from complicated decision-making processes – has not yet been realized, the cool thing about algorithms is that, by nature of their data-driven culture, many such algorithms still have the potential, if we work hard, to fulfill at least part of the promise. I don’t want to give up on algorithms, I want to ask more of them.

If you are going to the Forum, you can attend Cathy's keynote, Weapons of Math Destruction, on June 1 at 10:00 a.m. in the Gatlin Ballroom. 


 AIR Comment Board

To add a comment, Sign In
There are no comments.