Sorelle Friedler

Shibulal Family Associate Professor of Computer Science

Sorelle Friedler is the Shibulal Family Associate Professor of Computer Science at Haverford College. She served as the Assistant Director for Data and Democracy in the White House Office of Science and Technology Policy under the Biden-Harris Administration where her work included the AI Bill of Rights. Her research focuses on the fairness and interpretability of machine learning algorithms, with applications from criminal justice to materials discovery.

Sorelle is a Co-Founder and former Executive Committee Member of the ACM Conference on Fairness, Accountability, and Transparency (FAccT) as well as a former Program Committee Co-Chair of FAccT and FAT/ML. She has received grants for her work on fairness in machine learning, fairness and social networks, using interpretable machine learning techniques to inform scienfitic hypotheses, Responsible CS Education, and policy and discriminatory machine learning. Key papers include work on disparate impact in machine learning and on accelerating materials discovery with interpretable machine learning.

Before Haverford, Sorelle was a software engineer at Alphabet (formerly Google), where she worked in the X lab and in search infrastructure. She holds a Ph.D. in Computer Science from the University of Maryland, College Park, and a B.A. from Swarthmore College.

CV (pdf) Grants Papers Teaching

Grants

NSF IIS-1955321 (2020 - 2024): III: Medium: Collaborative Research: Evaluating and Maximizing Fairness in Information Flow on Networks. Suresh Venkatasubramanian, Aaron Clauset, Carlos Scheidegger, and Sorelle Friedler. $995,908. (Haverford portion: $128,670).

Mozilla Responsible Computer Science Challenge (2019 - 2020): Responsible Problem Solving: Focusing on the societal consequences of design choices in data structures and algorithms. Suresh Venkatasubramanian, Sorelle Friedler, and Seny Kamara. $150,000 (Haverford portion: $29,524).

DARPA Synergistic Discovery and Design (SD2) (2018 - 2021): TA2+TA3: Discovering Reactions and Uncovering Mechanisms of Hybrid Organohalide Perovskite Formation. Joshua Schrier, Sorelle Friedler, and Alexander Norquist. $3,604,943.

NSF DMR-1709351 (2017 - 2020): CDS&E: D3SC: The Dark Reaction Project: A machine-learning approach to exploring structural diversity in solid state synthesis. Joshua Schrier, Sorelle Friedler, and Alexander Norquist. $645,288.

NSF IIS-1633387 (2016 - 2019): BIGDATA: Collaborative Research: F: Algorithmic Fairness: A Systemic and Foundational Treatment of Nondiscriminatory Data Mining. Suresh Venkatasubramanian, danah boyd, and Sorelle Friedler. $953,432 (Haverford portion: $172,742).

Knight News Challenge Prototype Fund (2016): Could your data discriminate? Sorelle Friedler, Wilneida Negron, Surya Mattu, Suresh Venkatasubramanian. $35,000.

Data and Society Research Institute Fellow (2015 - 2016): Preventing Discrimination in Machine Learning: from theory to law and policy. $10,000.

NSF DMR-1307801 (2013 - 2016): The Dark Reaction Project: a machine learning approach to materials discovery. Joshua Schrier, Alexander Norquist, and Sorelle Friedler. $299,998.

Papers

Journal Papers

Venkateswaran Shekar, Gareth Nicholas, Mansoor Ani Najeeb, Margaret Zeile, Vincent Yu, Xiaorong Wang, Dylan Slack, Zhi Li, Philip W. Nega, Emory Chan, Alexander J. Norquist, Joshua Schrier, and Sorelle A. Friedler. Active Meta-Learning for Predicting and Selecting Perovskite Crystallization Experiments. The Journal of Chemical Physics, Feb. 14, 2022. [PDF | link]

Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. The (im)possibility of fairness: different value systems require different mechanisms for fair decision making. Communications of the ACM, April, 2021. [PDF | link]

Xiwen Jia, Oscar Huang, Allyson Lynch, Matthew Danielson, Immaculate Lang’at, Alexander Milder, Aaron Ruby, Hao Wang, Sorelle A. Friedler, Alexander J. Norquist, and Joshua Schrier. Anthropogenic biases in chemical reaction data hinder exploratory inorganic synthesis. Nature, 573: 251 - 255, Sept. 12, 2019. [PDF | link]

Harry Levin and Sorelle A. Friedler. Automated Congressional Redistricting. ACM Journal of Experimental Algorithmics, 24(1): 1-10, 2019. [PDF | link | code]

Philip Adler, Casey Falk, Sorelle A. Friedler, Tionney Nix, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing Black-box Models for Indirect Influence. Knowledge and Information Systems, 54(1): 95-122, 2018. [PDF | link | code]

Paul Raccuglia, Katherine C. Elbert, Philip D. F. Adler, Casey Falk, Malia B. Wenny, Aurelio Mollo, Matthias Zeller, Sorelle A. Friedler, Joshua Schrier, and Alexander J. Norquist. Machine-learning-assisted materials discovery using failed experiments. Nature, 533: 73 - 76, May 5, 2016. [PDF | link | project site]

Sorelle A. Friedler and David M. Mount. A Sensor-Based Framework for Kinetic Data Compression. Computational Geometry: Theory and Applications, 48(3): 147 - 168, March 2015. (doi: 10.1016/j.comgeo.2014.09.002) [PDF | link]

Sorelle A. Friedler and David M. Mount. Approximation algorithm for the kinetic robust k-center problem. Computational Geometry: Theory and Applications, 2010. (doi: 10.1016/j.comgeo.2010.01.001). [PDF (preprint) | link]

Sorelle A. Friedler, Yee Lin Tan, Nir J. Peer, and Ben Shneiderman. Enabling teachers to explore grade patterns to identify individual needs and promote fairer student assessment. Computers & Education, 51(4):1467-1485, December 2008. [PDF (preprint) | link] [code and help videos]

Peer-reviewed Conference Proceedings

Mohsen Abbasi, Calvin Barrett, Sorelle A. Friedler, Kristian Lum, Suresh Venkatasubramanian. Measuring and mitigating voting access disparities: a study of race and polling locations in Florida and North Carolina. Conference on Fairness, Accountability, and Transparency (FAccT), 2023. [PDF | link]

Ashkan Bashardoust, Sorelle A. Friedler, Carlos Scheidegger, Blair D. Sullivan and Suresh Venkatasubramanian. Reducing Access Disparities in Networks using Edge Augmentation. Conference on Fairness, Accountability, and Transparency (FAccT), 2023. [PDF | link]

Lydia Reader, Pegah Nokhiz, Cathleen Power, Neal Patwari, Suresh Venkatasubramanian, and Sorelle A. Friedler. Models for understanding and quantifying feedback in societal systems. Conference on Fairness, Accountability, and Transparency (FAccT), 2022. [PDF | link]

I. Elizabeth Kumar, Carlos Scheidegger, Suresh Venkatasubramanian, and Sorelle A. Friedler. Shapley Residuals: Quantifying the limits of the Shapley value for explanations. In Neural Information Processing Systems (NeurIPS), 2021. [PDF | link]

I. Elizabeth Kumar, Suresh Venkatasubramanian, Carlos Scheidegger, and Sorelle A. Friedler. Problems with Shapley-value-based explanations as feature importance measures. In International Conference on Machine Learning (ICML), 2020. [PDF | link]

Dylan Slack, Sorelle A. Friedler, and Emile Givental. Fairness Warnings and Fair-MAML: Learning Fairly with Minimal Data. In Conference on Fairness, Accountability, and Transparency (FAccT), 2020. [PDF | link]

Charles Marx, Richard Phillips, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Disentangling Influence: Using disentangled representations to audit model predictions. In Neural Information Processing Systems (NeurIPS), 2019. [PDF | link]

Benjamin Fish, Ashkan Bashardoust, danah boyd, Sorelle Friedler, Carlos Scheidegger and Suresh Venkatasubramanian. Gaps in Information Access in Social Networks. In The Web Conference (WWW), 2019. [PDF]

Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Fairness in representation: Quantifying stereotyping as a representational harm. In SIAM International Conference on Data Mining (SDM), 2019. [PDF]

Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2019. [PDF | code]

Andrew Selbst, danah boyd, Sorelle A. Friedler, Suresh Venkatasubramanian, and Janet A. Vertesi. Fairness and Abstraction in Sociotechnical Systems. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2019. [PDF | link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger, Suresh Venkatasubramanian. Decision Making with Limited Feedback: Error bounds for Recidivism Prediction and Predictive Policing. In Algorithmic Learning Theory (ALT) 2018. [PDF | link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Runaway Feedback Loops in Predictive Policing. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2018. [PDF | link]

Richard L. Phillips, Kyu Hyun Chang, and Sorelle A. Friedler. Interpretable Active Learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*), 2018. [PDF | link]

Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing Black-box Models for Indirect Influence. In Proceedings of the IEEE International Conference on Data Mining (ICDM), 2016. [PDF | code]

F. Betul Atalay, Sorelle A. Friedler, and Dianna Xu. Convex hull for probabilistic points. In Technical Papers of the 29th Conference on Graphics, Patterns and Images (SIBGRAPI '16), 2016. [PDF]

Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and Removing Disparate Impact. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015. [PDF | code]

Sorelle A. Friedler and David M. Mount. Spatio-temporal range searching over compressed kinetic sensor data. In Proc. of the European Symposium on Algorithms (ESA), pages 386-397, 2010. [PDF (preprint) | link] [TR]
     2nd Workshop on Massive Data Algorithmics, 2010 [PDF]
     Fall Workshop on Computational Geometry, 2009 [PDF]

Sorelle A. Friedler and David M. Mount. Compressing kinetic data from sensor networks. In Proc. of the 5th International Workshop on Algorithmic Aspects of Wireless Sensor Networks (AlgoSensors), pages 191 - 202, 2009. [PDF (preprint) | link] [TR]

Workshop Papers and Technical Reports

I. Elizabeth Kumar, Carlos Scheidegger, Suresh Venkatasubramanian, and Sorelle Friedler. Shapley Residuals: Quantifying the limits of the Shapley value for explanations. ICML Workshop on Workshop on Human Interpretability in Machine Learning (WHI), 2020. [link]

Dylan Slack, Sorelle Friedler and Emile Givental. Fairness Warnings. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Dylan Slack, Sorelle Friedler and Emile Givental. Fair Meta-Learning: Learning How to Learn Fairly. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Kadan Lottick, Silvia Susai, Sorelle Friedler, and Jonathan Wilson. Energy Usage Reports: Environmental awareness as part of algorithmic accountability. NeurIPS Workshop on Tackling Climate Change with Machine Learning, 2019. [link]

Charles Marx, Richard Phillips, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Disentangling Influence: Using disentangled representations to audit model predictions. arXiv:1906.08652, Jun. 20, 2019. [link]

Dylan Slack, Sorelle A. Friedler, Chitradeep Dutta Roy, and Carlos Scheidegger. Assessing the Local Interpretability of Machine Learning Models. NeurIPS Workshop on Human-Centric Machine Learning, 2019. [link]

Danielle Ensign, Sorelle A. Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Runaway Feedback Loops in Predictive Policing. Presented as a talk at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Aug. 14, 2017. [link]

Danielle Ensign, Sorelle Friedler, Scott Neville, Carlos Scheidegger and Suresh Venkatasubramanian. Decision Making with Limited Feedback: Error bounds for Recidivism Prediction and Predictive Policing. Presented as a poster at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Aug. 14, 2017. [PDF]

Richard L. Phillips, Kyu Hyun Chang, and Sorelle A. Friedler. Interpretable Active Learning. Presented at the ICML Workshop on Human Interpretability in Machine Learning, Aug. 10, 2017.

Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. On the (im)possibility of fairness. arXiv:1609.07236, Sept. 23, 2016. [link]

Nicholas Diakopoulos, Sorelle Friedler, Marcelo Arenas, Solon Barocas, Michael Hay, Bill Howe, HV Jagadish, Kris Unsworth, Arnaud Sahuguet, Suresh Venkatasubramanian, Christo Wilson, Cong Yu, and Bendert Zevenbergen. Principles for accountable algorithms and a social impact statement for algorithms. Dagstuhl working group write-up. July, 2016. [ PDF | link]

Ifeoma Ajunwa, Sorelle Friedler, Carlos E. Scheidegger, and Suresh Venkatasubramanian. Hiring by Algorithm: Predicting and Preventing Disparate Impact. Presented at the Yale Law School Information Society Project conference Unlocking the Black Box: The Promise and Limits of Algorithmic Accountability in the Professions, Apr. 2, 2016. [PDF]

Philip Adler, Casey Falk, Sorelle A. Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, Suresh Venkatasubramanian. Auditing Black-box Models by Obscuring Features. arXiv:1602.07043. [link]

Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. Certifying and Removing Disparate Impact. Presented at the Fairness, Accountability, and Transparency in Machine Learning Workshop, Dec. 12, 2014. [link]

F. Betul Atalay, Sorelle A. Friedler, and Dianna Xu. Probabilistic Kinetic Data Structures. Presented at the Fall Workshop on Computational Geometry, Oct. 25, 2013. [PDF | link]

Sorelle A. Friedler and David M. Mount. Realistic compression of kinetic sensor data. Technical Report CS-TR-4959, University of Maryland, College Park, 2010. [PDF | TR]

Thesis

Sorelle A. Friedler. Geometric Algorithms for Objects in Motion. Dissertation committee: Prof. David Mount (chair), Prof. William Gasarch, Prof. Samir Khuller, Prof. Steven Selden, Prof. Amitabh Varshney. Defense date: July 30, 2010. [PDF] [presentation]

Patents

Mohammed Waleed Kadous, Isaac Richard Taylor, Cedric Dupont, Brian Patrick Williams, Sorelle Alaina Friedler. Permissions based on wireless network data. US 20130244684 A1. Publication date: Sep. 19, 2013.

Sorelle Alaina Friedler, Mohammed Waleed Kadous, Andrew Lookingbill. Position indication controls for device locations. US 20130131973 A1 (also WO 2013078125 A1). Publication date: May 23, 2013.

Regularly Taught Classes

Haverford

CS 104: Topics in Introductory Programming
CS 106: Introduction to Data Structures
CS 340: Analysis of Algorithms
CS 399: Senior Thesis

Past Classes

Haverford

CS 101: Fluency with Information Technology
CS 105: Introduction to Computer Science
CS 207: Data Science and Visualization
CS 395: Mobile Development for Social Change

University of Maryland, College Park

Design and Analysis of Computer Algorithms, Summer 2009
Organization of Programming Languages, Summer 2007
Computer Organization (TA), Spring 2006
Introduction to Low-Level Programming Concepts (TA), Fall 2005