Earlier this month, our research team attended the inaugural IEEE Conference on Secure and Trustworthy Machine Learning (or SaTML for short), where we presented our paper, Tensions Between the Proxies of Human Values in AI. This is a first-of-its-kind conference, focusing on traditional responsible AI topics like fairness and explainability, while also welcoming work on robustness and adversarial machine learning. Many industry practitioners attended, such as those from Nvidia and Intuit, as well as more industry-focused academics, giving this conference a more practical feel without being dominated by Big Tech companies.
In this blog post, we reflect on the major theme from SaTML: we need to be careful about how we deploy and utilize machine learning solutions. We’ll highlight talks that particularly resonated with us and made us think about how to be more cautious, but note that most of the talks and the air of conversation were around this idea. As a community, we need to be more careful about how we do our research, how we deploy models, and how we evaluate consequences.
Patrick Altmeyer gave a compelling talk on his team’s paper, Endogenous Macrodynamics in Algorithmic Recourse. In their paper, Altmeyer et al. come up with a simulation, where a bank may use a counterfactual explanation method to perform some sort of algorithmic recourse on an algorithm that determines whether someone gets a loan or not. This would cause a domain shift—several individuals will go over the decision boundary and get loans. But Altmeyer et al. take this a step further. What if now you train that algorithm with those data points that shifted the boundary? Then you would get a model shift—the model’s decision boundary would move to accommodate more of the data points. Altmeyer illustrates in his talk that as this feedback loop continues, the model’s accuracy continues to decline. For the bank, however, this decline translates to risk of individual’s defaulting on loans. Thus, the question is, who should take on that risk?
Altmeyer et al. offer a regularizer that takes into account the external cost (in this case, the uncertainty that would be developed) to better the algorithmic recourse, but I think this hits something deeper. As we continue to develop new explanation techniques, such as our FastCFE algorithm, should we be concerned with the feedback loops they may generate? This question is something that has been explored before in the fairness space, starting with Delayed Impact of Fair Machine Learning, which has shown similar results.
We even discuss this in our Tensions paper, where we argue that the lack of contextual information combined with domain-agnostic definitions of our human values is inherently flawed. Without taking the context into consideration, we cannot appropriately evaluate the consequences of our deployments. This especially extends to the feedback loops generated, something most research does not take into account.
Anna Kawakami presented her paper, led by Amanda Coston, SoK: A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms. Coston et al. argue that the alignment of our decision making algorithms and the real world is one of the issues to watch, and in my opinion this is even a bigger issue. They even cite that many states in the U.S. have attempted to deploy machine learning algorithms to help, and decide to retire them soon after because they aren’t seeing the results they want. This is similar to research done by Virginia Eubanks, where well-meaning organizations have deployed models that actively harm the populations they are trying to serve. Recent research endeavors have even started exploring the mismatch between optimization procedures in classification models, primarily focused on predictive performance, and the actual decision making tasks they are deployed to do, arguing that this mismatch leads to illegitimate deployments that should have never happened.
In my opinion, this is one of the root issues of feedback loops. If we don’t have proper proxies, understandings of those proxies, and the contexts they will be utilized in, how can we hope to create models that are useful? I use the term useful here intentionally because, as has been famously said many times, “all models are bad, but some are useful.” The same goes with proxy variables.
I think the unasked question at SaTML, and honestly at most technology-oriented conferences, is: should we build these technologies? Timnit Gebru tackled this question directly during her keynote speech with a resounding no, pointing a finger directly towards those who think we should even try to achieve AGI. Because, as Gebru put it, even giving the perception that a form of AGI has been achieved can be harmful to us as humans. Although models like ChatGPT have achieved “human-like” performance, Jacob Steinhardt’s talk showed how aligning to what we think is human can be flawed, and those flaws are starting to show (thinking about you, romantic Bing).
SaTML began with Zico Kolter doing a retrospective on the past 5 years of adversarial machine learning. His talk was both sobering and informative, but I want to focus on one aspect of it. Kolter presented an argument that he has heard many times: we need to redefine what we mean by robust machine learning. Instead of defining robustness as an ability to handle worst-case scenarios, we should define it as an ability to handle distribution shifts, generalizability, and other things that practitioners care about. But then he shows the proverbial picture of a pig that got classified as an airplane, and tries to convince the audience that because technology cannot do this very basic task of identifying an object, even with an adversarial attack, then we should try to make sure it can do this task. I want to push back on this. Do we really want technology to mimic humans this much? Is there positive use in technology being able to do, quite literally, everything that makes us human? Or should we restrict ourselves to technologies that serve specific, meaningful purposes that actually improve our lives and outcomes? My opinion is that if we want to create responsible AI, we should be intentional about our needs and build specific technologies around them.
SHARE
