A conversation about the current inability of artificial intelligence to forget and what this means for the right to privacy and user agency. Miguel Luengo-Oroz, AI expert and social entrepreneur, speaks with Ashoka’s Hanae Baruchel. (Full biography below.)
Hana Baruchel: ChatGPT and other generative artificial intelligence (AI) apps have been making headlines lately for their unprecedented ability to have cohesive conversations with humans. You recently had an interaction yourself.
Founder and CEO of Spotlab
Miguel Luengo Oroz: Yes, I asked GPT-3 “Can neural networks forget?” and the answer was that “artificial intelligence systems, such as neural networks, cannot forget in the same way humans do. The network can adjust its weights to better fit the new data, which can lead it to make different predictions for the same input. However, this is not the same as forgetting in the sense that the network can still access all the information it previously learned, it just gives higher priority to the new information.”
Baruchel: This inability to forget does not get much attention and this worries you. Why?
Luengo Oroz: One of the core principles of online privacy regulations, such as the European General Data Protection Regulation (GDPR), is the idea that the data I produce belongs to me and that a company can only use it if I allow it. This means that I can always withdraw my consent and request my data back. I can even ask for the right to be forgotten. AI algorithms are partially trained on user data, yet almost none of the guidelines, frameworks, and regulatory proposals from governments and private sector companies explicitly focus on building AI models that can be left untrained. We have no way to undo the changes caused in their system by a single data point at the request of a data owner.
baruchel: So users should have the option to say “Stop using the AI model trained on my data”?
Luengo Oroz: Precisely. Let’s give AIs the ability to forget. Think of it like AI’s Ctrl-Z button. Let’s say my photo was used to train an AI model that recognizes blue-eyed people and I don’t give permission anymore, or never have. I should be able to ask the AI model to behave as if my photo was never included in the training dataset. In this way, my data would not contribute to fine-tuning the model’s internal parameters. In the end, this may not affect the AI that much, as my photo is unlikely to have made a substantial contribution on its own. But we can also imagine a case where all blue-eyed people ask that their data not affect the algorithm, making it impossible to recognize people with blue eyes. In another example, let’s imagine I’m Vincent Van Gogh and I don’t want my art to be included in an algorithm’s training dataset. Then if someone asks the machine to paint a dog in the style of Vincent Van Gogh, it would not be able to perform that task.
Baruchel: How would this work?
Luengo Oroz: In artificial neural networks, every time a data point is used to train an AI model, the way each artificial neuron behaves changes slightly. One way to remove this contribution is to completely retrain the AI model without the affected data point. But this is not a practical solution because it requires too much computational power and too many resources. Instead, we need to find a technical solution that reverses the influence of this data point and changes the final AI model without having to train it from scratch.
Baruchel: Do you see people in the AI community pursuing such ideas?
Luengo Oroz: So far, the AI community has done little specific research on the idea of detraining neural networks, but I’m sure smart solutions will come soon. There are adjacent ideas to take inspiration from, such as the concept of “catastrophic forgetting,” the tendency for AI models to forget previously learned information when learning new information. The big picture of what I’m suggesting here is that we build neural networks that aren’t just sponges immortalizing whatever data they soak up, like stochastic parrots. We need to build dynamic entities that adapt to and learn from the data sets they are allowed to use.
Baruchel: Aside from the right to be forgotten, you suggest that this kind of traceability can also bring great innovations when it comes to digital property rights.
Luengo Oroz: If we could trace what user data contributed to training specific AI models, it could become a mechanism to compensate people for their contributions. As I wrote in 2019, we could come up with a kind of Spotify model that rewards people with royalties every time someone uses an AI trained on their data. In the future, this type of solution could ease the strained relationship between the creative industries and generative AI tools such as DALL-E or GPT-3. It could also lay the groundwork for concepts like Forgetful advertising, a new ethical digital advertising model that would purposefully avoid storing personal behavioral data. Perhaps the future of AI isn’t just about learning everything — the bigger the data set and the bigger the AI model, the better — but about building AI systems that can learn and forget as humanity wants and needs.
Dr. Miguel Luengo-Oroz is a scientist and entrepreneur with a passion for inventing and building technology and innovation for social impact. As the former first chief data scientist at the United Nations, Miguel pioneered the use of artificial intelligence for sustainable development and humanitarian action. Miguel is the founder and CEO of the social enterprise Spotlab, a digital health platform that leverages the best AI and mobile technologies for clinical research and universal access to diagnosis. Over the past ten years, Miguel has built teams worldwide that bring AI to operations and policy in areas such as poverty, food security, refugees and migrants, conflict prevention, human rights, economic development, gender, hate speech, privacy and climate change. He is the inventor of Malariaspot.org – video games for collaborative image analysis of malaria – and is affiliated with the Universidad Politécnica de Madrid. He became an Ashoka Fellow in 2013.
Follow Next Nu/Tech & Humanity for more information on what works and what’s next.
