Data activist Renée Cummings says there’s an important question to ask when making responsible AI: ‘Where did this data come from?’

Jay Ganglani

One of AI’s thorniest issues is ensuring the new technology doesn’t end up reflecting all the same biases of its human creators have. Generative AI, trained on massive amounts of data, is compounding the problem, rather than mitigating it, as models deliver answers that align with pervasive prejudices about different races and genders.<div class="paywall-selector"></div><div class="paywall">



Data activist Renée Cummings, at Fortune’s Brainstorm Design event in Macau on Thursday, argued that the core of the issue is data. “We know that AI is a revolutionary technology, we know that AI is a radical technology, but we have to understand that there is no AI without data,” she said. 



Cummings, a professor of practice in data science at the University of Virginia, examines ethical risks in artificial intelligence and studies how to build a responsible and sustainable AI that can benefit everyone in society, regardless of background.&nbsp;



For example, a recent <a href="https://documents.un.org/doc/undoc/gen/g24/084/20/pdf/g2408420.pdf">United Nations study</a> recently pointed to predictive policing as an example of how racial biases are reproduced through technology. This practice involves making judgments regarding who will commit future crimes and where these incidents may take place. The study warns that predictive policing could further add to problems regarding over-policing communities based on race and ethnicity. 



“We know that data creates very unique challenges when it comes to bias, when it comes to discrimination, when it comes to systemic challenges,” Cummings said.&nbsp;



“It&#8217;s so important when we&#8217;re designing to ask ourselves, where did this data come from?” she asked. &#8220;Sometimes we’re using data that is being duplicated in very weird ways. Data that’s actually missing critical pieces of the stories that need to be told.”



That means AI designers need to have a heightened sense of “ethical vigilance,” she explained. “We have a responsibility to ensure that we bring a framework to&#8230;audit the ways in which we are building models, the ways in which we are building algorithms,” she said. 



AI also brings its share of environmental concerns. The metals used in AI hardware, for instance, can lead to both pollution and soil erosion. “We speak about building a sustainable and resilient future using this technology, but we&#8217;ve got to think about the ways in which the environment is also being impacted.”



Due diligence can help AI designers work around these issues. They could design new models while ensuring that concepts such as equity, access, and fairness are respected, yet also in a manner that doesn’t undermine the accuracy, or the validity of data, she said



“How involved are communities in the process? It’s very critical the ways in which we engage communities to design, because we’re realizing that many of our data sets…are creating very critical risks,” she explained.&nbsp;



AI does have the potential to do “extraordinary things,” Cummings noted. AI tools have led to increased efficiency, reduced costs, and greater convenience. But that only heightens the need to make sure the new technology works in a responsible way.



If AI designers don’t take data equity into account, then “what we are going to create are systems that not only undermine our own progress, but systems that could create situations that undermine the ways in which we move into the future together,” she warned.&nbsp;</div>

“Sometimes we’re using...data that’s actually missing critical pieces of the stories that need to be told," Cummings said at Fortune Brainstorm Design.

Designers must have 'ethical vigilance' to make responsible AI: data activist Renee Cummings

F-Conference

Brainstorm Design

Conferences

“Sometimes we’re using...data that’s actually missing critical pieces of the stories that need to be told," Cummings said at Fortune Brainstorm Design on Thursday.

Designers must have 'ethical vigilance' to make responsible AI: data activist Renee Cummings | Fortune Asia

Fortune Editors

An ‘inverse cyborg’ named Botto is using AI and blockchain technology to create and auction its own art. See how it works

/asia/2025/01/06/botto-artist-creates-sells-art-ai-blockchain-mario-klingemann-brainstorm-design/

Nicholas Gordon

Inside the secret history of Design for Good, an alliance between companies with $4 trillion in market cap to devise solutions to the world’s biggest problems

/asia/2024/12/13/story-behind-design-for-good-alliance-solve-world-problems/

General Mills’ top designer refreshed the Pillsbury Doughboy and risked defying its No. 1 rule: ‘Don’t break the billion-dollar brand’

/asia/2024/12/11/general-mills-poppin-fresh-pillsbury-doughboy-teman-evans-brainstorm-design/

Lionel Lim

Founders must embrace ‘design with a capital D’ if they want to convince investors, says partner at Alphabet’s GV

/asia/2024/12/10/founders-embrace-design-capital-d-convince-investors-partner-alphabet-gv/

Beats by Dre designer Robert Brunner’s advice for creatives trying to make themselves heard: ‘Go take the comedy class’

/asia/2024/12/09/robert-brunner-beats-by-dre-designer-advice-take-a-comedy-class/

/asia/2024/12/06/renee-cummings-responsible-ethical-ai-brainstorm-design/

Uncategorized

homepage

Fortune Asia

Finance

Leadership

Tech

Irina Ivanova

Elon Musk's DOGE slapped a $1 limit on government credit cards and now workers say they can’t do their jobs

https://fortune.com/2025/03/05/doge-federal-credit-card-elon-musk/?itm_source=parsely-api

Emma Burleigh

Melinda French Gates got her start at Microsoft because an IBM hiring manager told her to turn down their job offer—‘It dumbfounded me’

https://fortune.com/2025/03/05/melinda-french-gates-career-advice-microsoft-ibm/?itm_source=parsely-api

Christiaan Hetzner

As Tesla stock plummets, boardroom insider and Elon Musk supporter Robyn Denholm piles on the pain

https://fortune.com/2025/03/04/tesla-elon-musk-chair-robyn-denholm-stock-sale/?itm_source=parsely-api

Diane Brady

Canadian CEO on tariffs: ‘This game of chicken has to stop’

https://fortune.com/2025/03/04/this-game-of-chicken-has-to-stop/?itm_source=parsely-api

Amazon CEO says he's cutting middle managers because they want to 'put their fingerprint on everything.' That's music to the ears of Gen Z

https://fortune.com/2025/03/04/amazon-ceo-andy-jassy-middle-managers-rto-gen-z/?itm_source=parsely-api

Eleanor Pringle

Trump may roll back Mexico and Canada tariffs as soon as today, but conflicting messages are giving consumers and Wall Street whiplash

https://fortune.com/2025/03/05/trump-mexico-canada-tariff-roll-back-howard-lutnick/?itm_source=parsely-api