Liwaiwai Liwaiwai
  • /
  • Artificial Intelligence
  • Machine Learning
  • Robotics
  • Engineering
    • Architecture
    • Design
    • Software
    • Hybrid Cloud
    • Data
  • Learning
  • About
  • /
  • Artificial Intelligence
  • Machine Learning
  • Robotics
  • Engineering
    • Architecture
    • Design
    • Software
    • Hybrid Cloud
    • Data
  • Learning
  • About
Liwaiwai Liwaiwai
  • /
  • Artificial Intelligence
  • Machine Learning
  • Robotics
  • Engineering
    • Architecture
    • Design
    • Software
    • Hybrid Cloud
    • Data
  • Learning
  • About
  • Artificial Intelligence
  • Research

IBM Researchers Check AI Bias With Counterfactual Text

  • February 9, 2021
  • liwaiwai.com

From recruitment to credit risk apps, or use in the healthcare and criminal justice system, AI unreliability has been giving researchers headaches for years.

We’ve tackled the issue by flipping it upside down.


Partner with liwaiwai.com
for your next big idea.
Let us know here.



From our partners:

CITI.IO :: Business. Institutions. Society. Global Political Economy.
CYBERPOGO.COM :: For the Arts, Sciences, and Technology.
DADAHACKS.COM :: Parenting For The Rest Of Us.
ZEDISTA.COM :: Entertainment. Sports. Culture. Escape.
TAKUMAKU.COM :: For The Hearth And Home.
ASTER.CLOUD :: From The Cloud And Beyond.
LIWAIWAI.COM :: Intelligence, Inside and Outside.
GLOBALCLOUDPLATFORMS.COM :: For The World's Computing Needs.
FIREGULAMAN.COM :: For The Fire In The Belly Of The Coder.
ASTERCASTER.COM :: Supra Astra. Beyond The Stars.
BARTDAY.COM :: Prosperity For Everyone.

Our team has developed an AI that verifies other AIs’ “fairness” by generating a set of counterfactual text samples and testing machine learning systems without supervision.

In our recent paper, “Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text” accepted to AAAI 2021, we describe how our software, dubbed GYC, generates test cases to check the reliability of AI models. GYC can evaluate other AIs for accuracy, gender bias, and run a sensitivity analysis, as well as check a model’s adversarial robustness and its ability to cope with spurious correlations. It also verifies machine learning and natural language systems for trustworthiness.

 

What is counterfactual text, anyway?

Any text is a description of a scenario or a setting — and counterfactual text is a synthetically generated one that an AI model is forced to treat differently.

One well-known way to generate counterfactual texts is by using pre-defined templates and dictionaries — this is how, for instance, Checklist works. But in that case, the generated set of counterfactual samples is very rigid. GYC, instead, generates such samples in an unsupervised way so that they appear in user inputs as plausible, diverse, goal-oriented, and effective.

To create the AI, we used different loss functions to make sure that the desired properties of counterfactuals are preserved during the counterfactual generation. To guide the generation, we used the GPT-2 decoder, a tool in data science that relies on specific proximity and diversity constraints to change the input sentence in a variety of ways.

Read More  AI, Blockchain, and Healthcare: The Three Musketeers

The main goal of our AI is to drive generation around a specific condition, such as sentiment. To enforce the condition, we assume that we have access to a function for sentiment that takes the text and returns the probability of the text being positive or negative. This function could either be available to us openly or as a black box, with hidden contents. We propose different ways to deal with “black-box” and “white-box” access to the condition model.

 

So how does it work?

Consider the text “my boss is a man.” A counterfactual text could then be “my boss is a woman.” A hypothetical sentiment model would give a “positive” label to the first statement and a “negative” to the second one. GYC’s counterfactual text tests the reliability of the AI that produced the original text by doing this intervention.

The system’s output indicates that after GYC changes “man” to “woman,” the sentiment exhibited by the model changes. Typically, for a condition model, changing some minimal part of the text — in this case, gender — shouldn’t impact the output sentiment label at all. But it does — and that’s where our AI comes in. Counterfactual samples can be fed as training data for data augmentation algorithms, and used to de-bias the underlying sentiment model.

Having ran our experiments on three datasets, we observed that GYC generates a high label-flip score in counterfactuals belonging to a different class than the input sentence. GYC does this by maintaining diversity and by preserving semantic content and syntactic structure of the input sentence.

Read More  Addressing Climate Risks In Africa With AI And Hybrid Cloud

For example, consider as text input this named-entity recognition (NER) model: “My friend lives in beautiful London.” GYC could then generate high quality counterfactual samples, such as “My friend lives in majestic downtown Chicago” or “My friend lives in gorgeous London” or “My friend lives in the city of New Orleans.”

This means that GYC can generate variations of the “location” tag by generating a diverse set of counterfactual samples. With these samples, it’s possible to check AI reliability by analyzing the difference in behavior of a given model on input and the counterfactual set. Samples like that could be training data for de-biasing any model that differentiates on the basis of location.

GYC is the first method that generates test cases by changing multiple elements in text without any rule-based cues. The research is still ongoing and we are now trying to improve the reconstruction step — currently expensive to run for sentences longer than about 15 words. We are also working on getting GYC to generate counterfactuals using multiple condition models, which should significantly improve the automatic counterfactual generation.

 

Where could GYC be used?

Our GYC model could help test natural language processing models to perform a behavioral check. Such test-cases complement the traditional test cases designed by software engineers, and seem to be relevant with the increased adoption of NLP algorithms. GYC can be easily adapted to test any classification model, even with a black-box access to it. One can plug any score function and generate test cases around a specific condition.

Read More  The Evolution of AI From Programmed to Self-Learning Systems

GYC could also be beneficial for data augmentation and counterfactual logit pairing algorithms that require counterfactual text samples of the training data to de-bias language models for sentiment. Such samples should satisfy a specific condition, for example the presence of a protected attribute like gender, age or race. While these techniques claim to be highly successful, in reality getting enough data for a corresponding protected attribute is tricky. GYC could boost the performance of these algorithms by generating counterfactual text samples with high generation quality.

Finally, models that require explainability could benefit from our research as well. Blindly following the decisions of AI models has triggered issues with AI fairness, reliability and privacy — leading to the emergence of explainability in AI. GYC could generate textual explanations on a given input and a given model, to identify and fix ethical issues in AI models.

IBM Research’s Inkit Padhi, Naveen Panwar, Diptikalyan Saha also contributed to this article.

IBM Research AI is proudly sponsoring AAAI2021 as a Platinum Sponsor. We will present 40 main track papers, in addition to at least seven workshop papers, 10 demos, four IAAI papers, and one tutorial. IBM Research AI is also co-organizing three workshops. We hope you can join us from February 2-9 to learn more about our research. To view our full presence at AAAI 2021, visit here.

 

By Nishtha Madaan
Source IBM Research


For enquiries, product placements, sponsorships, and collaborations, connect with us at [email protected]. We'd love to hear from you!

Our humans need coffee too! Your support is highly appreciated, thank you!

liwaiwai.com

Related Topics
  • Bias
  • GYC
  • IBM
  • IBM Research AI
  • Language
You May Also Like
View Post
  • Artificial Intelligence
  • Technology

NASA’s Mars Rovers Could Inspire A More Ethical Future For AI

  • September 26, 2023
View Post
  • Artificial Intelligence
  • Platforms

Oracle CloudWorld 2023: 6 Key Takeaways From The Big Annual Event

  • September 25, 2023
View Post
  • Artificial Intelligence

3 Ways AI Can Help Communities Adapt To Climate Change In Africa

  • September 25, 2023
Robotic Hand | Lights
View Post
  • Artificial Intelligence
  • Technology

Nvidia H100 Tensor Core GPUs Come To Oracle Cloud

  • September 24, 2023
View Post
  • Artificial Intelligence
  • Engineering
  • Technology

AI-Driven Tool Makes It Easy To Personalize 3D-Printable Models

  • September 22, 2023
View Post
  • Artificial Intelligence
  • Data

Applying Generative AI To Product Design With BigQuery DataFrames

  • September 21, 2023
View Post
  • Artificial Intelligence
  • Platforms

Combining AI With A Trusted Data Approach On IBM Power To Fuel Business Outcomes

  • September 21, 2023
Microsoft and Adobe
View Post
  • Artificial Intelligence
  • Machine Learning
  • Platforms

Microsoft And Adobe Partner To Deliver Cost Savings And Business Benefits

  • September 21, 2023
A Field Guide To A.I.
Navigate the complexities of Artificial Intelligence and unlock new perspectives in this must-have guide.
Now available in print and ebook.

charity-water



Stay Connected!
LATEST
  • 1
    NASA’s Mars Rovers Could Inspire A More Ethical Future For AI
    • September 26, 2023
  • 2
    Oracle CloudWorld 2023: 6 Key Takeaways From The Big Annual Event
    • September 25, 2023
  • 3
    3 Ways AI Can Help Communities Adapt To Climate Change In Africa
    • September 25, 2023
  • Robotic Hand | Lights 4
    Nvidia H100 Tensor Core GPUs Come To Oracle Cloud
    • September 24, 2023
  • 5
    AI-Driven Tool Makes It Easy To Personalize 3D-Printable Models
    • September 22, 2023
  • 6
    Applying Generative AI To Product Design With BigQuery DataFrames
    • September 21, 2023
  • 7
    Combining AI With A Trusted Data Approach On IBM Power To Fuel Business Outcomes
    • September 21, 2023
  • Microsoft and Adobe 8
    Microsoft And Adobe Partner To Deliver Cost Savings And Business Benefits
    • September 21, 2023
  • Coffee | Laptop | Notebook | Work 9
    First HP Work Relationship Index Shows Majority of People Worldwide Have an Unhealthy Relationship with Work
    • September 20, 2023
  • 10
    Huawei Connect 2023: Accelerating Intelligence For Shared Success
    • September 20, 2023

about
About
Hello World!

We are liwaiwai.com. Created by programmers for programmers.

Our site aims to provide materials, guides, programming how-tos, and resources relating to artificial intelligence, machine learning and the likes.

We would like to hear from you.

If you have any questions, enquiries or would like to sponsor content, kindly reach out to us at:

[email protected]

Live long & prosper!
Most Popular
  • Intel Innovation 1
    Intel Innovation 2023
    • September 15, 2023
  • 2
    Microsoft And Oracle Expand Partnership To Deliver Oracle Database Services On Oracle Cloud Infrastructure In Microsoft Azure
    • September 14, 2023
  • 3
    Real-Time Ubuntu Is Now Available In AWS Marketplace
    • September 12, 2023
  • 4
    IBM Brings Watsonx To ESPN Fantasy Football With New Waiver Grades And Trade Grades
    • September 13, 2023
  • 5
    Document AI Workbench Is Now Powered By Generative AI To Structure Document Data Faster
    • September 15, 2023
  • /
  • Artificial Intelligence
  • Explore
  • About
  • Contact Us

Input your search keywords and press Enter.