Skip to content

Code for "Human-Aligned Calibration for AI-Assisted Decision Making", NeurIPS 2023

License

Notifications You must be signed in to change notification settings

Networks-Learning/human-aligned-calibration

Repository files navigation

human-aligned-calibration

Install and download prerequisites

Install required packages with pip install -r requirements.txt

Download Human-AI Interactions Dataset and dependency haiid.py (used to load dataset in hac.py) from https://github.com/kailas-v/human-ai-interactions into the directory ./human_ai_interactions_data :

├── human_ai_interactions_data
│   ├── haiid.py
│   ├── haiid_dataset.csv

Repository Structure

  • hac.py: code to pre-process the dataset and run experiment on "Art", "Sarcasm", "Cities" and "Census" tasks
  • ./plots: contains sub-directories ./barplot, ./roc and ./hist where corresponding plots are saved to

Running the Experiment

  • Run:
        python3 hac.py
    
  • Output:
    • Table of metrics for all tasks printed to console
    • Generated plots saved under ./plots sub-directories