Week 1: Introduction to Human-AI Interaction

August 27th        Course Introduction


New assignment: Assignment #0 - DUE BEFORE 2nd class

August 29th        Practical ML


        Due by class:  Assignment #0

New assignment:  Assignment #1, due Sunday September 8th by 11:59pm         


Week 2: Perspectives on Human-AI Interaction

September 3rd   A History of Artificial Intelligence



  1. Ben Schneiderman and Pattie Maes. Direct Manipulation vs. Interface Agents. interactions 1997.
  2. Eric Horvitz. Principles of Mixed-Initiative Interaction. CHI 1999 (in Pittsburgh 😍)
  3. (grad only)  Man-Computer Symbiosis J. C. R. Licklider IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4-11, March 1960(!!)

Quiz 2: CMU Students | General public (ungraded)

PLEASE SIGN UP: for being a discussant. If you are a discussant for next week, please plan to come to office hours this week.


September 5th   A History of Humans Interacting with AI + AI vs. IA


        Quiz 3: CMU students | General public (ungraded)


Due by class:        reading reflections on piazza

Week 3: Designing AI/ML User Experience

September 10th   Matchmaking Needs and Risks for Adding AI/ML


        Design workshop

  1. Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S. T., Bennett, P., Inkpen, K., Teevan, J., Kikin-Gil, R., and Horvitz, E. (2019) Guidelines for Human-AI Interaction (CHI 2019). [pdf]
  2. Amershi, Saleema, et al. "Software engineering for machine learning: a case study." Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice. IEEE Press, 2019. [pdf]
  3. (grad only) Kay, Patel, and Kientz. How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy. CHI 2015.

Worksheets for design workshop:

  1. AI and mental models
  2. Designing AI for different stakes
  3. AI and coadaptation

        New assignment: Assignment 2 Loan application front-end

September 12th   Communicating Predictions & Recommendations with Users


Worksheets for design workshop:

  1. Feedback in AI worksheet

        Discussion panel

Week 4: Designing for failure

September 17th   Failure & Feedback with Users


  1. Kocielnik, R., Amershi, S., and Bennett, P. (2019) Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-User Expectations of AI Systems. (CHI 2019). [pdf]
  2. Cai, Carrie Jun, et al. "``Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making." (2019). [pdf]
  3. Can we make Montreal’s buses more predictable? No. But machines can. (What? A Medium blog as academic reading?!?!)


September 19th   Product Workshop Day

        Make one new product based on prompt. Group activity.  


Discussion panel

Week 5: Data & Knowledge. Where does it come from? Who owns it?

September 24th: Why would people give you data anyway? Data ethics and laws



  1. Jeffrey Bigham, Michael Bernstein, and Eytan Adar. Human-Computer Interaction and Collective Intelligence
  2. Zimmerman, J., Tomasic, A., Garrod, C., Yoo, D., Hiruncharoenvate, C., Aziz, R., ... & Steinfeld, A. (2011, May). Field trial of tiramisu: crowd-sourcing bus arrival times to spur co-design. CHI 2011. [pdf]

September 26th: Using human-centric data in an ML pipeline (wait, should it even be a pipeline??)



Assignment 2 due on Sep 29

Week 6: Data Visualization & Data Communication

October 1st: Visualizations to improve human-AI interaction



  1. Carrie Cai et al (2019) Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making 
  2. Matthew Kay, Tara Kola, Jessica R. Hullman, Sean Munson (2016) When (ish) is My Bus? User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

October 3rd


New assignment: Assignment 3 visualization -- delayed! Sorry!

Guest lecture: Adam Perer: Visualization in AI (Slides)


Week 7: Interpreting and Explaining Algorithms

October 8th: How does telling people how an algorithm works change their experience?


  1. Intelligible Artificial Intelligence [pdf]
  2. Why these Explanations? Selecting IntelligibilityTypes for Explanation Goals [pdf]
  3. [grad only] The Mythos of Model Interpretability [pdf]


October 10th



Assignment 3 due on October 20th

Week 8 - 9: AI Ethics, Fairness, Social Acceptability, and Trust

October 15th  

Please take this ungraded pre-class survey in the first 10 min of class.


Reading: Medical devices: the Therac-25 by Nancy Leveson (This is the only reading for the week because a) it is somewhat longer, and b) I want you to think about it carefully.

[Warning: this reading has certain graphic, but textual, descriptions of the results of accidental radiation exposure during clinical therapy. ]

This reading is ostensibly not about AI. But it may be one that allows you to draw many parallels to our AI space. When you read the paper, consider these questions:

- What parallels can you draw from the reading to the design of human-AI systems? (For instance, the Tyler incident is caused by a "race" condition -- a hard-to-find bug that is the result of exact timing of operations, which are largely determined by chance and so hard to inspect ahead of time. AI systems can similarly rely on things that are hard to inspect ahead of time.)

- What are the roles that people played in this story in making the error, diagnosing it, and fixing it?

- Who are the heroes of this story? Who are the villains? Is it useful to think of them this way?

- What roles did users play in this story?

- Was the response from the manufacturer ethical? What about the regulatory governmental agencies, and the attending doctors?

- If something similar were to happen today with an AI-infused system, what would you expect to go differently?

October 17th   Guest Lecture Ken Holstein  Improving fairness in ML systems: What do industry practitioners need?

Slides and other information

A paper mentioned during class on Thursday, regarding limits/pitfalls of post-hoc, algorithmic de-biasing:



Assignment 3 is due Oct 20th!

Week 9

October 22nd



  1. Disparate Interactions [pdf] How humans interact with risk predictions and how it leads to differences in fairness.
  2. Do artifacts have politics? [pdf]
  3. [Optional] Intelligible Models for HealthCare: Predicting PneumoniaRisk and Hospital 30-day Readmission [pdf] No reflection to be submitted!

October 24th: class is canceled for the HCII 25th celebrations

        No Quiz

No Discussion panel

Week 10: Human in the loop with AI/ML & Recommendations

October 29th  

Guest lecture Julian Ramos: personalized context aware health interventions.

        New assignment: Assignment 4 Make a chatbot 



The focus of this week is to rethink what it means for a human to be “in the loop”. The readings reflect this focus. If you’re interested in the more traditional view of humans “in the loop”, make sure you read the grad reading.

  1. Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, Eric Horvitz (2019): Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
  2. Sharon Zhou, Melissa Valentine, Michael S. Bernstein (2018) In Search of the Dream Team: Temporally Constrained Multi-Armed Bandits forIdentifying Effective Team Structures
  3. [grad only] Ting-Hao (Kenneth) Huang, Joseph Chee Chang, and Jeffrey P. Bigham Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time

October 31st



Discussion Panel

Week 11: Natural Language & Speech Applications

November 5th


  1. [optional, no reflection needed] ELIZA—a computer program for the study of natural language communication between man and machine [pdf]
  2. [required, but no reflection needed] How to build a neural network
  3. Voice Interfaces in Everyday Life [pdf]
  4. Designing a Workflow-Based Scheduling Agent with Humans in the Loop [pdf]

November 7th



Assignment 4 due on November 10th

Week 12: Vision, Images, & Art

November 12th


November 14th


New assignment: Assignment 5 Vision with GANs

New Final Project Released: Make something cool or write something awesome!

Week 13: Special Topic/Application Area (class polling to decide)

November 19th



        Assignment #5 prompt & code materials

November 21st

        GANs + Special Topics Slides

Week 14: Break

November 26th        Thanksgiving Break; no class or normal office hours

November 29th        Thanksgiving Break; no class or normal office hours (🦃 or 🥧?)

Week 15: Final Projects & Special Topic (class polling to decide)

December 3rd:   Project presentations

December 5th:   Closing


Final Project due on Dec 8th