Schedule

Week 1: Introduction to Human-AI Interaction

August 27th        Course Introduction

Slides

New assignment: Assignment #0 - DUE BEFORE 2nd class

August 29th        Practical ML

Slides

        Due by class:  Assignment #0

New assignment:  Assignment #1, due Sunday September 8th by 11:59pm         

        

Week 2: Perspectives on Human-AI Interaction

September 3rd   A History of Artificial Intelligence

Slides

Readings:

  1. Ben Schneiderman and Pattie Maes. Direct Manipulation vs. Interface Agents. interactions 1997.
  2. Eric Horvitz. Principles of Mixed-Initiative Interaction. CHI 1999 (in Pittsburgh 😍)
  3. (grad only)  Man-Computer Symbiosis J. C. R. Licklider IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4-11, March 1960(!!)

Quiz 2: CMU Students | General public (ungraded)

PLEASE SIGN UP: for being a discussant. If you are a discussant for next week, please plan to come to office hours this week.

 

September 5th   A History of Humans Interacting with AI + AI vs. IA

Slides

        Quiz 3: CMU students | General public (ungraded)

        

Due by class:        reading reflections on piazza

Week 3: Designing AI/ML User Experience

September 10th   Matchmaking Needs and Risks for Adding AI/ML

Slides

        Design workshop

  1. Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S. T., Bennett, P., Inkpen, K., Teevan, J., Kikin-Gil, R., and Horvitz, E. (2019) Guidelines for Human-AI Interaction (CHI 2019). [pdf]
  2. Amershi, Saleema, et al. "Software engineering for machine learning: a case study." Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice. IEEE Press, 2019. [pdf]
  3. (grad only) Kay, Patel, and Kientz. How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy. CHI 2015.

Worksheets for design workshop:

  1. AI and mental models
  2. Designing AI for different stakes
  3. AI and coadaptation

        New assignment: Assignment 2 Loan application front-end

September 12th   Communicating Predictions & Recommendations with Users

        Quiz

Worksheets for design workshop:

  1. Feedback in AI worksheet

        Discussion panel

Week 4: Designing for failure

September 17th   Failure & Feedback with Users

Readings:

  1. Kocielnik, R., Amershi, S., and Bennett, P. (2019) Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-User Expectations of AI Systems. (CHI 2019). [pdf]
  2. Cai, Carrie Jun, et al. "``Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making." (2019). [pdf]
  3. Can we make Montreal’s buses more predictable? No. But machines can. (What? A Medium blog as academic reading?!?!)

Slides

September 19th   Product Workshop Day

        Make one new product based on prompt. Group activity.  

        Quiz

Discussion panel

Week 5: Data & Knowledge. Where does it come from? Who owns it?

September 24th: Why would people give you data anyway? Data ethics and laws

        Slides

        Readings

  1. Jeffrey Bigham, Michael Bernstein, and Eytan Adar. Human-Computer Interaction and Collective Intelligence
  2. Zimmerman, J., Tomasic, A., Garrod, C., Yoo, D., Hiruncharoenvate, C., Aziz, R., ... & Steinfeld, A. (2011, May). Field trial of tiramisu: crowd-sourcing bus arrival times to spur co-design. CHI 2011. [pdf]

September 26th: Using human-centric data in an ML pipeline (wait, should it even be a pipeline??)

Slides

        Quiz

Assignment 2 due on Sep 29

Week 6: Data Visualization & Data Communication

October 1st: Visualizations to improve human-AI interaction

slides

Readings:

  1. Carrie Cai et al (2019) Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making 
  2. Matthew Kay, Tara Kola, Jessica R. Hullman, Sean Munson (2016) When (ish) is My Bus? User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems

October 3rd

Quiz

New assignment: Assignment 3 visualization -- delayed! Sorry!

Guest lecture: Adam Perer: Visualization in AI (Slides)

        

Week 7: Interpreting and Explaining Algorithms

October 8th: How does telling people how an algorithm works change their experience?

Readings:

  1. Intelligible Artificial Intelligence [pdf]
  2. Why these Explanations? Selecting IntelligibilityTypes for Explanation Goals [pdf]
  3. [grad only] The Mythos of Model Interpretability [pdf]

Slides

October 10th

Quiz

Slides

Assignment 3 due on October 20th

Week 8 - 9: AI Ethics, Fairness, Social Acceptability, and Trust

October 15th  

Reading: Medical devices: the Therac-25 by Nancy Leveson (This is the only reading for the week because a) it is somewhat longer, and b) I want you to think about it carefully.

[Warning: this reading has certain graphic, but textual, descriptions of the results of accidental radiation exposure during clinical therapy. ]

This reading is ostensibly not about AI. But it may be one that allows you to draw many parallels to our AI space. When you read the paper, consider these questions:

- What parallels can you draw from the reading to the design of human-AI systems? (For instance, the Tyler incident is caused by a "race" condition -- a hard-to-find bug that is the result of exact timing of operations, which are largely determined by chance and so hard to inspect ahead of time. AI systems can similarly rely on things that are hard to inspect ahead of time.)

- What are the roles that people played in this story in making the error, diagnosing it, and fixing it?

- Who are the heroes of this story? Who are the villains? Is it useful to think of them this way?

- What roles did users play in this story?

- Was the response from the manufacturer ethical? What about the regulatory governmental agencies, and the attending doctors?

- If something similar were to happen today with an AI-infused system, what would you expect to go differently?

October 17th   Guest Lecture Ken Holstein  Improving fairness in ML systems: What do industry practitioners need?

        Quiz

Assignment 3 is due Oct 20th!

Week 9

October 22nd

October 24th: class is canceled for the HCII 25th celebrations

        No Quiz

No Discussion panel

Week 10: Human in the loop with AI/ML & Recommendations

October 29th  

Guest lecture Julian Ramos: personalized context aware health interventions.

        New assignment: Assignment 4 Make a chatbot with humans in the loop that recommends stuff to you.

Readings:

The focus of this week is to rethink what it means for a human to be “in the loop”. The readings reflect this focus. If you’re interested in the more traditional view of humans “in the loop”, make sure you read the grad reading.

  1. Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, Eric Horvitz (2019): Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
  2. Sharon Zhou, Melissa Valentine, Michael S. Bernstein (2018) In Search of the Dream Team: Temporally Constrained Multi-Armed Bandits forIdentifying Effective Team Structures
  3. [grad only] Ting-Hao (Kenneth) Huang, Joseph Chee Chang, and Jeffrey P. Bigham Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time

October 31st

        Quiz

Discussion Panel

Week 11: Natural Language & Speech Applications

November 5th

November 7th

Quiz

Discussion panel

Assignment 4 due on November 10th

Week 12: Vision, Images, & Art

November 12th

New assignment: Assignment 5 Vision with GANs

Final Project Released: Make something cool & interactive with AI/ML

November 14th

        Quiz

        Discussion panel

Week 13: Special Topic/Application Area (class polling to decide)

November 19th

November 21st

        Quiz

Discussion panel

Week 14: Break

November 26th        Thanksgiving Break; no class or normal office hours

November 29th        Thanksgiving Break; no class or normal office hours (🦃 or 🥧?)

Week 15: Final Projects & Special Topic (class polling to decide)

December 3rd:   Project presentations

December 5th:   Closing

        Quiz

Final Project due on Dec 8th