Week 1: Introduction to Human-AI Interaction
August 27th Course Introduction
New assignment: Assignment #0 - DUE BEFORE 2nd class
August 29th Practical ML
Due by class: Assignment #0
New assignment: Assignment #1, due Sunday September 8th by 11:59pm
Week 2: Perspectives on Human-AI Interaction
September 3rd A History of Artificial Intelligence
- Ben Schneiderman and Pattie Maes. Direct Manipulation vs. Interface Agents. interactions 1997.
- Eric Horvitz. Principles of Mixed-Initiative Interaction. CHI 1999 (in Pittsburgh 😍)
- (grad only) Man-Computer Symbiosis J. C. R. Licklider IRE Transactions on Human Factors in Electronics, volume HFE-1, pages 4-11, March 1960(!!)
Quiz 2: CMU Students | General public (ungraded)
PLEASE SIGN UP: for being a discussant. If you are a discussant for next week, please plan to come to office hours this week.
September 5th A History of Humans Interacting with AI + AI vs. IA
Quiz 3: CMU students | General public (ungraded)
Due by class: reading reflections on piazza
Week 3: Designing AI/ML User Experience
September 10th Matchmaking Needs and Risks for Adding AI/ML
- Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S. T., Bennett, P., Inkpen, K., Teevan, J., Kikin-Gil, R., and Horvitz, E. (2019) Guidelines for Human-AI Interaction (CHI 2019). [pdf]
- Amershi, Saleema, et al. "Software engineering for machine learning: a case study." Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice. IEEE Press, 2019. [pdf]
- (grad only) Kay, Patel, and Kientz. How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy. CHI 2015.
Worksheets for design workshop:
- AI and mental models
- Designing AI for different stakes
- AI and coadaptation
New assignment: Assignment 2 Loan application front-end
September 12th Communicating Predictions & Recommendations with Users
Worksheets for design workshop:
- Feedback in AI worksheet
Week 4: Designing for failure
September 17th Failure & Feedback with Users
- Kocielnik, R., Amershi, S., and Bennett, P. (2019) Will You Accept an Imperfect AI? Exploring Designs for Adjusting End-User Expectations of AI Systems. (CHI 2019). [pdf]
- Cai, Carrie Jun, et al. "``Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making." (2019). [pdf]
- Can we make Montreal’s buses more predictable? No. But machines can. (What? A Medium blog as academic reading?!?!)
September 19th Product Workshop Day
Make one new product based on prompt. Group activity.
Week 5: Data & Knowledge. Where does it come from? Who owns it?
September 24th: Why would people give you data anyway? Data ethics and laws
- Jeffrey Bigham, Michael Bernstein, and Eytan Adar. Human-Computer Interaction and Collective Intelligence
- Zimmerman, J., Tomasic, A., Garrod, C., Yoo, D., Hiruncharoenvate, C., Aziz, R., ... & Steinfeld, A. (2011, May). Field trial of tiramisu: crowd-sourcing bus arrival times to spur co-design. CHI 2011. [pdf]
September 26th: Using human-centric data in an ML pipeline (wait, should it even be a pipeline??)
Assignment 2 due on Sep 29
Week 6: Data Visualization & Data Communication
October 1st: Visualizations to improve human-AI interaction
- Carrie Cai et al (2019) Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Making
- Matthew Kay, Tara Kola, Jessica R. Hullman, Sean Munson (2016) When (ish) is My Bus? User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems
New assignment: Assignment 3 visualization -- delayed! Sorry!
Guest lecture: Adam Perer: Visualization in AI (Slides)
Week 7: Interpreting and Explaining Algorithms
October 8th: How does telling people how an algorithm works change their experience?
- Intelligible Artificial Intelligence [pdf]
- Why these Explanations? Selecting IntelligibilityTypes for Explanation Goals [pdf]
- [grad only] The Mythos of Model Interpretability [pdf]
Assignment 3 due on October 20th
Week 8 - 9: AI Ethics, Fairness, Social Acceptability, and Trust
Please take this ungraded pre-class survey in the first 10 min of class.
Reading: Medical devices: the Therac-25 by Nancy Leveson (This is the only reading for the week because a) it is somewhat longer, and b) I want you to think about it carefully.
[Warning: this reading has certain graphic, but textual, descriptions of the results of accidental radiation exposure during clinical therapy. ]
This reading is ostensibly not about AI. But it may be one that allows you to draw many parallels to our AI space. When you read the paper, consider these questions:
- What parallels can you draw from the reading to the design of human-AI systems? (For instance, the Tyler incident is caused by a "race" condition -- a hard-to-find bug that is the result of exact timing of operations, which are largely determined by chance and so hard to inspect ahead of time. AI systems can similarly rely on things that are hard to inspect ahead of time.)
- What are the roles that people played in this story in making the error, diagnosing it, and fixing it?
- Who are the heroes of this story? Who are the villains? Is it useful to think of them this way?
- What roles did users play in this story?
- Was the response from the manufacturer ethical? What about the regulatory governmental agencies, and the attending doctors?
- If something similar were to happen today with an AI-infused system, what would you expect to go differently?
October 17th Guest Lecture Ken Holstein Improving fairness in ML systems: What do industry practitioners need?
Slides and other information https://kenholstein.myportfolio.com/ai-fairness-and-explainability
A paper mentioned during class on Thursday, regarding limits/pitfalls of post-hoc, algorithmic de-biasing:
Assignment 3 is due Oct 20th!
- Disparate Interactions [pdf] How humans interact with risk predictions and how it leads to differences in fairness.
- Do artifacts have politics? [pdf]
- [Optional] Intelligible Models for HealthCare: Predicting PneumoniaRisk and Hospital 30-day Readmission [pdf] No reflection to be submitted!
October 24th: class is canceled for the HCII 25th celebrations
No Discussion panel
Week 10: Human in the loop with AI/ML & Recommendations
Guest lecture Julian Ramos: personalized context aware health interventions.
New assignment: Assignment 4 Make a chatbot
The focus of this week is to rethink what it means for a human to be “in the loop”. The readings reflect this focus. If you’re interested in the more traditional view of humans “in the loop”, make sure you read the grad reading.
- Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, Eric Horvitz (2019): Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff
- Sharon Zhou, Melissa Valentine, Michael S. Bernstein (2018) In Search of the Dream Team: Temporally Constrained Multi-Armed Bandits forIdentifying Effective Team Structures
- [grad only] Ting-Hao (Kenneth) Huang, Joseph Chee Chang, and Jeffrey P. Bigham Evorus: A Crowd-powered Conversational Assistant Built to Automate Itself Over Time
Week 11: Natural Language & Speech Applications
- [optional, no reflection needed] ELIZA—a computer program for the study of natural language communication between man and machine [pdf]
- [required, but no reflection needed] How to build a neural network
- Voice Interfaces in Everyday Life [pdf]
- Calendar.help: Designing a Workflow-Based Scheduling Agent with Humans in the Loop [pdf]
Assignment 4 due on November 10th
Week 12: Vision, Images, & Art
New assignment: Assignment 5 Vision with GANs
New Final Project Released: Make something cool or write something awesome!
Week 13: Special Topic/Application Area (class polling to decide)
Assignment #5 prompt & code materials
GANs + Special Topics Slides
Week 14: Break
November 26th Thanksgiving Break; no class or normal office hours
November 29th Thanksgiving Break; no class or normal office hours (🦃 or 🥧?)
Week 15: Final Projects & Special Topic (class polling to decide)
December 3rd: Project presentations
December 5th: Closing
Post class Questionnaire
Final Project due on Dec 15th