Bingsheng Arthur Yao
Associate Research Scientist at Northeastern University
I am an associate research scientist in the Khoury College of Computer Sciences at Northeastern University (PI: Prof. Dakuo Wang). I received my PhD from Rensselaer Polytechnic Institute, advised by Prof. Jim Hendler.
I study how LLM agents can genuinely collaborate with humans through the lens of remote human collaboration. My research sits at the intersection of Human-Computer Interaction and Natural Language Processing, with a focus on designing the interaction patterns and methodologies for effective human-agent collaboration in practice, and on developing and evaluating LLM agents that can think and behave collaboratively.
Research
I. Design and Study Human-Centered AI Systems for Clinical Care
Healthcare workflows often break down at the seams between provider teams, patients, and caregivers. I design, deploy, and study AI/LLM-powered multi-modal systems that work in those gaps to support diverse stakeholders involved in different clinical settings. My work spans clinical decision-making, patient-provider communication, provider coordination, remote patient monitoring, and post-surgical care.
- RECOVER (CSCW '26) — LLM-based remote patient monitoring for post-surgical GI cancer care
- Serious Illness Conversations (CHI '26) — Balancing efficiency and empathy in serious-illness conversations in the ED
- Collaboration Breakdown (CHI '26) — Within provider teams and between patients in post-surgery care
- CardioAI (CHI '25) — Multimodal AI plus wearable for cancer treatment-induced cardiotoxicity
- Talk2Care (IMWUT '24) — Asynchronous communication between older-adults and care providers
- Sepsis Diagnosis Support (CHI '24) — AI-assisted clinical decision making for sepsis diagnosis
II. Genuine Human-Agent Collaboration
I envision a near future where LLM agents can genuinely work with us, like a remote human collaborator. To get there, I design interaction patterns and study methodologies for human-agent collaboration, and develop and benchmark LLM agents that can think and behave collaboratively. My work spans collaboration frameworks and design philosophy, trust and shared context in human-agent teams, evaluation methods for collaboration quality, agent oversight, and role-playing agents.
- Configurable Research Platform for Human-Agent Collaboration (CHI '26)
- Dark Patterns Meet GUI Agents (CHI '26) — Examining LLM agent susceptibility to manipulative interfaces
- CHI '26 Workshop "Human-Agent Collaboration" (CHI EA '26) — A vision, design philosophy, and empirical framework for treating LLM agents as remote human collaborators
- Agent A/B (CHI EA '26) — Automated and scalable A/B testing on live websites with interactive LLM agents
- Dynamic Persona Refinement Framework (preprint, 2025) — Iteratively optimizing behavioral and cognitive alignment
- Survey of LLM Role-Playing Agent Evaluation (ACL Findings '25) — Guidelines for evaluating LLM role-playing agents
Note
Please refer to my Google Scholar page for the most up-to-date publication record.
The best way to reach out is through emails: b [dot] yao [at] northeastern [dot] edu.
News
| 2025.02 | Co-first authored paper Survey of LLM Role-Playing Agent Evaluation is now publicly available on arXiv |
|---|---|
| 2025.01 | Five papers were accepted to CHI 2025. Thanks for the hard work by collaborators and mentees! |
| 2024.10 | Our paper StorySparkQA Dataset with Real-World Knowledge for Children Education was accepted to EMNLP 2024 |
| 2024.09 | Our paper Secret Use of Large Language Models was accepted to CSCW 2025 |
| 2024.07 | Our paper Early Sepsis Prediction with Uncertainty Quantification and Active Sensing was accepted to KDD 2024 |
| 2024.04 | Our papers LLM-based Voice Assistant for Asynchronous Older Adults-Care Provider Communication and Mental-LLM were accepted to IMWUT 2024 |
| 2024.03 | First-authored paper In-Context Sampling Strategy for Reliable LLM Prompting was accepted to NAACL 2024 Findings |
| 2024.03 | Guest talk at USC titled “Bridging AI Research and Real-world Scenarios”. Thanks Prof. Yao Du for the invitation! |
| 2024.02 | I am joining Prof. Dakuo Wang’s Human-Centered AI Lab at Northeastern University as a postdoc associate! |
| 2024.01 | Two of our papers, Human-AI Collaboration in Sepsis Diagnosis and User’s Sensitive Disclosure with LLM were accepted to CHI 2024 |
| 2024.01 | I passed the Ph.D. dissertation defense. My deepest gratitude to all those who supported and helped me, especially Prof. Jim Hendler and Prof. Dakuo Wang |
| 2023.10 | Our paper Discourse Framework for Science Journalism was accepted to EMNLP 2023, and another first-authored paper Active Learning Empowered by Natural Language Explanations was accepted to EMNLP 2023 Findings |
| 2023.07 | First-authored paper Objective Evaluation of Human Explanations was accepted to ACL 2023 for Oral Presentation |
| 2022.05 | Two first-authored papers, QA-Pair Generation for Story Books and FairytaleQA Dataset were accepted to ACL 2022 |
| 2022.04 | Our paper StoryBuddy was accepted to CHI 2022 |
| 2021.09 | Our paper Narrative Open-Domain QA Techniques was accepted to TACL 2021 |
| 2020.03 | Our paper Trust in AutoML was accepted to IUI 2020 |