Bingsheng Arthur Yao


profile-square.jpeg

I am an associate research scientist in the Khoury College of Computer Sciences at Northeastern University (PI: Prof. Dakuo Wang).

My research at the intersection of NLP and HCI focuses on human-centered design of NLP applications to support collaborative work in domain-specific scenarios, including medical, healthcare, and education. Recently, I am working on LLM role-playing agents for human behavior simulation and human-agent collaboration.

Our lab is looking for self-motivated research assistants and Ph.D. students. Please refer to our lab website for more information.

Research

Patient Care

A critical sector with pressing needs that involves care provider teams, caregivers, and patients. A considerable amount of my recent work focuses on the human-centered design of AI/LLM systems to support stakeholder collaboration for remote patient monitoring (RPM) and clinical decision-makings, include identifying needs and challenges encountered by diverse stakeholders, designing and evaluating AI/LLM systems with these stakeholders.

Sepsis [CHI ‘24]; older adults [IMWUT ‘24a], post-operative patients [1] [2] [3] [CHI ‘25], concussion [4]

Children Education

Another unique collaborative scenario of my HCI & NLP research. I worked with stakeholders (i.e., education experts and parents) to understand their painpoints in storytelling activities, curate high-quality Question-Answering (QA) datasets, develop domain-specific NLP models, and design AI-supported interactive storytelling systems.

FairytaleQA [ACL ‘22a] & StorySparkQA [EMNLP ‘24] datasets, QA Generation Models [ACL ‘22b], StoryBuddy system [CHI ‘22]

Enhance AI With Human Cognitive Behaviors

This summarizes my pursuit of the long-term research vision. AI could outperform humans in task-solving performance, however, humans’ rational cognitive behaviors form real intelligence. My research on low-resource learning techniques pave the path for personalized AI when we provide individualized rationales as additional signals. Lately, I am devoted to advance LLM Role-Playing Agents to realistically mimic human behaviors, simulate human social dynamics, and establish human-agent collaborations.

Active Learning [EMNLP ‘23], In-Context Sampling [NAACL ‘24], Mental-LLM [IMWUT ‘24b], LLM Role-Playing Agents [ACL ‘25]

Appointment

  • Associate Research Scientist
    2025 - Current, Northeastern University
  • Postdoc Research Associate
    2024 - 2025, Northeastern University

Education

  • Ph.D. in Computer Science
    2019 - 2024, Rensselaer Polytechnic Institute (Advisor. Prof. Jim Hendler)
  • M.S. in Information Technology
    2018 - 2019, Rensselaer Polytechnic Institute
  • B.S. in Computer Engineering
    2014 - 2018, Rensselaer Polytechnic Institute

Note

Please refer to my Google Scholar page for the most up-to-date publication record.

The best way to reach out is through emails: b [dot] yao [at] northeastern [dot] edu.


News

2025.02 Co-first authored paper Survey of LLM Role-Playing Agent Evaluation is now publically available on arXiv
2025.01 Five papers were accepted to CHI 2025. Thanks for the hard working by collaborators and mentees!
2024.10 Our paper StorySparkQA Dataset with Real-World Knowledge for Children Education was accepted to EMNLP 2024
2024.09 Our paper Secret Use of Large Language Models was accepted to CSCW 2025
2024.07 Our paper Early Sepsis Prediction with Uncertainty Quantification and Active Sensing was accepted to KDD 2024
2024.04 Our paper LLM-based Voice Assistant for Asynchronous Older Adults-Care Provider Communication and Mental-LLM were accepted to IMWUT 2024
2024.03 First-authored paper In-Context Sampling Strategy for Reliable LLM Prompting was accepted to NAACL 2024 Findings
2024.03 Guest talk at USC titled “Bridging AI Research and Real-world Scenarios”. Thanks Prof. Yao Du for the invitation!
2024.02 I am joining Prof. Dakuo Wang’s Human-Centered AI Lab at Northeastern University as a postdoc associate!
2024.01 Two of our papers, Human-AI Collaboration in Sepsis Diagnosis and User’s Sensitive Disclosure with LLM were accepted to CHI 2024
2024.01 I passed the Ph.D. dissertation defense. My deepest gratitude to all those who supported and helped me, especially Prof. Jim Hendler and Prof. Dakuo Wang
2023.10 Our paper, Discourse Framework for Science Journalism, was accepted to EMNLP 2023, and another first-authored paper, Active Learning Empowered by Natural Language Explanations, was accepted to EMNLP 2023 Findings
2023.07 First-authored paper Objective Evaluation of Human Explanations was accepted to ACL 2023 for Oral Presentation
2022.05 Two first-authored papers, QA-Pair Generation for Story Books and FairytaleQA Dataset were accepted to ACL 2022
2022.04 Our paper StoryBuddy was accepted to CHI 2022
2021.09 Our paper Narrative Open-Domain QA Techniques was accepted to TACL (2021) 9
2020.03 Our paper Trust in AutoML was accepted to IUI 2020