Week 22 / May 2025

Systems Outpace Verification in High-Stakes Domains

From nuclear control rooms to accessibility tools, users can't evaluate what AI executes

Synthesized using AI

Analyzed 143 papers. AI models can occasionally hallucinate, please verify critical details.

AutoGraph's work in nuclear power plant control rooms crystallizes a problem appearing across domains: systems can execute complex procedures, but users cannot verify the execution was correct. The knowledge-graph framework semantically links procedural steps to interface actions, but the deeper finding is that operators must trust automation they cannot evaluate. This verification gap appears in Vid2Coach's accessibility work, where blind users following how-to videos need confirmation they're performing tasks correctly, and in OMNIGUARD's safety moderation, where multilingual and multimodal attacks exploit the mismatch between what models can do and what moderators can verify. The pattern cuts across healthcare platforms (MAIA, CASE), city government AI adoption, and even Wikipedia knowledge gaps—capability without verifiability creates risk.

A second pattern complicates the first: building for imagined users systematically misses what actual users need. ToPSen's comparison of sighted and blind programmers using audio feedback reveals that traditional blindness simulation fails to capture real coding constraints—sensory deprivation studies don't replicate the technical requirements blind developers actually face. City government employees pursuing equity goals operate under constraints researchers don't anticipate, while WikiGap shows English Wikipedia readers assume comprehensiveness that marginalizes knowledge in other languages. The participatory design work here isn't about inclusion theater—it's about catching fundamental misalignments between what technologists think matters and what enables agency in practice.

The tension between these patterns reveals the field's current bind: we're building increasingly capable systems for high-stakes domains while losing the ability to validate they work as intended. The crowdsourced simulated agents work offers one path forward—accepting that scale can compensate for fidelity in evaluation—but it doesn't solve verification in deployment. What's needed are workflows that make errors observable, not explanation interfaces that assume users can evaluate correctness.

Featured(1/5)

2505.22981

Free Lunch for User Experience: Crowdsourcing Agents for Scalable User Studies

Siyang Liu, Sahand Sabour, Xiaoyang Wang, Rada Mihalcea

Preprint·2025-05-29

Run your next usability test with 1,000 agents before recruiting humans. Use it for rapid iteration on information architecture or onboarding flows. Validate edge cases (low-literacy users, non-native speakers) that are expensive to recruit traditionally.

User studies cost $50-150 per participant and take weeks to recruit. You're bottlenecked by budget and timeline, so you test with 12 people and call it validated.

Method: This team recruited simulated users from billion-scale profile datasets—not generic LLM personas, but agents instantiated from real demographic and behavioral data. They ran studies at 100x scale compared to traditional recruitment. The key mechanism: agents inherit attributes from actual user profiles (age, tech literacy, domain expertise) rather than relying on prompt engineering alone. This grounds the simulation in empirical distributions, not researcher assumptions about 'typical users.'

Caveats: Fidelity doubts remain—agents may miss embodied or emotional nuances. Best for cognitive tasks, not affective design decisions.

Reflections: Which UX tasks show the largest fidelity gap between agent and human responses? · Can agents simulate frustration or fatigue during long tasks? · How do you validate that agent diversity matches real population variance?

evaluation-methodsai-interactiondesign-tools

2505.19317

Effort-aware Fairness: Incorporating a Philosophy-informed, Human-centered Notion of Effort into Algorithmic Fairness Metrics

Tin Trung Nguyen, Jiannan Xu, Zora Che, Phuong-Anh Nguyen-Le, Rushil Dandamudi, Donald Braman, Furong Huang, Hal Daumé, Zubin Jelveh

Preprint·2025-05-25

Audit your ML models for effort-aware disparities. If your credit model penalizes someone who rebuilt their score after medical bankruptcy equally to someone who never faced hardship, you're missing structural inequity. Integrate effort trajectories into training data.

Demographic parity treats a first-generation college applicant with a 3.8 GPA the same as a legacy admit with identical scores. Existing fairness metrics ignore the effort trajectory behind the numbers.

Method: They formalize 'Force'—the temporal trajectory of effort required to reach a given feature state. Instead of comparing static outcomes (e.g., credit scores), the metric compares the integral of effort over time. For example, two applicants with identical test scores may have radically different Force values if one attended under-resourced schools or worked full-time. The framework uses counterfactual reasoning: 'What effort would this person need to exert to reach the favorable outcome?' This shifts fairness evaluation from outcome parity to opportunity cost parity.

Caveats: Requires longitudinal data on user histories—often unavailable in production datasets. Effort is also culturally constructed and hard to operationalize universally.

Reflections: How do you measure 'effort' for non-linear life events like caregiving or illness? · Can effort-aware metrics be gamed by fabricating hardship narratives? · What happens when effort and merit conflict in high-stakes decisions?

ethicstrust-safetyevaluation-methods

2506.00717

Vid2Coach: Transforming How-To Videos into Task Assistants

Mina Huh, Zihui Xue, Ujjaini Das, Kumar Ashutosh, Kristen Grauman, Amy Pavel

UIST 2025·2025-05-31

Stop building screen readers for video content—they narrate but don't guide. Build task-state detectors that translate visual goals into tactile or auditory checkpoints. Prioritize recipes, exercises, and crafts where procedural knowledge is locked in visual media.

Blind users can't follow YouTube cooking tutorials because they rely on visual comparison—'stir until the batter looks smooth' is useless without sight. Visual rehabilitation therapists fill this gap manually, but don't scale.

Method: Vid2Coach converts how-to videos into wearable camera-based assistants that provide proactive and responsive guidance. The system uses computer vision to detect task states (e.g., 'batter is lumpy' vs. 'batter is smooth') and translates visual cues into non-visual workarounds. For example, it replaces 'fold until peaks form' with 'tilt the bowl—if batter slides slowly, you're done.' It also gives real-time progress feedback: 'You've completed step 3 of 7.' The system was trained by observing how VRTs guide BLV users through video tasks.

Caveats: Requires wearable camera setup and works best for structured tasks with clear state transitions. Struggles with ambiguous visual goals like 'golden brown.'

Reflections: Can this generalize to unstructured tasks like home repair or gardening? · How do you handle tasks where visual aesthetics matter (e.g., cake decorating)? · What's the latency tolerance for real-time feedback during fast-paced cooking?

accessibilityai-interactionwearablesdesign-tools

1 / 5

Featured

Findings(1/5)

Fairness metrics shift from outcome parity to process legibility·Simulation replaces recruitment as the default for early-stage user research·Knowledge graphs formalize operational context to enable automation handoffs·Participatory platforms adopt event-driven architectures to support adaptive research·Accessibility research pivots from compensating for deficits to reframing sensory constraints

Fairness research is moving beyond demographic parity to incorporate temporal effort and institutional context. Papers model fairness as force over time in feature space, trace how intermediary organizations codify algorithmic folk theories into governance structures, and map how practitioners balance competing stakeholder interests in dynamic environments. The shift reframes fairness from a post-hoc audit problem to a design constraint requiring visibility into both individual trajectories and organizational power structures. Implication: fairness tooling must expose process, not just outcomes.

2505.19317

Effort-aware Fairness: Incorporating a Philosophy-informed, Human-centered Notion of Effort into Algorithmic Fairness Metrics

2505.20623

Institutionalizing Folk Theories of Algorithms: How Multi-Channel Networks (MCNs) Govern Algorithmic Labor in Chinese Live-Streaming Industry

2505.19441

Fairness-in-the-Workflow: How Machine Learning Practitioners at Big Tech Companies Approach Fairness in Recommender Systems

Surprises(1/3)

English Wikipedia dominance creates knowledge asymmetry, not knowledge access·Deployment failure in civic tech stems from sociotechnical misalignment, not technical debt·Embodied presence in VR co-viewing increases engagement but reveals absence of design patterns

Readers assume English Wikipedia is a superset of all language editions, but other editions contain complementary facts rooted in local cultures and media environments that are marginalized in English. The dominance of English Wikipedia—with 11x more pageviews than the next largest edition—actively obscures knowledge rather than centralizing it. The reframe: linguistic hegemony in knowledge platforms creates epistemic inequity, not efficiency.

Reveals that computer use agents with 90% correct knowledge still fail 59% of tasks—the retrieval-execution gap in action. Proposes self-evolving knowledge systems that learn from failure patterns.

2505.19325

What do Blind and Low-Vision People Really Want from Assistive Smart Devices? Comparison of the Literature with a Focus Study

Compares what researchers build for BLV users versus what they actually need. Spoiler: the mismatch is systematic, not accidental.

2505.21752

Experimental Evidence That AI-Managed Workers Tolerate Lower Pay Without Demotivation

Tracks workers in a custom Minecraft workplace and finds they accept lower wages from AI managers without performance drops. Trust miscalibration meets labor economics.

2505.20916

Imago Obscura: An Image Privacy AI Co-pilot to Enable Identification and Mitigation of Risks

Builds an AI copilot that spots privacy risks in photos before you post them. Tests whether users actually understand and trust the mitigations it suggests.

2505.22906

HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding

Interrupts AI code generation mid-stream to force programmers to make decisions, preventing the "brain off" problem. Human-in-the-loop as a feature, not a bug.

2505.24115

FeatureSense: Protecting Speaker Attributes in Always-On Audio Sensing System

Strips speaker identity from always-on audio while preserving activity recognition signals. Addresses the trust problem by redesigning what the microphone can actually know.

2505.23310

Investigating A Geometrical Solution to the Vergence-Accommodation Conflict for Targeted Movements in Virtual Reality

Tests a geometric fix for the eye-strain problem in VR headsets during reaching tasks. Wild methodology: measuring how your eyes betray the display's lies.

2505.21604

Public Discourse Sandbox: Facilitating Human and AI Digital Communication Research

Creates a synthetic social media platform for research that sidesteps platform API restrictions and ethical landmines. Infrastructure for studying discourse without real-world collateral damage.

REFLECTION(3)

Transparency theater replaces actual trust

Systems that disclose their limitations often obscure them further—guardrails and refusal mechanisms create new opacity rather than clarity. The research shows that appropriate trust isn't solved by showing users more; it's broken by design choices that hide the real failure modes.

Disclosure mechanisms (confidence scores, uncertainty estimates, refusal explanations) can paradoxically reduce calibration by shifting user attention to theater rather than actual capability boundaries. If a system refuses a task with a polite message, does the user learn what it can actually do, or just that it sometimes says no?

1 / 3

Week 21May 2025

Week 23June 2025

ABOUT THIS ISSUE

How this newsletter was synthesized?

Methodology

This newsletter is generated by an AI pipeline (leveraging Anthropic Sonnet 4.5 & Haiku 4.5) that processes the metadata and abstracts of every new arXiv HCI paper from the past week—143 this issue. Each paper is scored on three dimensions: Practice (applicability for practitioners), Research (scientific contribution), and Strategy (industry implications), with scores from 1-5. Papers passing threshold are grouped into topic clusters, and each cluster is summarized to capture what that body of research is exploring.

Selection Criteria

The pipeline builds a curated selection that balances high scores with topic diversity—and deliberately includes at least one 'contrarian' paper that challenges prevailing assumptions. This selection is then analyzed to identify key findings (patterns across multiple papers) and surprises (results that contradict conventional wisdom). A narrative synthesis ties the week's research together under a unifying frame.

Key Themes Discovered

Field Report: ai-interaction

Trust, Transparency, and Task Execution

This cluster examines how users calibrate trust in AI systems and whether AI can reliably execute real-world tasks. Core tensions emerge: AI safety mechanisms (guardrails, refusals) trade user satisfaction for protection; knowledge gaps between what systems retrieve and what they execute persist even with correct information; and users struggle to assess when AI outputs warrant reliance. Research spans multimodal safety, agent reliability, preference alignment, and the psychological mechanisms underlying disclosure and intimacy formation. Implications center on designing AI interactions that balance autonomy with accountability.

1/10

Top Papers in this Theme

2505.23856

OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities

2505.21964

UI-Evol: Automatic Knowledge Evolving for Computer Use Agents

2505.19335

Knoll: Creating a Knowledge Ecosystem for Large Language Models

2505.19897

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows

2505.21604