Computers’ Control over Bodies: Experts Identify Week-Long Grand Challenges

Researchers are increasingly exploring how computers can directly control our bodies through technologies like on-body mechanical actuators and electrical muscle stimulation. Florian Mueller from Monash University, Nadia Bianchi-Berthouze of University College London, and Misha Sra from University of California Santa Barbara, alongside Gonzalez-Franco, Pohl, and Boll, have collaboratively identified critical ‘grand challenges’ in this rapidly evolving field. Their work, stemming from an expert seminar, moves beyond purely technical considerations to address the fundamental questions of agency, bodily experience, and ethical implications when computers exert control over human movement. By outlining these challenges, this research establishes a vital agenda for future human-computer interaction, ensuring bodily control is approached as a holistic, experiential, and ethically-grounded concern.

Computers taking control of human bodies seems like

Scientists have demonstrated a pivotal shift in human-computer interaction, moving beyond the traditional controller-responder model to one where computers actively take control over the human body. These challenges are multifaceted, spanning technical hurdles, crucial design considerations, user experience complexities, and pressing ethical concerns. Experiments show computers are now capable of influencing behaviours and even actuating our bodies across diverse applications, from healthcare and fitness to industrial assistance and entertainment. For instance, neuromuscular electrical stimulation aids physical therapy, while lower-limb exoskeletons assist post-surgery patients by dynamically adjusting control levels, demonstrating a direct impact on physical rehabilitation.
In fitness, electrical muscle stimulation systems are enhancing athletic performance, and in industrial settings, devices are reducing strain on warehouse workers and soldiers alike. This study unveils a broad spectrum of control mechanisms, ranging from subtle influences to forceful actuations, facilitated by a variety of technologies as illustrated in Figure 1. The team achieved a comprehensive overview of these technologies, including guiding motor actions, automatic calibration of high-density electrical muscle stimulation, and even inducing sensations like itching or tickling. Furthermore, the work opens avenues for innovative applications such as gameplay enhanced by electrical muscle stimulation, shape displays providing haptic encounters, and pneumatic bodily extensions.

The researchers prove that computers can now surrendering a sense of balance, visualize robot motion intent, and even inhibit movement through techniques like galvanic vestibular stimulation. The implications of this research extend far beyond mere technological advancement. The study reveals critical questions surrounding user safety in the event of malfunctions, and the necessity of aligning computer actions with user intentions and expectations. This work initiates a research agenda focused on understanding the consequences of shifting control from human to machine, and ensuring responsible design and implementation of these powerful new technologies. By articulating these grand challenges, the team hopes to foster a future where bodily control is not just technologically feasible, but also ethically sound and experientially enriching for all users.

Expert Workshop Identifying Human-Computer Interface Challenges concluded successfully

The research team meticulously recruited participants to ensure a diverse cohort, prioritising international and institutional representation, a broad range of research expertise, and varied career stages, detailed in Table 1. This deliberate composition, spanning assistive technology, ethics, and haptic interfaces, guaranteed discussions benefited from both practical constraints and broader societal considerations. The study pioneered an incremental methodology, beginning with individual presentations where each expert articulated challenges encountered throughout their research, these were immediately recorded on four A2 sheets, initially clustered under technology, users, design, and ethics. Researchers also documented reasons for relinquishing bodily control, prompting consideration of diverse theoretical perspectives for framing the grand challenges.

This initial phase yielded 118 distinct challenges, meticulously logged in a Google sheet and categorised to facilitate subsequent analysis. Following the initial challenge identification, the team encouraged ongoing contributions, allowing participants to add comments and opportunities to the A2 sheets throughout the sessions. Organisers then split participants into groups, tasking them with refining these clusters and identifying overarching themes, this collaborative process ensured a comprehensive and nuanced understanding of the challenges. Each group subsequently presented their refined clusters, sparking further discussion and debate in a plenary session. Finally, the research culminated in a rigorous filtering process, selecting challenges deemed critical, significant, unaddressed, and feasible, further refinement occurred within groups before final discussion and selection. The workshop’s innovative process, inspired by prior HCI efforts [38, 40, 105], successfully consolidated fragmented threads into a cohesive framework of grand challenges.

Bodily control challenges action-perception alignment

This work identifies technical, design, user, and ethical aspects crucial for advancing bodily control research. Researchers focused on understanding how computers can interact with our bodies, moving beyond simple actuation to consider agency and the experiential nature of control. The study highlights that actions, whether biomechanical or digital, form interconnected systems susceptible to failure or success depending on alignment between actions and perceptions. Experiments revealed that a failure occurs when actions are based on insufficient or misinterpreted sensory information, or when there is no intention to act.

The team measured goal attainment at multiple levels, from micro-goals like a hand grasp, assessed by whether an EMS-controlled hand successfully grasped an object, to macro-goals encompassing user well-being. Data shows a shift towards goal-oriented computing, contrasting with traditional “prompt and repeat” systems where repeated re-prompting is necessary if the initial prompt fails to achieve the desired outcome. This is particularly critical in dynamic control scenarios such as driving. Results demonstrate a layered interpretation process, ranging from sub-perceptual control, like controlling a grasp, to social and ethical reflections spanning time and societal impact.

Scientists recorded that humans are judged by their intentions, while computers are judged by outcomes, suggesting user participation is vital throughout the entire action to ensure the computer retains human intent. The research emphasizes the dynamic triangle of interaction between user, system, and environment, shaped by continuous perception-action cycles. Measurements confirm that a successful system must discern not only how to assist a user, but when and to what extent, even accommodating user requests that the system might deem unnecessary. For instance, the study considered a scenario where a user requests walking assistance from a leg-control system, despite the system assessing the user should walk independently due to weight; this raises complex control questions. Tests prove that understanding the temporal aspects of control, from short tasks to long-term rehabilitation, is crucial, with the perception-action cycle serving as the foundation for adaptive interaction. Activity