Research

Scale-Invariant Human-Swarm Control

A core challenge in swarm robotics is enabling human operators to flexibly control large numbers of agents without becoming overwhelmed. While many existing algorithms and interfaces support only a fixed set of pre-defined behaviors, 1) few allow operators to re-specify swarm behavior on the fly; 2) some exclude the human operator from the control loop entirely; and 3) many do not enable the human operator to create commands in a continuous space that can encode diverse, evolving objectives—instead only allowing discrete commands that must be pre-determined before a mission.

In this work, my colleagues and I developed an end-to-end system that enables a human operator to issue scale-invariant tactical commands to their swarm in a flexible, real-time manner. The system continuously adapts swarm behavior based on the operator’s high-level input—without requiring the user to reason about individual agents, agent-agent interactions, or how the swarm size might affect execution. Because objectives are defined in terms of spatial distributions (i.e., desired swarm density), the operator can focus on what the swarm should do, rather than how it should do it.

This abstraction significantly reduces cognitive load by allowing operators to plan and re-plan behavior at a higher level of intent. The system automatically adjusts swarm behavior as agents are added to or removed from the group, preserving task execution and responsiveness without additional user intervention. We demonstrate the effectiveness of this approach in a field-relevant simulator with real-world demonstrations, showing that the system supports dynamic, flexible, and scalable control across a variety of high-level objectives.

Associated Publications

Scale-Invariant Specifications for Human-Swarm Systems
Joel Meyer*, Ahalya Prabhakar*, Allison Pinosky, Ian Abraham, Annalisa Taylor, Millicent Schlafly, Katarina Popovic, Giovani Diniz, Brendan Teich, Borislava Simidchieva, Shane Clark, Todd D. Murphey
IEEE Transactions on Field Robotics 1, 276-291 Nov 2024.
* denotes equal contribution. PDF

A Game Benchmark for Real-Time Human-Swarm Control

Most existing testbeds for human-swarm systems fall short in three critical areas: 1) Lack of high-cadence decision-making—they rarely evaluate how well a human or autonomous operator performs when rapid, successive decisions must be made in real time; 2) Absence of dynamic, adversarial scenarios—they do not test the operator’s ability to adapt under pressure from an intelligent, evolving opponent; and 3) Insufficient scale or realism—many are not truly real-time or involve too few agents to reflect the coordination demands of a full swarm.

To address these gaps, I developed an open-source game benchmark specifically designed to evaluate real-time, adversarial, high-cadence human-swarm interaction. The objective of the game is to capture members of the opposing swarm by surrounding them with agents from your own swarm, similar to a dynamic, tactical version of Go. Human operators can compete against each other or face off against an autonomous adversary trained via deep reinforcement learning. The RL agent outputs spatial distributions that encode high-level tactical intent, which its swarm uses to maneuver and surround opponents to "win" the game. This benchmark enables researchers to empirically compare algorithms and interfaces under real-time, rapidly changing conditions, and its modular design allows for easy customization of environments and game rules to suit different research goals.

Associated Publications

Scale-Invariant Reinforcement Learning for Swarm Control
Joel Meyer, Muye Jia, and Todd D. Murphey
In Preparation, 2025.

A Game Benchmark for Real-Time Human-Swarm Control
Joel Meyer, Allison Pinosky, Thomas Trzpit, Ed Colgate, and Todd D. Murphey
IEEE International Conference on Automation Science and Engineering, Aug 2022. PDF

Decentralized Active Learning of Novel Features

Multi-robot teams operating in real-world environments often encounter assets—such as objects, people, or other robots—that were not known prior to deployment and are not represented in pre-existing datasets. To operate effectively in these settings, robot teams must be able to identify novel assets in their environments and learn representations of them in real time, enabling those assets to be recognized in different contexts and incorporated into future planning.

In team settings, individual robots may encounter visual novelties that other robots cannot directly observe due to limited sensor coverage or environmental constraints. As a result, it is essential that robots not only learn models of new features on their own, but also share and integrate that knowledge across the team in a way that enables all agents to recognize and reason about the novelty. To ensure robustness and eliminate single points of failure, this learning process must be conducted in a decentralized fashion—posing the additional challenge of keeping each robot’s model consistent across the team despite distributed data access.

To address these challenges, I developed a system for decentralized active learning of novel visual features using a team of mobile robots. When exploring an environment, each robot estimates the entropy of different regions and prioritizes collecting additional data from high-entropy areas that are likely to contain novel features. When a robot identifies a new object or visual pattern, it updates its own latent model and shares the resulting neural network weights with other members of the team. These shared weights are then incorporated via a consensus protocol, allowing all robots to update their models in a way that reflects the combined discoveries of the team—even for novelties they did not directly observe. This approach enables scalable, decentralized learning of useful visual representations in real-time, previously unexplored environments.

Associated Publications

Decentralized Active Learning of Novel Visual Features
Joel Meyer and Todd D. Murphey
In Preparation, 2025.