Publications
- TAP-MPC: Transferable Action Priors for Model Predictive Control NEW!
Do Robots Need World Models? Workshop @ CoRL 2026 - DefWAM: 3D Point Flow Prediction Learns Effective Goal-Conditioned Deformable Object Manipulation NEW!
DAROMA Workshop @ RSS 2026 PhysWM Workshop @ CoRL 2026 · [PDF]click 2 expand abstract
Manipulating deformable objects toward desired 3D configurations is a fundamental capability for robots operating in unstructured physical environments. However, goal-conditioned deformable object manipulation in 3D requires reasoning over high-dimensional shape variation, complex dynamics, and complex gripper-object interactions. We propose DefWAM, a 3D World Action Modeling approach that learns goal-conditioned 3D point-flow prediction as a unified objective for modeling both robot behavior and deformable scene evolution. By representing the robot and scene as point clouds in a unified 3D space, and actions as 3D point flows, DefWAM predicts goal-conditioned joint robot and scene flow and models deformable dynamics directly in particle space. This joint 3D point-flow objective enables our approach to learn when and how to use different contact modes, such as grasping, releasing, and re-grasping, to drive deformable objects toward diverse target shapes. Our experiments show that DefWAM outperforms action-only 3D goal-conditioned imitation learning baselines, particularly under large variation in initial and goal configurations. Furthermore, DefWAM's predicted scene evolution enables inference-time steering to avoid collisions by implicitly simulating possible robot-scene futures.
- Adapting Diffusion Policies to Novel Environments via Policy-Steered Optimization NEW!
ReALM-GEN Workshop @ ICLR 2026 · [PDF] - Learning Equivariant Neural-Augmented Object Dynamics from Few Interactions
RINO Workshop @ CoRL 2025 🏆 Best Paper Award · [PDF]
In submission to RA-L 2026 - Robot Task Planning under Local Observability
ICRA 2024 · [PDF]click 2 expand abstract
Real-world planning robot task planning is computationally intractable in part due to the complexity of dealing with partial observability. One common approach to reducing planning complexity is to introduce additional structure into the decision process, such as mixed-observability, factored state representations, or temporally-extended actions. We introduce a novel formulation, the locally observable Markov decision process, which models the case where a robot has access to subroutines for seeking and accurately observing objects using its sensors. The remaining partial observability stems from the fact that robot sensors are range-limited and line-of-sight---objects occluded or outside sensor range are unobserved, but objects that fall within view of its sensors can be fully observed using its observation subroutine. This model results in a three-stage planning process: first, the robot attempts to solve the task using only observed objects; if that fails, it generates a set of candidate objects that, if observed, could result in a feasible plan; finally, it plans to find those candidate objects by searching unobserved regions using its seek subroutine, replanning after each new object is observed. By combining this formulation with off-the-shelf Markov planners we are able to outperform state of the art solvers for both object oriented POMDP and MDP analogues with the same task specification. We then demonstrate the usefulness of our formulation by solving a partially-observable planning task on a mobile robot platform.
- Synthesizing Navigation Abstractions for Planning with Portable Manipulation Skills
CoRL 2023 · [PDF]click 2 expand abstract
We address the problem of efficiently learning high-level abstractions for task-level robot planning. Existing approaches require large amounts of data and fail to generalize learned abstractions to new environments. To address this, we propose to exploit the independence between spatial and non-spatial state variables in the preconditions of manipulation and navigation skills, mirroring the manipulation-navigation split in robotics research. Given a collection of portable manipulation abstractions (i.e., object-centric manipulation skills paired with matching symbolic representations), we derive an algorithm to automatically generate navigation abstractions that support mobile manipulation planning in a novel environment. We apply our approach to simulated data in AI2Thor and on real robot hardware with a coffee preparation task, efficiently generating plannable representations for mobile manipulators in just a few minutes of robot time, significantly outperforming state-of-the-art baselines.