Replicating DDAT: Diffusion Policies for Robot Trajectories

An independent reproduction of DDAT in MuJoCo, evaluating the necessity of projection operators across four underactuated robotic systems.

Inspired by the ICON Lab’s research, I independently replicated the DDAT (Diffusion Dynamic Allocation Task) framework. This evaluation analyzes how projection operators enforce dynamical admissibility during open-loop execution in high-dimensional state-action spaces.

🦿 Agent 1: Hopper (12 states, 3 actions)

The projection layer is vital for the Hopper’s vertical stability. Without it, the agent fails to maintain its single-legged equilibrium.

With Projection: Stable periodic gait.
Without Projection: Immediate collapse.
Projected plots showing bounded state variables.
Unprojected plots showing rapid divergence.

🏃 Agent 2: Walker2d (18 states, 6 actions)

For the multi-link Walker2d, the DDAT framework prevents the “drifting” effect common in standard generative policies.

With Projection: Continuous forward locomotion.
Without Projection: Loss of gait coordination.
Stable joint trajectories (Projected).
Unstable joint space divergence (None).

🐆 Agent 3: HalfCheetah (18 states, 6 actions)

The HalfCheetah agent demonstrates the scalability of DDAT to high-velocity movement patterns.

Successful high-speed running.
Immediate diving/failure.
Projected performance curve.
Non-projected divergence curve.

🚁 Agent 4: Quadcopter (17 states, 4 actions)

While MuJoCo rendering is omitted for trajectory focus, the quantitative analysis shows DDAT’s ability to maintain dynamically admissible flight paths during open-loop control.

Projected Trajectory: Successful waypoint tracking.
Unprojected Trajectory: Violation of dynamics constraints.

🔍 Technical Insights

  • Projection Mechanism: My results confirm that the differentiable projection operator is the cornerstone of DDAT, ensuring that diffusion-generated samples remain within the feasible physics manifold.
  • Generative Training: I curated and trained diffusion-based datasets for each agent, experiencing the complexities of multi-modal task allocation.
  • Stability Analysis: Comparative plots highlight that without projection, high-dimensional open-loop systems suffer from accumulated error and rapid loss of equilibrium.

⚠️ Note on Variations

Minor discrepancies in performance compared to the original DDAT paper may arise from differences in MuJoCo engine versions and subtle variations in hyper-parameter tuning during the independent replication process. Despite these minor deviations, the fundamental role of the projection operator in ensuring open-loop stability remains consistently verified across all tested agents.