Robots deployed for competitive tasks must outmaneuver their opponents without sacrificing safety. Existing approaches, including safe reinforcement learning, train a single policy to achieve task success and avoid failures simultaneously, which can leave the learned policy exploitable by deliberate attacks. We propose Safety to Competence (S2C), a two-stage reinforcement learning framework that separates safety synthesis from competitive task learning. S2C learns a robust safety filter via adversarial reinforcement learning, embeds it during task policy training, and retains the same filter at deployment. In simulated touchdown games, S2C achieves the highest win rate and Elo rating and the lowest exploitability among safe reinforcement learning baselines; hardware stress tests against a human opponent further confirm its competence.
@article{wu2026competence,title={Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning},author={Wu, Ruihan and Yang, Rui and Oh, Donggeon David and Nguyen, Duy and Hu, Haimin},journal={arXiv preprint arXiv:2609.27312},year={2026},}
ICMA
Deep Hydro-Sim: Structured Residual Hydrodynamics for the RL-Based AUV Control
Ruihan Wu, Chunying Li, Qirong Lei, and 3 more authors
In 2026 IEEE International Conference on Mechatronics and Automation (ICMA), 2026
Deep Hydro-Sim bridges a high-fidelity underwater simulator and a fast rigid-body engine by learning residual hydrodynamic forces from Stonefish and injecting them into MuJoCo as a real-time plugin. The result improves open-loop replay fidelity and stabilizes downstream SAC training for AUV control without sacrificing throughput.
@inproceedings{wu2026deepsim,title={Deep Hydro-Sim: Structured Residual Hydrodynamics for the RL-Based AUV Control},author={Wu, Ruihan and Li, Chunying and Lei, Qirong and Huang, Le and Li, Pengcheng and Guo, Shuxiang},booktitle={2026 IEEE International Conference on Mechatronics and Automation (ICMA)},year={2026},}
