Simulation software platform for training and evaluation of neural-network policies for autonomous navigation of a ground unmanned vehicle

The study addresses reproducible development and testing of autonomous navigation algorithms for ground unmanned vehicles before transfer to physical devices. The article presents a simulation software platform combining a three-dimensional environment, a control server interface, an operator web interface, demonstration recording, model training, and visual analysis of autonomous agent decisions. The purpose of the work is to develop and experimentally validate a platform for studying neural-network navigation policies in a partially observable maze environment. The main approach is simulation modeling in a three-dimensional environment followed by evaluation of learning-from-demonstration algorithms on generated scenarios. The experimental case considers a mobile robotic agent that receives a camera image and range sensor data. A behavior cloning baseline is compared with a variant using agent-centric spatial context. In the correct visual runtime after retraining, the baseline achieved 10 successful episodes out of 30 in the latency scenario, while the contextual hybrid variant achieved 17 successful episodes out of 20 in a basic demonstration scenario. Physical tests of transfer from simulation to a real device were also carried out on L- and Z-shaped cardboard stands: they confirmed the operability of the transfer pipeline on a physical mobile robot and showed that the main reproducibility limitation is the low-level actuation loop without accurate odometry. The results show that the platform is suitable for demonstrating trained models and identifying discrepancies between metrics computed on static data and closed-loop agent behavior.

1. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction. Cambridge: MIT Press; 2018. 552 p.

2. Kober J., Bagnell J.A., Peters J. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research. 2013;32(11):1238–1274. https://doi.org/10.1177/0278364913495721

3. Mnih V., Kavukcuoglu K., Silver D., et al. Human-level control through deep reinforcement learning. Nature. 2015;518(7540):529–533. https://doi.org/10.1038/nature14236

4. Pomerleau D. ALVINN: An Autonomous Land Vehicle In a Neural Network. In: Advances in Neural Information Processing Systems, Denver, CO, USA. Morgan Kaufmann; 1989. P. 305–313.

5. Bojarski M., Del Testa D., Dworakowski D., et al. End to End Learning for Self-Driving Cars. arXiv. URL: https://arxiv.org/abs/1604.07316 [Accessed 30th May 2026].

6. Ross S., Gordon G., Bagnell D. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, 11–13 April 2011, Fort Lauderdale, USA. PMLR; 2011. P. 627–635. URL: https://proceedings.mlr.press/v15/ross11a.html

7. Tobin J., Fong R., Ray A., Schneider J., Zaremba W., Abbeel P. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. arXiv. URL: https://arxiv.org/abs/1703.06907 [Accessed 30th May 2026].

8. Juliani A., Berges V.-P., Teng E., et al. Unity: A General Platform for Intelligent Agents. arXiv. URL: https://arxiv.org/abs/1809.02627 [Accessed 30th May 2026].

9. Zhao W., Queralta J.P., Westerlund T. Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey. In: 2020 IEEE Symposium Series on Computational Intelligence, 01–04 December 2020, Canberra, ACT, Australia. IEEE; 2020. P. 737–744. https://doi.org/10.1109/SSCI47803.2020.9308468

10. Muratore F., Ramos F., Turk G., Yu W., Gienger M., Peters J. Robot Learning From Randomized Simulations: A Review. Frontiers in Robotics and AI. 2022;9:799893. https://doi.org/10.3389/frobt.2022.799893

11. Koenig N., Howard A. Design and use paradigms for Gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 28 September – 02 October 2004, Sendai, Japan. IEEE; 2004. P. 2149–2154. https://doi.org/10.1109/IROS.2004.1389727

12. Shah Sh., Dey D., Lovett Ch., Kapoor A. AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. In: Field and Service Robotics: Results of the 11th International Conference, 12–15 September 2017, Zurich, Switzerland. Cham: Springer; 2018. P. 621–635. https://doi.org/10.1007/978-3-319-67361-5_40

13. Dosovitskiy A., Ros G., Codevilla F., Lopez A., Koltun V. CARLA: An Open Urban Driving Simulator. In: Proceedings of the 1st Annual Conference on Robot Learning, 13–15 November 2017, Mountain View, CA, USA. PMLR; 2017. P. 1–16. URL: https://proceedings.mlr.press/v78/dosovitskiy17a.html

14. Raffin A., Hill A., Gleave A., Kanervisto A., Ernestus M., Dormann N. Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research. 2021;22(268):1–8.

15. Towers M., Kwiatkowski A., Terry J., et al. Gymnasium: A Standard Interface for Reinforcement Learning Environments. arXiv. URL: https://arxiv.org/abs/2407.17032 [Accessed 30th May 2026].