ARXIV:2603.12020 · AGENTS · SUBMITTED 02 APR · 02:30 UTC · FRESHNESS STALE

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application

arXiv

A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

Blocked on Code›Score7.0Evidence unverified

Opportunity summary

Pain A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

Evidence 0 refs | 0 sources | 17% coverage

Blocker Evidence unverified

Open Build Read PDF Signal Canvas Track

PROBLEM

A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking. However, bridging the "sim-to-real" gap and managing high training latencies remain significant bottlenecks for practical deployment.

METHOD

Full abstract

Deep Reinforcement Learning (DRL) offers a robust alternative to traditional control methods for autonomous underwater docking, particularly in adapting to unpredictable environmental conditions. However, bridging the "sim-to-real" gap and managing high training latencies remain significant bottlenecks for practical deployment. This paper presents a systematic approach for autonomous docking using the Girona Autonomous Underwater Vehicle (AUV) by leveraging a high-fidelity digital twin environment. We adapted the Stonefish simulator into a multiprocessing RL framework to significantly accelerate the learning process while incorporating realistic AUV dynamics, collision models, and sensor noise. Using the Proximal Policy Optimization (PPO) algorithm, we developed a 6-DoF control policy trained in a headless environment with randomized starting positions to ensure generalized performance. Our reward structure accounts for distance, orientation, action smoothness, and adaptive collision penalties to facilitate soft docking. Experimental results demonstrate that the agent achieved a success rate of over 90% in simulation. Furthermore, successful validation in a physical test tank confirmed the efficacy of the sim-to-reality adaptation, with the DRL controller exhibiting emergent behaviors such as pitch-based braking and yaw oscillations to assist in mechanical alignment.

RESULT

ScienceToStartup currently rates this 7.0/10 on the public viability pass. Experimental results demonstrate that the agent achieved a success rate of over 90% in simulation.

WHY NOW

Agents moved forward this cycle; last verified April 2026. Public score 7.0/10.

Continue into Read for claims, analysis, references, and neighboring papers.

Opportunity summary

Score7.0

PainA systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

Evidence0 refs | 0 sources | 17% coverage

Blockermissing authors

Analysis summary

A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

VerifiedSource: PDF linkedPartialPaperPack: 3 of 4 citation fields filledMissingMissing fields: authorsPartialProof: unverified proof status

Competitive landscape

A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

Segment

Agents

Adoption evidence

No public code link in the paper record yet

Commercial read

7.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

References(17)

Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles

2025Sümer Tunçay, Alain Andres et al.

A Sim-to-Real Transfer Framework for Enhancing Marine Vehicle Performance in Ocean Environments

2025Ze Zheng, Zihao Wang et al.

Three-Dimensional Binary Marker: A Novel Underwater Marker Applicable for Long-Term Deployment Scenarios

2025A. Chaarani, P. Cieślak et al.

From Simulation to Reality: Deep Reinforcement Learning for Autonomous Underwater Vehicle Docking

2025Vibhav Bharti, Sümer Tunçay et al.

A Docking Station Proposal for the Girona I-AUV

2025A. Chaarani, Joan Esteba et al.

Stonefish: Supporting Machine Learning Research in Marine Robotics

2025Michele Grimaldi, P. Cieślak et al.

Adaptive reward shaping based reinforcement learning for docking control of autonomous underwater vehicles

2025Shuguang Chu, Mingwei Lin et al.

The docking control system of an autonomous underwater vehicle combining intelligent object recognition and deep reinforcement learning

2025Chao-Ming Yu, Yu-Hsin Lin

Comparative Evaluation of Reinforcement Learning and Model Predictive Control for 6DoF Position Control of an Autonomous Underwater Vehicle

2025Sümer Tunçay, Alain Andres et al.

Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning

2025Nvidia Mayank Mittal, Pascal Roth et al.

Autonomous Underwater Vehicle Docking Under Realistic Assumptions Using Deep Reinforcement Learning

2024N. Palomeras, P. Ridao

Auv 3d Docking Control Using Deep Reinforcement Learning

2023Tianze Zhang, X. Miao et al.

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

2021Mihir Patil, Bilal Wehbe et al.

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

2020Qilei Zhang, Jinying Lin et al.

Docking Control of an Autonomous Underwater Vehicle Using Reinforcement Learning

2019E. Anderlini, G. Parker et al.

Stonefish: An Advanced Open-Source Simulation Tool Designed for Marine Robotics, With a ROS Interface

2019P. Cieślak

Girona 500 AUV: From Survey to Intervention

2012D. Ribas, N. Palomeras et al.

{ "contract_version": "paper-r2", "paper_id": "2c153124-cce2-40ee-a2bd-a1555f7e1946", "arxiv_id": "2603.12020", "canonical_route": "/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "active_tab": "synced from current hash by the drawer client", "selected_artifact": "sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "endpoints": { "paper_pack": "/api/v1/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application/paper-pack", "build_passport": "/api/v1/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application/build-passport", "mcp_resource": "sciencetostartup://surfaces/paper-workspace" } }

{ "surface": "paper", "mode": "paper", "query": "Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application", "normalized_query": "2603.12020", "route": "/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "paper_ref": "sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "topic_slug": null, "benchmark_ref": null, "dataset_ref": null }

{ "@context": "https://schema.org", "@graph": [ { "@type": "WebPage", "@id": "https://sciencetostartup.com/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application#webpage", "url": "https://sciencetostartup.com/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "name": "Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application", "description": "A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.", "isPartOf": { "@id": "https://sciencetostartup.com/#website" } }, { "@type": "ScholarlyArticle", "@id": "https://sciencetostartup.com/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application#scholarlyArticle", "headline": "Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application", "description": "A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.", "url": "https://sciencetostartup.com/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application", "sameAs": "https://arxiv.org/abs/2603.12020", "identifier": { "@type": "PropertyValue", "propertyID": "arXiv", "value": "2603.12020" }, "isAccessibleForFree": true, "isPartOf": { "@id": "https://sciencetostartup.com/#website" }, "datePublished": "2026-03-12T15:01:25.000Z", "citation": [ { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "2efedec98f76851b8007353fcb9d50b4bed88a1d" }, "url": "https://www.semanticscholar.org/paper/2efedec98f76851b8007353fcb9d50b4bed88a1d" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "6f47f2e754bebe054ee7d4527a835149257309e6" }, "url": "https://www.semanticscholar.org/paper/6f47f2e754bebe054ee7d4527a835149257309e6" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "fe64ee016f93aaf12d7222f2bb1fee718f2acadf" }, "url": "https://www.semanticscholar.org/paper/fe64ee016f93aaf12d7222f2bb1fee718f2acadf" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "287e3a04a9398679358a021cf2e53e67dace74c1" }, "url": "https://www.semanticscholar.org/paper/287e3a04a9398679358a021cf2e53e67dace74c1" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "fc139cb82664fd43ac981eee061324fc0aec2fa9" }, "url": "https://www.semanticscholar.org/paper/fc139cb82664fd43ac981eee061324fc0aec2fa9" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "13982217c2542ed89519ba0295c9ccaa7fecb147" }, "url": "https://www.semanticscholar.org/paper/13982217c2542ed89519ba0295c9ccaa7fecb147" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "b0be2ba27d119248c806b3eefb6bd5ad254266af" }, "url": "https://www.semanticscholar.org/paper/b0be2ba27d119248c806b3eefb6bd5ad254266af" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "c39a2f2cddf744552e143ce33e19b6b1aaf2208b" }, "url": "https://www.semanticscholar.org/paper/c39a2f2cddf744552e143ce33e19b6b1aaf2208b" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "953d621a9573572fadbc4832dc77a0958723bc25" }, "url": "https://www.semanticscholar.org/paper/953d621a9573572fadbc4832dc77a0958723bc25" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "98fbe6e1b37f1c5b1c2e3edff512517c41b56cf3" }, "url": "https://www.semanticscholar.org/paper/98fbe6e1b37f1c5b1c2e3edff512517c41b56cf3" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "6f1c4a6d6dfb120df816d7e488babb16587c7406" }, "url": "https://www.semanticscholar.org/paper/6f1c4a6d6dfb120df816d7e488babb16587c7406" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "0d82360a4da311a277607db355dda3f196e8eb3d" }, "url": "https://www.semanticscholar.org/paper/0d82360a4da311a277607db355dda3f196e8eb3d" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "ce803622f12b240bc34617140b15e4618c0391aa" }, "url": "https://www.semanticscholar.org/paper/ce803622f12b240bc34617140b15e4618c0391aa" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "285f587c11aef3a60cb27693b4166d000bd1ddfa" }, "url": "https://www.semanticscholar.org/paper/285f587c11aef3a60cb27693b4166d000bd1ddfa" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "375d5a31e1f15b4b4dab3428f0517e014fa61a91" }, "url": "https://www.semanticscholar.org/paper/375d5a31e1f15b4b4dab3428f0517e014fa61a91" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "a78a199d2391d03b92ac59d09a43dbeede4f4c40" }, "url": "https://www.semanticscholar.org/paper/a78a199d2391d03b92ac59d09a43dbeede4f4c40" }, { "@type": "ScholarlyArticle", "identifier": { "@type": "PropertyValue", "propertyID": "SemanticScholar", "value": "cf599304de118f5fba1f4ae3ba4e307064382bd8" }, "url": "https://www.semanticscholar.org/paper/cf599304de118f5fba1f4ae3ba4e307064382bd8" } ], "additionalProperty": [ { "@type": "PropertyValue", "propertyID": "viabilityScore", "value": 7 }, { "@type": "PropertyValue", "propertyID": "researchDomain", "value": "Agents" } ] }, { "@type": "BreadcrumbList", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://sciencetostartup.com" }, { "@type": "ListItem", "position": 2, "name": "Agents", "item": "https://sciencetostartup.com/topics" }, { "@type": "ListItem", "position": 3, "name": "Sim-to-reality adaptation for Deep Reinforcement Learning ap", "item": "https://sciencetostartup.com/paper/sim-to-reality-adaptation-for-deep-reinforcement-learning-applied-to-an-underwater-docking-application" } ] } ] }

Competitive landscape

A systematic approach for sim-to-reality adaptation in Deep Reinforcement Learning for autonomous underwater docking.

Segment

Agents

Adoption evidence

No public code link in the paper record yet

Commercial read

7.0/10 public viability

Direct

not classified

Adjacent

not classified

Substitute

not classified

Unknown

not classified

References(17)

Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles

2025Sümer Tunçay, Alain Andres et al.

A Sim-to-Real Transfer Framework for Enhancing Marine Vehicle Performance in Ocean Environments

2025Ze Zheng, Zihao Wang et al.

Three-Dimensional Binary Marker: A Novel Underwater Marker Applicable for Long-Term Deployment Scenarios

2025A. Chaarani, P. Cieślak et al.

From Simulation to Reality: Deep Reinforcement Learning for Autonomous Underwater Vehicle Docking

2025Vibhav Bharti, Sümer Tunçay et al.

A Docking Station Proposal for the Girona I-AUV

2025A. Chaarani, Joan Esteba et al.

Stonefish: Supporting Machine Learning Research in Marine Robotics

2025Michele Grimaldi, P. Cieślak et al.

Adaptive reward shaping based reinforcement learning for docking control of autonomous underwater vehicles

2025Shuguang Chu, Mingwei Lin et al.

The docking control system of an autonomous underwater vehicle combining intelligent object recognition and deep reinforcement learning

2025Chao-Ming Yu, Yu-Hsin Lin

Comparative Evaluation of Reinforcement Learning and Model Predictive Control for 6DoF Position Control of an Autonomous Underwater Vehicle

2025Sümer Tunçay, Alain Andres et al.

Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning

2025Nvidia Mayank Mittal, Pascal Roth et al.

Autonomous Underwater Vehicle Docking Under Realistic Assumptions Using Deep Reinforcement Learning

2024N. Palomeras, P. Ridao

Auv 3d Docking Control Using Deep Reinforcement Learning

2023Tianze Zhang, X. Miao et al.

Deep Reinforcement Learning for Continuous Docking Control of Autonomous Underwater Vehicles: A Benchmarking Study

2021Mihir Patil, Bilal Wehbe et al.

Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle

2020Qilei Zhang, Jinying Lin et al.

Docking Control of an Autonomous Underwater Vehicle Using Reinforcement Learning

2019E. Anderlini, G. Parker et al.

Stonefish: An Advanced Open-Source Simulation Tool Designed for Marine Robotics, With a ROS Interface

2019P. Cieślak

Girona 500 AUV: From Survey to Intervention

2012D. Ribas, N. Palomeras et al.

Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application

Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(17)

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline

Claim map

Constellation map

Competitive landscape

Buzz

PDF

References(17)

Related Papers

Related Resources

Subscribe to the weekly brief

Build artifacts

Brief

Experiment plan

Validation checklist

Scientific founder

Translational engineer

Domain operator

GTM lead

Regulatory/clinical advisor

Timeline