Advanced Perception & Sensing

   

Robots operating in space cannot rely on direct human oversight. Their onboard perception must be reliable, rich, and interpretable under the extreme conditions of planetary surfaces, orbital proximity, and night-sky observation. Our perception research spans three fronts. For planetary rovers we develop multimodal sensor systems combining RGB, depth, and long-wave infrared (thermal) cameras to characterize soil mechanical properties and terrain traversability, including the world's first systematic study of thermal signatures under simulated Martian atmospheric conditions. For orbital robotics, the ECHO2 project contributes a markerless vision pipeline (RGB + thermal) that detects anomalies and reconstructs 4D shape of large space structures without any cooperative targets. We also contribute to robotic telescope networks through real-time transient detection pipelines applied to the BOOTES global observatory. A signature contribution is the open BASEPROD dataset—1.7 km of field rover data from the Bardenas Reales semi-desert including force–torque, thermal, LIDAR, and stereo imagery—enabling community-wide benchmarking of planetary perception methods.

KEY CAPABILITIES
Thermal terrain characterization LWIR camera fusion for soil-type classification and trafficability estimation, validated under Martian pressure and lighting.	Multimodal deep learning  Swin-based architecture trained jointly on RGB, depth, and thermal for unstructured terrain segmentation.
Markerless orbital inspection RGB+T fusion pipeline for 4D shape reconstruction and anomaly detection of large space structures without cooperative markers	Force–torque sensing Field evaluation of six-axis F/T sensors on a planetary rover for terrain interaction estimation and traction monitoring.

 

IN THE LAB & IN THE FIELD

 

SELECTED PUBLICATIONS
01	SPICE-HL3: Single-photon, inertial, and stereo camera dataset for exploration of high-latitude lunar landscapes Rodríguez-Martínez, D., van der Meer, D., Song, J., Bera, A., Pérez-del-Pulgar, C.J. and Olivares-Mendez, M.A. Scientific Data 13, 364 (2026).
02	Field Assessment of Force Torque Sensors for Planetary Rover Navigation Gerdes, L., Pérez del Pulgar, C., Castilla Arquillo, R. and Azkarate, M Journal of Intelligent & Robotic Systems, 111(4), p.122. (2025)
03	Fast Vision in the Dark: A Case for Single-Photon Imaging in Planetary Navigation Rodríguez-Martínez, D. and del Pulgar, C.J. 18th Symposium on Advanced Space Technologies in Robotics and Automation (ASTRA), Leiden, The Netherlands (2025)
04	OmniUnet: A Multimodal Network for Unstructured Terrain Segmentation on Planetary Rovers Using RGB, Depth, and Thermal Imagery Castilla-Arquillo R., Pérez-del-Pulgar C.J., Gerdes L., García-Cerezo A., Olivares-Mendez M. International Conference on Space Robotics (iSpaRo), Sendai, Japan (2025)
05	BASEPROD: The Bardenas Semi-Desert Planetary Rover Dataset Gerdes, L., Wiese, T., Castilla Arquillo, R., Bielenberg, L., Azkarate, M., Leblond, H., Wilting, F., Ortega Cortés, J., Bernal, A., Palanco, S. and Pérez del Pulgar, C. Scientific Data, 11(1), 1054 (2024).
06	Thermal Imagery for Rover Soil Assessment Using a Multipurpose Environmental Chamber Under Simulated Mars Conditions R. Castilla-Arquillo, A. Mandow, C. J. Pérez-del-Pulgar, C. Álvarez-Llamas, J. M. Vadillo and J. Laserna IEEE Transactions on Instrumentation and Measurement, 73, 1-12 (2023).

OPEN-SOURCE SOFTWARE & DATASETS
	SPICE-HL3: Single-Photon, Inertial, and Stereo Camera dataset for Exploration of High-Latitude Lunar Landscapes Access the dataset >  Supplementary material >
	BASEPROD: The Bardenas Semi-Desert Planetary Rover Dataset Access the dataset >  Supplementary material >

SEE ALSO