04.04.24 - SkySense: Setting a New Standard for Earth Observation Imagery Interpretation

Introducing SkySense, the groundbreaking Remote Sensing Fundamental Model (RSFM) recently accepted by CVPR 2024 Conference. Jointly developed by Ant’s AI innovation research department NextEvo and Wuhan University, SkySense represents a significant leap forward in Earth Observation (EO) technology.

SkySense stands out as the first billion-level multi-modal RSFM, pre-trained on a massive dataset of 21.5 million samples spanning over 8.78 million square kilometers across 40 countries and areas. This extensive dataset, totaling around 300TB of data, includes optical images, multi-spectral satellite images, and Synthetic Aperture Radar (SAR) data. Pre-training SkySense required 24,600 A100 GPU hours, highlighting the computational intensity of this endeavor.

What sets SkySense apart is its performance in downstream tasks. Through rigorous evaluations, SkySense has consistently outperformed most recent RSFMs, demonstrating significant improvements across various tasks. This achievement underscores the model's robustness and effectiveness in real-world applications.

SkySense's ability to integrate diverse modalities and handle vast amounts of data positions it as a game-changer in the field of EO imagery interpretation. Its performance and scalability make it a valuable asset for researchers and practitioners seeking advanced solutions for remote sensing tasks.

The team behind SkySense plans to incorporate the language modality for the next step, expanding its applications to encompass even more EO tasks. While the pretrained model, weights, and dataset haven't been published yet, we're keeping a close eye on developments in this area. As STDL and HEIG-VD are jointly conducting a Vision Transformer project with multi-spectral and elevation data from Switzerland, France and Italy, we're excited to see how the open-source of SkySense can benefit the community and inspire future innovation.

Link to paper: https://lnkd.in/e63VQv4M

Hashtag#RemoteSensing Hashtag#CVPR Hashtag#AI Hashtag#EarthObservation Hashtag#Research Hashtag#Innovation