Overcoming cross-satellite challenges: HKBU Professor Michael Ng pioneers a new era in remote sensing scene classification

Scene classification is a vital component of remote sensing image interpretation, paving the way for efficient environmental conservation, land use management, and disaster monitoring. However, as satellite technologies advance and environments continue to evolve, new land-cover categories are encountered sequentially. This poses a significant challenge to traditional deep learning methods, which often require resource-intensive retraining from scratch to expand their recognisable scope.

The challenge of domain shifts and generalisation

To adapt to evolving data streams, Professor Michael Ng, Chair Professor in Mathematics and Data Science, led a team to apply incremental learning. This technique aims to acquire new knowledge continuously, while maintaining previously learned information, avoiding what is known as catastrophic forgetting. Yet one significant hurdle remains: the domain shift problem.

Remote sensing data collected from different satellites and diverse geographic regions exhibit substantial distribution gaps, which occur due to differences in imaging conditions and sensor parameters. When an incremental learning model is trained on a source domain and then applied to an unseen target domain, it frequently experiences severe performance degradation. Consequently, mitigating these domain shifts is vital for cross-satellite generalisable incremental scene classification.

Historically, alleviating domain shifts has relied on two primary paradigms: Domain Adaptation and Domain Generalisation. Domain Adaptation focuses on aligning feature distributions between source and target domains. However, its real-world feasibility is limited because it relies heavily on prior access to target domain data. In contrast, Domain Generalisation trains a model to generalise to unseen target domains using only source data. "This approach offers realisable feasibility, providing a practical solution for the continuous evolution of remote sensing data. Given the constant variability of satellite imagery, Domain Generalisation aligns perfectly with practical deployment needs," Professor Ng said.

Introducing Adaptive Mixture-of-Experts Distillation

To tackle these complex problems, the research team has proposed Adaptive Mixture-of-Experts Distillation (AMoED) in their published paper "Adaptive Mixture-of-Experts Distillation for Cross-Satellite Generalizable Incremental Remote Sensing Scene Classification".

Formulated for cross-satellite generalisable incremental remote sensing scene classification, this novel framework takes a high-level semantic learning approach, and the professor explained that "Instead of updating the model directly from raw data streams, which exacerbates the erasure of previous knowledge, AMoED continuously acquires new information through the coordinated guidance of multiple domain-specific experts".

The process unfolds strategically by first training domain-specific experts independently, using data from each source domain. Next, the predictions of these experts are adaptively integrated based on a domain-agnostic confidence measure. This high-level semantic learning pipeline facilitates the formation of universal class concepts with strong generalisability across domains. By relying on expert guidance rather than direct exposure to large volumes of new data, the model can focus on task-specific category updates without severely disrupting the parameters associated with previously learned knowledge.

Innovative training mechanisms and benchmark performance

AMoED employs unique joint training strategies to ensure stable knowledge acquisition. The standard approach of combining an exemplar set with full training data often creates an imbalance, inducing a task-recency bias. To rectify this, AMoED constructs an equi-partite subset, which combines exemplars with uniformly sampled instances from new classes, balancing the data perfectly to maintain both plasticity and memory stability.

Furthermore, the model addresses geospatial and sensor-induced deviations through shallow-layer style mixing. Because texture patterns and spatial structures primarily encode discrepancies in the network's shallow layers, this mixing operation effectively mitigates the interference of domain discrepancies during the learning process.

Extensive experiments were conducted on four distinct remote sensing datasets: Merced, AID, NWPU, and PatternNet. The evaluations tested shared land cover types such as airports, beaches, forests, and storage tanks across several geographic regions. The proposed AMoED method consistently achieved state-of-the-art performance across different settings, and Professor Ng said "By pioneering the exploration of domain-generalised incremental learning, AMoED establishes a crucial benchmark for the future of cross-satellite scene classification amidst continuously evolving data streams."

Full paper on IEEE Transactions on Circuits and Systems for Video Technology: https://ieeexplore.ieee.org/document/11124203
Professor Ng's research profile: https://scholars.hkbu.edu.hk/en/persons/kwok-po-ng