Vision Transformer (ViT)

The Vision TransformerTransformerThe Transformer is an attention-based neural network architecture that processes entire sequences in parallel, enabli... (ViT) adapts the Transformer architecture, originally designed for natural language processing, to computer visionComputer VisionComputer Vision is a field of artificial intelligence that enables machines to interpret and understand visual inform... tasks. Instead of processing images through convolutional layers, ViT divides an image into fixed-size patches, linearly embeds each patch, adds positional encodings, and processes the resulting sequence through standard Transformer encoder layers. Self-attention allows each patch to attend to every other patch, enabling the model to capture global spatial relationships that convolutional networks can only approximate through deep stacking of local operations. Impact on Remote SensingRemote SensingRemote sensing is the science of collecting data about Earth's surface without direct physical contact, primarily usi... and Earth ObservationViT and its variants have rapidly become foundational in geospatial AI. Remote sensing classification benefits from ViT's ability to model long-range spatial context, such as recognizing that a patch of green pixels near water is likely a riparian zone rather than a crop field. Swin Transformer introduces hierarchical feature maps and shifted windows for efficient processing of high-resolution satellite imagerySatellite ImagerySatellite imagery consists of photographs and data captured by Earth observation satellites orbiting the planet. Thes.... BEiT and MAE variants use masked patch prediction for self-supervised pretraining on unlabeled satellite data, creating powerful foundation models. ViT-based models have achieved new state-of-the-art results on benchmarks for land cover classificationLand Cover ClassificationLand cover classification is the process of categorizing Earth's surface into distinct classes such as forest, cropla..., semantic segmentationSemantic SegmentationSemantic Segmentation is a computer vision technique that assigns a class label to every pixel in an image, enabling ..., and change detectionChange DetectionChange detection uses geospatial data and imagery to track and analyze alterations in landscapes, infrastructure, or ... in remote sensing imagery. Practical Considerations for Geospatial DeploymentViTs typically require large training datasets or transfer learningTransfer LearningTransfer Learning is a machine learning technique where a model trained on one task is repurposed for a different but... from pretrained models to outperform CNNs, which have stronger inductive biases for spatial data. They are more computationally demanding than comparably-sized CNNs, particularly for high-resolution imagery. Hybrid architectures that combine convolutional feature extraction with Transformer attention layers often provide the best balance of performance and efficiency. Multi-scale ViT variants that process images at multiple resolutions handle the diverse object sizes found in satellite imagery more effectively than single-scale approaches.