EC-WAMI: SfM and 3D From Neuromorphic (Event) Cameras for Remote Sensing

Paper

Poster

Dataset

Wide-Area Motion Imagery (WAMI) is a Remote Sensing (RS) approach for continuous surveillance across vast areas, used in urban planning, disaster response, and security. Traditionally, RGB cameras have been the primary sensors in WAMI but suffer from motion blur, low resolution, and limited dynamic range.

This project presents the first application of Event Cameras (ECs) for camera pose optimization in WAMI, leveraging their asynchronous brightness detection, ultra-high resolution, and low power consumption. ECs enable robust 3D reconstruction in challenging conditions.

Sample Corrupted Image and recovered camera poses

Limitations of RGB Cameras in Adverse WAMI Scenarios

Since real RGB images from applicable scenarios were unavailable, we simulated traditional camera-captured scenes using systematic image perturbation methods.

Neuromorphic Camera

Rebecq, H.; Ranftl, R.; Koltun, V.; Scaramuzza, D. High Speed and High Dynamic Range Video with an Event Camera. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 1964–1980.

ABQ Dataset

abq_combine_final.mp4

RIT Dataset

rit_combine_final.mp4

Event data are simulated from RGB images using an event simulator and frames are reconstructed with a frame reconstructor.

Pose Optimization

Camera trajectory and differences in position compared to groundtruth positions

3D Reconstructed Scenes

Real Dataset: eABQ

abq_gs_clipped.mp4

Synthetic Dataset: eRIT

rit_gs_fitted.mp4

References

Nkrumah, I., Moshrefizadeh, M., Tahri, O., Blasch, E., Palaniappan, K., & AliAkbarpour, H. (2024). EC-WAMI: Event Camera-Based Pose Optimization in Remote Sensing and Wide-Area Motion Imagery. Sensors, 24(23), 7493. pdf download BibTex

Page updated

Google Sites

Report abuse