RESEARCH PROGRAM 01

Marine AI & Ocean Understanding

See the ocean in new ways.

This program develops marine AI, computer vision, multimodal sensing, 3D/4D mapping, foundation models, resilient communications and environmental intelligence. It builds persistent observation systems and shared data resources that turn heterogeneous visual, acoustic, physical, chemical and biological measurements into actionable understanding of coastal environments.

SUB-DIRECTION 01

Marine AI & Foundation Models

We build general-purpose models that see marine life the way specialists do. They find, outline and classify organisms and habitats across reefs, species and imaging conditions, and adapt to new tasks with little or no new labelling.

What we study

Detection & segmentation
Outlining corals, fish and other organisms, down to individual instances.
Classification & recognition
From coral or non-coral to growth form, genus and species.
Label-efficient adaptation
Adapting foundation models to marine imagery without new annotation.
Generalizable learning
Models that hold up in murky water, at new sites and on unseen corals.

Representative work · 3

SUB-DIRECTION 02

Multimodal Marine Reasoning & Understanding

We link what models see with language and expert knowledge, so that they can describe marine scenes, answer questions and reason about behaviour and events in images and long underwater video.

What we study

Captioning & description
Scene- and instance-level descriptions of marine life.
Visual question answering
Expert-level answers about species, behaviour and conservation.
Video & event reasoning
Finding brief, sparse events in long footage and explaining them.
Open-world perception
Recognising species and objects beyond a fixed list of labels.

Representative work · 3

SUB-DIRECTION 03

Marine 3D/4D Mapping

We turn underwater video into photorealistic 3D maps. We estimate how the camera moves, build the map in real time and follow the scene through time, separating moving animals and removing the effects of water, so that repeated surveys can show how habitats change.

What we study

Camera motion
Robust motion from a single 360° camera, the basis for mapping.
Real-time photorealistic mapping
A photorealistic 3D map built as the video streams in.
Feed-forward 3D reconstruction
Point maps and camera poses from underwater video in one pass.
Dynamic, media-aware 4D
Separating moving animals from the reef and undoing water effects.

360DVO estimating the camera trajectory and a sparse 3D map from underwater 360-degree video — **360DVO** IEEE RA-L 2026Deep visual odometry for a single 360° camera: distortion-aware spherical features and omnidirectional bundle adjustment.Project page ↗Code ↗arXiv ↗

Marine Photo-SLAM mapInteractive 3D viewer · needs WebGL 2

Drag to orbit · scroll to zoom

Photo-SLAM CVPR 2024Real-time localisation and photorealistic mapping; it even runs on an embedded Jetson AGX Orin.Project page ↗Code ↗

WAT3R reconstructing an underwater scene: input frame and depth map with the growing 3D point map and camera poses — **WAT3R** arXiv 2026Feed-forward underwater 3D reconstruction: point maps and camera poses from video in one pass, adapting to light attenuation and backscatter.Project page ↗arXiv ↗

NemoSplat demo: 4D Gaussian reconstruction of a turtle and fish moving over a reef; the other tabs show result figures — **NemoSplat** arXiv 2026Feed-forward 4D Gaussian splatting from uncalibrated marine video: moving animals are separated from the static reef and water effects removed in one pass.Project page ↗arXiv ↗

How they fit together

1
Where is the camera?360DVO · Robust motion from a single 360° camera.
2
What does it look like?Photo-SLAM · A photorealistic map, built in real time.
3
What is the 3D shape?WAT3R · Point maps and poses in one pass, despite the water.
4
What moves, what does water hide?NemoSplat · Moving animals and water effects, for 4D.

Connection with Program 02. Here the focus is the map, and the camera motion it is built from: reconstructing the environment and its change. Robot navigation and autonomy that use such maps sit in Marine Embodied Intelligence & Systems.

SUB-DIRECTION 04

Persistent Ocean Observation

We keep sensors in the water long enough to see trends rather than snapshots, and we share the data we collect. Our open datasets cover coral reefs and marine life in images and video, and the AI methods we build and test on them can analyse the continuous records of future observation systems.

What we study

Multimodal sensing
Visual, acoustic, physical, chemical and biological measurements together.
Platforms & communications
Fixed stations and repeated surveys, with data links that keep sensors connected.
Open datasets & benchmarks
Annotated coral and marine data for training and testing models.
AI for continuous monitoring
Applying our models to long-term observation records.

Our open datasets · 11

Coral AI 4

Marine AI 7

From datasets to future observation. The AI methods built on these datasets, from coral segmentation to video reasoning, can be applied to the continuous records of future observation systems such as SeaSeeUST, turning long-term monitoring into timely, actionable information.

Marine AI & Ocean Understanding

Marine AI & Foundation Models

Detection & segmentation

Classification & recognition

Label-efficient adaptation

Generalizable learning

Multimodal Marine Reasoning & Understanding

Captioning & description

Visual question answering

Video & event reasoning

Open-world perception

Marine 3D/4D Mapping

Camera motion

Real-time photorealistic mapping

Feed-forward 3D reconstruction

Dynamic, media-aware 4D

Persistent Ocean Observation

Multimodal sensing

Platforms & communications

Open datasets & benchmarks

AI for continuous monitoring

Coral AI 4

Marine AI 7

Connections with other programs

Marine Embodied Intelligence & Systems

Marine Ecosystem & Biodiversity Intelligence

Marine Conservation, Restoration & Nature-based Solutions