Teaching a network to read the Earth, with help from a second satellite
A convolutional network looks at a thumbnail of Earth from a million miles away and guesses the longitude and latitude the camera was pointed at. This is how Copernicus Sentinel-2 imagery, a circular way of handling longitude, and an ensemble of models took a badly retrained network from 1,938 km off to under 200 km, past the shipped model that trained on eleven years and 51,000 frames.
Try it live
What runs here is the Sentinel-2 model with the circular-longitude head, which averages 401 km. It is a hand-written TypeScript port of the PyTorch CNN and runs entirely in your browser, with no server and no WebGL. Pick a NASA EPIC frame and it places a pin on the globe. The 197 km figure further down is the full ensemble, which is measured offline.
The problem
An easy question with an expensive answer
NASA’s DSCOVR spacecraft sits at Lagrange point 1, a million miles sunward of Earth, and its EPIC camera looks straight down at the day side of the planet. Given one of those full-disk pictures, where was the sub-satellite point, the spot on the ground directly beneath the camera’s axis?
The shipped model answers that to within 596 km on average, but it earned that by training on the entire EPIC archive, roughly 51,000 frames spanning eleven years. I wanted to know whether I could retrain it locally, from almost nothing, and get that accuracy back using a completely different satellite as a teacher.
The second satellite
Enlisting Copernicus Sentinel-2
A network that only ever sees Earth from one vantage point has a brittle sense of geography, so I gave it a second teacher. The Copernicus Data Space serves a small true-colour thumbnail for every Sentinel-2 scene, anonymously and without an API key, each one tagged with an exact footprint. I harvested 2,812 of them, spread from −56° to +72° latitude and across the full sweep of longitude, and framed them as the same task: image in, coordinates out.
Same question, very different pictures: 100-km close-ups of coastlines, deserts, ice and farmland instead of one blue marble. That is the variety a single vantage point cannot supply.
The descent
Every idea, measured
Mean error on the same 75 frames as each change is applied. Shorter is better.
A CNN trained only on the handful of full-disk frames available locally. It memorises them and flails on anything new.
More local data helps a little, but a lone vantage point can only teach so much. It still overfits.
Bolting a frozen Sentinel-trained backbone straight onto EPIC. The domains are too different and it collapses.
Warm-start the convolutions on Copernicus imagery, then fine-tune on EPIC. First real signal.
Train on both at once. 2,800+ Sentinel scenes regularise the features while EPIC steers the output.
Average the model across the tail of training. Sands down the worst outliers.
The shipped model, trained on eleven years and ~51,000 EPIC frames.
Predict longitude as (sin, cos) instead of a raw number, so there is no seam at the ±180° dateline. The Pacific outliers vanish and the model overtakes the incumbent.
The breakthrough
Co-training, not hand-me-downs
The obvious move, pre-training on Sentinel and then fine-tuning on EPIC, helped, but the 284-frame fine-tune kept overwriting what the Sentinel data had taught. Freezing the backbone instead was worse (8,713 km): a feature extractor tuned for 100-km crops has nothing useful to say about a whole planet.
What worked was co-training: feeding balanced batches of EPIC and Sentinel to one shared network at the same time. The thousands of Sentinel scenes hold the convolutional features honest while the EPIC frames steer the final coordinate mapping. The data budget is the same as the plain baseline and the only difference is that second teacher, and it cut the error from 1,628 to 747 km in one step. Against the plain baseline, the Sentinel signal alone is worth a 75% reduction by the time the pipeline is done.
Stochastic Weight Averaging, which averages the model over the tail of training, then filed down the worst mistakes and closed to within ~60 km of the incumbent. What remained was a single stubborn failure mode.
The scoreboard
Loading metrics
The unlock
The whole gap was one seam on the map
Reading the errors one by one showed a pattern. Latitude was nearly perfect, and every large miss sat in the middle of the Pacific, near the ±180° dateline. The cause was not the ocean, it was the maths. Regressing longitude as a plain number puts a discontinuity exactly there: 179° and −179° are neighbours on the globe but a world apart to the loss, so the model flips hemispheres.
Predict longitude as (sin θ, cos θ) instead of a raw degree and the seam disappears. The dateline becomes another place on a smooth circle.
That one change did most of the remaining work. The 95th-percentile error fell from ~1,900 km to 804 km, the mean dropped to 401 km, and the model landed within 2,000 km on 100% of test frames. The Sentinel-taught geography had been there all along. It needed a coordinate system that respects a round planet.
One caveat on the scoreboard: the incumbent probably trained on these exact test dates, so it had home advantage. The enhanced model, seeing them for the first time, still won on every measure.
Breaking 200
Ten small models beat one big one
Past the incumbent, the obvious next levers stalled. Quadrupling the input resolution to 128×128 and swapping in a deeper network barely moved the result. A pretrained ImageNet ResNet made things worse: filters tuned for cats and cars have nothing to say about a whole planet. Every model plateaued around 380–400 km, capped by the same handful of near-featureless mid-Pacific frames.
What worked was not a bigger model but more of them. Train a spread of circular co-trained models with different seeds and recipes, then average their predictions, and each model’s idiosyncratic mistakes cancel. A single model scores 376 km, two averaged 281, and ten 191 km, under the target, with 100% of frames placed inside 2,000 km.
The number is the average of every trained member, not the best one. With a small, hard dataset, variance is usually the problem rather than bias.
The long road here
Three years, 88 commits, two long silences
None of this arrived in a weekend. The repository’s history is messier than the write-up: detours, long gaps, and a project picked up and put down until it worked.
- Jun 2023
First commit
An empty repo and a hunch that a network could learn geography from pictures of Earth.
- Nov–Dec 2023
Autoencoders & detours
Downloaders, TensorBoard, an autoencoder, then side quests into MediaPipe hand-tracking, an Orca-2 LLM and a Siamese MNIST net. One commit reads: "Don't know what i'm doing."
- 2024
Silence
Fourteen months of nothing. The side project sits, untouched.
- Feb 2025
The coordinate idea
It crystallises: world-map plots, pulling lat/lon out of image filenames, a first training loop. Commit log: “Overwhelmed rn.” Then: “Make it learn.”
- 2025
Silence, again
Another eleven-month gap. Life happens; the blue marble waits.
- Feb 2026
The rebuild
Thirty-seven commits in one month. NASA EPIC API, async downloaders, a date-folder pipeline, a full module consolidation, EDA, and the LocationRegressor CNN that ships today.
- Apr–May 2026
Polish
An eleven-bug correctness sweep and a tool that renders the model's learned world map, clusters of feature-space that line up with real continents.
- Jul 2026
This chapter
Copernicus Sentinel-2 enters. Co-training, weight averaging and a circular-longitude head take the model past the incumbent it was measured against.
How to read this
Fair test. Every model is scored on one pinned split of 75 EPIC frames from 6 dates. The retrained baseline and enhanced models never see those dates during training.
What each change bought. A baseline with no Sentinel scores 1,628 km. Adding Sentinel-2 co-training, a circular-longitude head, weight averaging and finally an ensemble drives the mean under 200 km, a reduction of about 88%, past the incumbent that trained on 25× more data.
Tools. PyTorch on Apple MPS for training, NASA EPIC and Copernicus Sentinel-2 STAC for data, and a from-scratch TypeScript CNN for the demo above.