This background sound leaks your exact location

TL;DR: Using barely audible Doppler curves from propeller planes in a video's background audio, combined with ADS-B flight data, delayed CPA times, and silence filtering, the author narrows the search from millions of candidate points to the final location. # Pinpointing Your Exact Location Using Aircraft Doppler Curves in Background Audio ## A Video with Almost No Visual Information I've found off-grid buildings deep in the rainforest, things 600 meters underwater, and even things I can't share on YouTube. But what I'm sharing today is, without exaggeration, the most powerful geolocation clue I've ever encountered. To demonstrate this, I'm going to use it to locate this video. It is a video, but the picture barely changes. The only information you have is that it was filmed on September 8, somewhere in the contiguous United States. That's it. But with this information, this is possible. I'll show you exactly how. If you want to try it yourself, pause here. The link is in the description. ## Comparing It with Geolocation Using Thunder I previously made a video about geolocating using thunder. The advantage of thunder is that it travels very far and penetrates buildings extremely well. The downside is that we can only achieve an accuracy of around 5 kilometers—basically, it can tell you which city something was filmed in, and perhaps which part of that city. The method we're using today has the same advantages, but a completely different level of accuracy. I'm going to play two audio clips. One contains the information needed to find this location; the other doesn't. See if you can tell which is which. You probably can't, because you can't hear it at all. But if I open it here in a spectrogram, we can see all the frequencies. The x-axis is time, and the y-axis is frequency. Even then, you still can't make anything out. Only when I zoom in to the right frequency and then increase the resolution do you start to see it. Here's another example: this is an aircraft's Doppler curve, and we're going to use it to locate this video. ## A Note on Complexity To be clear, this method is extremely powerful, but it also requires a lot of complex technical work. It took me months to build, with more than 5,000 lines of code. I was working with 50 GB of PARK files, 3D physics modeling, nationwide DEM data, and more. Anyway, this is going to be a long video, but I'll explain every step in a way anyone can understand, even if you know nothing about any of this. ## Reading Two Doppler Curves from the Spectrogram Let's look at the rest of this audio. Here's one Doppler curve. Moving farther to the right, you can see another one that's much louder, with harmonics above it. There should be another one here too. So these two are the ones we're going to use. These Doppler curves are produced by aircraft—specifically, propeller planes. I've boosted some frequencies so you can hear them more clearly. When I play the louder one, you can easily hear that it's a propeller plane. ## Narrowing the Search with ADS-B Data Why are we interested in Doppler curves? There's something called ADS-B data, which contains information about almost every aircraft in the sky—position, speed, altitude, and lots of other data. My initial idea was: could we narrow the search to places where you would hear one aircraft and then another 920 seconds later? To do this, we first need to obtain all the ADS-B data. They have a paid API and a Python package specifically for fetching the data, but there's a huge amount of it. So I ran it for several hours and retrieved all the flight data for the contiguous United States on September 8. By default, it downloads the entire flight path, so you'll see flights going to Hawaii or other countries, but every flight that passed over the United States that day is in this dataset, including the two planes in our recording. ADS-B data has a field called the type code, but it doesn't distinguish propeller planes from jets. I did some research and found a lookup table that distinguishes the two, but it had some missing data. So instead of directly filtering for propeller planes, I excluded all known jet aircraft types. ## CPA and Delayed CPA Time Now for the time-offset filter. This pink line is the ADS-B data for a flight, including coordinates, altitude, timestamps, and so on. What we're going to do is lay out a grid of points spaced 200 meters apart, then record when each point would hear this flight. There are two new terms here. **CPA (Closest Point of Arrival)**: For an observer, CPA is the position where the aircraft is closest to that observer. This matters because "hearing an aircraft" isn't a single point in time—the entire Doppler curve lasts about 2 minutes. We need to use the CPA time, roughly in the middle, when the aircraft is closest and loudest. **Delayed CPA time**: The time when the observer should hear the CPA. This accounts for the sound having to travel from here to the listener more than 3 kilometers away. In this example, the CPA time is 22:46:34, but the delayed CPA time is 22:46:46. The second important thing is that the aircraft is in the sky, so the distance has to be the slant range, not just the horizontal distance. ADS-B data gives us the aircraft's altitude, but that's its altitude above sea level, not above the ground. So to obtain the slant range, we need to know the elevation of each grid point. For this, we use a **DEM (Digital Elevation Model)**. I do this for every point, using a fairly conservative radius of 10 kilometers—beyond that, you might not hear the plane even under ideal conditions. After doing this for all flights in the area, essentially every point has heard at least one flight. ## The 920-Second Offset Filter Back to our goal: somewhere heard one aircraft, then another 920 seconds later. For each point, we check this time offset—whether another aircraft's delayed CPA time falls within the window 920 seconds later. When we say they are 920 seconds apart, it matters where we measure from. There's a margin of error of plus or minus 5 seconds here, so the 920-second offset window allows some flexibility. So how many points pass this filter? Around 35,000. And that's just in this small test area, which is less than 0.05% of the area of the contiguous United States that we need to search. By the way, I don't think people understand how large the United States is. If you divided the contiguous United States into football fields and placed them end to end, they could stretch around the Earth and to the Moon and back nearly 200 times. That's the area we need to search. So we need to do better—we need to learn something about doves. ## Reading Speed and Distance from Doppler Curves Jokes aside, this is really cool. A Doppler curve gives us a lot of additional information, and you already understand this intuitively. Listen to two clips, and you can already hear that the aircraft in clip 2 is flying faster, because the frequency drops by a larger amount. Assuming straight-line motion at a constant speed, only two factors determine the shape of this Doppler curve: - **Speed** determines how much the frequency drops - **CPA distance** determines how quickly the drop happens So we can fit a curve to the Doppler curve in the audio and determine how fast the aircraft is flying and how far it is from the listener. The filter then becomes: you heard an aircraft flying at a particular speed and a particular CPA distance, and 920 seconds later, you heard another aircraft flying at a particular speed and a particular CPA distance. The measurements: we get a good fit here at 100 m/s. I can't be certain this aircraft was flying at exactly 100, but we can be fairly confident it wasn't 70. Similarly, the time offset still has a potential measurement error of a few seconds. And since we now know the slant range, we don't need to lay out the entire grid from 0 to 10 kilometers. There are about 73,000 candidate points for the first event. ## The Airport Problem and "Silence Is Also a Signal" But if I zoom in on any of these clusters, what do you notice? Airports. Most of those 73,000 points are simply near airports. This makes sense, because that's where the planes are, and more air traffic means a higher chance of happening to encounter two flights that meet the conditions. But the signal in our video includes both what we hear and what we don't hear. There are no Doppler curves between these two—that's nearly 15 minutes without hearing an aircraft. You wouldn't get that just outside a major airport. This is part of what makes this method so useful: if you're near an airport, you'll hear lots of flights and get lots of signals; if you're elsewhere, the silence itself is a signal. So it works anywhere along that spectrum. I apply a **silence filter** to all 73,000 points, excluding any points where a flight would have been audible between the two events, using a fairly conservative threshold. This reduces 70,000 points to 15,000. ## The Final Layer: Modeling Real Flight Paths Even with all these constraints, 15,000 points still meet every condition. Usually, at this stage of a search, I'd simply start looking at some points and encode the reasons they don't work as another filter. But the problem is that we only have this image, and we've already extracted almost everything useful from the spectrogram—speed, slant range, time offset, and even the absence of aircraft between the two planes. Remember how I said this curve doesn't quite match a perfect Doppler curve? The reason they don't perfectly match ideal Doppler curves is that aircraft don't fly in steady, perfectly straight lines. They change heading, speed up, and slow down. All of that information is encoded in this Doppler curve. But to extract it, we need to go one level deeper. You can't determine the exact flight geometry directly from the spectrogram. It could be a change in speed, heading, altitude, or any combination of these. So instead of trying to reconstruct what kind of flight geometry would produce this result, we can **model the actual flight geometry of each flight and calculate exactly what its Doppler curve should look like**, then do this for every candidate point along every flight path to find the match. So how do we model a Doppler curve? A Doppler curve is a function of the aircraft's **radial velocity** relative to the observer—how fast the aircraft is moving toward or away from the observer. - If you're directly in front of the aircraft and its speed is 90, its radial velocity is 90 - If you're directly to the side and its speed is 90, its radial velocity is essentially zero, with no Doppler effect - Anywhere in between, you can see the red line representing the radial component of the aircraft's velocity What we want to know is what frequency the observer hears over time. So starting at t=0, or CPA—which in this particular example from the flight data is 71230 (the actual time in the recording is unknown; this is just an illustrative example)—we look at the flight data at 71230 and obtain the aircraft's radial velocity. In this example, because we're at the CPA, the radial component is zero. Source: https://youtu.be/UytYrhYLOKI