It’s so common to come across an article trying to arbitrarily rank airlines or airports one way or another that I generally have little trouble ignoring them all. But when I saw Airways cite a study saying that Los Angeles International (LAX) was the least reliable in the US, I couldn’t help but let curiosity get the best of me. How is that possible? This is just more proof that you can make numbers say anything you want. This isn’t helpful for travelers at all. Instead, it’s just clickbait. (And yes, I know I’m helping fuel that fire just by writing this, which is unfortunate.)
A website called Finder.com had someone pull up the flight status information that is publicly reported through the Bureau of Transportation Statistics. The article doesn’t make it very clear what data is being used, and I can’t get it to match exactly in my initial search. (Frankly, it’s not worth wasting time on a deeper dive.) But I’m guessing that this is likely using something similar to the arrivals at each airport by all reporting carriers for full year 2017. Regardless, the “analysis” itself was pretty simple. They just added up the number of delays of 15 minutes or more by airport and divided by the total. The winner? Well, LAX was responsible for 8.03 percent of delays with exactly 52,056 reported. So it’s clearly the least reliable since it had the most delays… right?
There are so many problems with this it’s hard to know where to start. Of course, the first issue is that this only looks at the data that’s reported. That means it’s for only the biggest US-based airlines on domestic flights, so it’s far from presenting a full picture. (Smaller US airlines started reporting in January of this year, but that’s probably not in here, or at best only for part of it.)
Then there’s the bigger issue at hand that the analysis doesn’t look at the denominator.
They say LAX had 52,056 delays. My look at DOT data shows that there were closer to 54,500 delays during that time period, but that’s close enough. My data also shows during 2017 that there were 214,312 arrivals on reporting carriers giving the airport a 74.5 percent on time rate. That’s not great on-time performance, and I’ll talk about that further later, but it’s still misleading to say LAX is the least reliable airport.
Size Skews Everything
You probably just assume LaGuardia is going to be at the top of the list, right? Well it’s not even on this list, and there’s a simple reason why… it’s just not that big. According to the data I’m using, LaGuardia had just over 26,000 delays for 2017. That must mean it’s twice as reliable as LAX, obviously. Though the reality is that LaGuardia had only 93,334 arrivals recorded, and that means it had worse on-time performance, 72 percent, than LAX. Neither are good, but you get the point. You need to know the denominator to really be able to judge someone’s chances for encountering a delay when they fly.
Another example is this gem: “When choosing among airlines, note than Allegiant Airlines accounted for only 0.10% of all delays of 15 minutes or more at minor airports, whereas Southwest Airlines accounted for 25% of such delays.” First, yes, Southwest will have the most delays in minor airports, because a lot of smaller regionals that fly for the legacy airlines didn’t start reporting until 2018. (I don’t know the cut-off for considering an airport “minor,” but San Diego is considered minor, so it’s not that small.) Allegiant didn’t start reporting data until this year either (this part must have used some 2018 data, I guess), so the data is automatically going put that airline in a favorable light.
But let’s say Allegiant’s data was fully reported just for argument’s sake. It’s going to look better than Southwest no matter what, because it’s smaller. Southwest flew 1,347,893 flights last year. Allegiant flew 93,061. Let’s say Southwest pulled off the impossible and ran on time 93 percent of the time. If that happened, Allegiant could run every single flight late and still look better than Southwest in this so-called analysis.
Context Should Matter
The other issue here is that the data has no context. In Los Angeles in 2017, the first five months of the year were pretty terrible. Once the whole Delta terminal swap completed in May, however, on-time performance rates jumped up for nearly everyone. Take a look:
It’s a tale of two very different airports once the switch occurred. I mean, if you just took the number of delays from the last six months of the year and doubled them, you’d be shy of 40,000 on the year. That’s a far cry from the actual 50,000+ delays that allows Finder to call LAX the least reliable even though it’s not today’s reality. Context makes a huge difference.
So is this entirely garbage? Pretty much. There are interesting ways to slice and dice this data, but Finder apparently chose to just take the easy way out.