It seems like we see new airline surveys and lists every other week. Which airline is best? Which has the highest quality? Which treats their customers best? 
The media may love to talk about these things, but you know what? They really aren’t worth much at all in my eyes.
Let’s take a look at the recently released Zagat Survey (.pdf) to illustrate my point. We’ll just focus on the domestic airline results for now, since that’s what’s been getting the most press. First, what do we know about the people who took the survey? Not much. From the survey itself, we know they talked to 7,498 frequent fliers and travel professionals (how they define a frequent flier or travel professional, I don’t know). Of those, 40% are women and 60% men. Only 8% are in their 20s, 22% in their 30s, 23% in their 40s, 26% in their 50s, and 21% age 60 or higher.
That’s all we know. I was happy when the company’s PR firm contacted me trying to sell the story, because I figured I could get some more info on how they actually came to these conclusions. Sadly they refused to provide any info above and beyond what’s already published in the survey. Without this information, I find the survey to be mostly unhelpful. Here’s why.
- Let’s look at the top of the list, since that’s where most of the attention is focused. In the Premium Cabin, Virgin America was #1 and they were tied for #2 in Coach. Whoa, that immediately throws up a red flag. The airline started in August and only recently added its fifth city. How many of those 7,498 could have even had the opportunity to fly Virgin America? We’ll never know since Zagat won’t say, but it can’t be that many. And if it is that many, then this is hardly a random sample. Hmmm.
- In Coach, Midwest Airlines was the big winner. These guys have historically finished at the top of the pack in most surveys, but the airline really has multiple personalities. There’s the traditional (called “Signature”) Midwest experience which has wide leather seats (four across instead of the usual five). That is what generally gets rave reviews. But then there is the ever-growing “Saver” service that the airline provides which has the usual five across seating. It’s now going to be the largest cabin on all planes with Signature service being only several rows at the front of the plane. So in a way, the traditional Midwest model should be considered Premium now. How many of those taking the survey were in Saver and how many were in the original Signature seat? It’s a very different product that should be rated differently, but it isn’t.
- And how about Frontier? They get high marks for comfort, but their comfort is changing. The airline will be adding rows of seats to its planes, trimming the seat pitch and probably ending up with a less comfortable product. Even though this was announced long ago, it probably isn’t reflected in these rankings. How many people experienced the new tighter pitch? We don’t know, but it doesn’t really matter. Comfort will be going down.
- Then there’s the “food” rating. What exactly do people use to rate their impression of food? Midwest got highest marks but you have to pay for it on them. Continental came in a distant fourth, but it’s included for free when you fly them. Some travelers may find that the cost is more important than the actual taste, but a survey won’t give you that information.
- Let’s also look at the opposite end of the spectrum. US Airways didn’t fare well at all in this survey, and that’s to be expected. They’ve had some well-publicized operational issues this year, but again we’re talking about a tale of two airlines here. The old US Airways passengers (primarily east coast) were used to a higher level of service that the new US Airways is not trying to maintain. Those people are unhappy with a lot of the changes that they’re seeing so they’re probably going to rank the airline much worse. Out in the west, I would bet that the US Airways passengers are happier (or less unhappy) because they’re used to the America West standard of service which is being rolled out throughout the operation. They’re probably getting closer to what they’d expect so ratings should be higher. Of course, we can’t get geographic breakdowns from Zagat so I can’t prove this at all.
So, as you can tell, airline ratings boil something down to a nice media snippet, but they don’t tell enough of the story to be very helpful. While the media will continue to eat up surveys and lists like this, we should all know better than to trust a single one of them. Every person has very different views on what makes a good airline, and only more in-depth personal research can determine what is right for you.
