UNITE HERE Strikes Back: Tries to Cover Bad Analysis By Insulting Cranky, Earns the Jackass

If you decide to put out bad analysis, as UNITE HERE did last week regarding Lufthansa complaints, and someone calls you on it, what would Cranky Jackass Awardyou do? The smart answer would probably be to just let it go and stop calling attention to the work, especially since it has more holes than Swiss cheese. But fortunately for us, UNITE HERE has decided to go the opposite route.

The union is using one of oldest tricks in the book: going after my credibility to muddy the water. This is just dumb. They really shouldn’t want to bring more attention to a flawed report like this. Now I’ve just dug in deeper and found even more problems with it. While I was waffling before, now I’m not. UNITE HERE has truly earned the Cranky Jackass Award.

You can read the union’s entire response here (pdf) if you’d like, but I’ll pull out the most fun parts. Let’s start with the opening.

One of the things I appreciate about your site is you are very open about your relationship to companies in the airline industry. And just one month after Lufthansa gave you a free round-trip, business class ride on its A380 from San Francisco to Frankfurt, perhaps I should not be surprised at your dismissive response to my report.

Ah yes, the back-handed compliment. A time-honored tradition that’s used to cover bad work. If someone calls out real issues, just call his or her credibility into question but look completely pleasant while doing so. This takes the focus off your bad analysis and tries to shift the issue. (Sounds like the author may have a future in politics.) It’s true, I’m very open about these things, and I did just fly Lufthansa at the airline’s expense. That doesn’t mean I won’t gladly rip Lufthansa a new one if it’s deserved. The problem here for the union is that it’s not.

You can read the rest of the response yourself if you’re interested in more sugar-coated insults, but let’s focus on the weak defense of the report itself and break that down.

The Department of Transportation data in the report is real, and to my knowledge is the only reliable U.S. source of compiled complaint information on international
airlines. If the DOT is willing to use these numbers to “to determine the extent to which carriers are in compliance with federal aviation consumer protection regulations,” then they’re good enough for me. Even if I am just a research analyst at a union.

*sigh* The issue is not whether this is the only place to get complaint data or not but whether or not it’s statistically valid and can be used to explain a trend or not. In this case, the year-over-year change in complaints from 2009 to 2010 moved by roughly less than one-thousandth of one percent over total passengers carried by Lufthansa to and from the US (using my rough passenger estimate). Even the initial number itself is so tiny that it’s not significantly different from zero. So regardless of what the purpose of the complaint reports are in the eyes of the DOT, that doesn’t magically mean that we can consider each number valid for any kind of analysis.

You’re right, I could have used the raw numbers, but I sort of agree with you that the raw numbers themselves aren’t incredibly exciting on their own. They’re small
because, well, how many people actually go through the effort to submit their airline complaints to the U.S. government? (If you care about an answer, you can look at the DOT analysis for the new passenger rights rule, where the DOT uses the ratio that every 1 complaint submitted to the DOT represents about 61 complaints submitted to the foreign airlines.

Excellent. Let’s just forget about using raw numbers because they aren’t “exciting.” I see. So we’re not looking for statistical validity here. We’re looking for excitement. You can apply any ratio you want to these numbers, but that still doesn’t make the small change valid. And this ratio is just an estimate by the DOT anyway, so using it would make a statistically insignificant change even less valid, if that’s possible.

The result of that comparison was clear. Lufthansa complaints went up, Air France and British Airways complaints went down. Is the sample number of complaints
small? Yes. But if the increases were random, would Lufthansa have seen them in 7 out of 8 top categories from 2009 to 2010? If they were random, wouldn’t Air France and British Airways have seen more fluctuation too?

This is my favorite part. I hadn’t even touched the Air France and British Airways numbers in my initial post, so I should thank the union for giving me even more firepower to show how awful the analysis is. The result is far from “clear” as proposed.

When I spoke with the research analyst, he told me that he didn’t bother looking at the monthly complaint reports. He just looked at the year-end summary and called it a day. That makes the analysis even worse because it doesn’t look for outliers. And that’s exactly why BA’s numbers are so different. In 2009, BA saw 347 complaints while dropping to a mere 120 in 2010. That’s great improvement, right? Wrong.

A look at the monthly data shows that in October 2009, BA received an incredible 244 complaints for reservations/ticketing/boarding. Why? According to the Bureau of Transportation Statistics, “The increase in the number of October complaints is attributed primarily to British Airways’ erroneous offer of $40 fares between the United States and India.” That’s an outlier and can’t be used to judge overall performance for an entire year. Guess what happens if we just substitute a more typical monthly result that month? We see an increase in complaints year-over-year approaching 20 percent. Fun with numbers, right? (Not that this is a significant change either.)

I highly recommend reading the entire response. In particular, I like the union’s effort to call into question the safety of the engines on the A380. Enjoy.

(Visited 556 times, 1 visits today)

Get Posts via Email When They Go Live or in a Weekly Digest

Leave a Reply

23 Comments on "UNITE HERE Strikes Back: Tries to Cover Bad Analysis By Insulting Cranky, Earns the Jackass"

avatar
Sort by:   newest | oldest | most voted
Jim
Guest
If Gallup puts out a poll saying that X percent of people support candidate Y, they likely only polled about 1,000 people, or .0003% of the population. Yet the results are statistically valid, because they used a fair randomization process. Similarly, if you assume that the odds that a given passenger who has a problem will complain about it are the same for each airline and in each year, then the results are valid even with a small sample. Just wondering, does anyone know of a formula to calculate the minimum amount that would be statistically significant? There must be… Read more »
Bob K.
Guest
> Just wondering, does anyone know of a formula to calculate the minimum amount that would be statistically significant? There must be a way of doing that. I’ve long since forgotten most of my statistics, but it really comes down to two things: the size of the sample and how carefully you select it so that the sample is as “random” as possible and so that it reflects the general population (the main group you’re studying, in this case airline pax) as closely as possible. Ignoring sample size (not going to research all the tests needed here for that), the… Read more »
DAB
Guest
Specifically because the DOT analysis is self reporting, you can’t assume that a small number of reports extrapolates for any purpose at all (or anything under a very large number of reports for that matter, like at least 25% of the whole maybe). If the DOT had a reliable way to take a random sample from all passengers (which wouldn’t be all that hard to do if they cared to do it, but I don’t see why they would without someone passing a law telling them to…), then you have something that is statistically valid. Justification of random vs non-random… Read more »
andy
Member

Jim

There is simply no correlation between a Gallup (or similar) poll and a self generated complaint line. Any discussion of same is meaningless.

The Unite “researcher” needs some training in the use (and abuse) of stats – and if Unite really wants to hammer an airline then commission a statistically valid survey sample by Gallup (or similar)!

Keep on keeping on Cranky.

Bob K.
Guest
FIGHT FIGHT FIGHT FIGHT! Cranky, A couple of points… First off, you should seriously consider getting the kid who made that Cranky Jackass an internship… if not with you, then with another aviation company, and if not this summer than in the next few years. Gotta keep those kids interested in aviation! As a political organization (and let’s face it, many unions, especially airline unions, really do act as political organizations as they try to influence public opinion), the union has taken a page from the “best defense is a good offense” book… And failed. Guess they’re more used to… Read more »
Ron
Guest
“But if the increases were random, would Lufthansa have seen them in 7 out of 8 top categories from 2009 to 2010?” — Cranky, they do have a point here, albeit a very weak one. Run a binomial test on seeing an increase in complaints in 7 or more categories and you get p=0.035 (one-tailed), which is typically considered statistically significant. Now there are several reasons to not take this seriously: First of all, for a one-tailed test you have to start out with the assumption that Lufthansa is going to be bad rather than good, and there’s no a-priori… Read more »
David SF eastbay
Member

Well if they are coming back and attacking you, means you tailed it on what was wrong with what they were saying.

DesertGhost
Guest

A deserving winner.

scott.wintner
Member
The real question is: did the analyst give you and clue as to WHY they were waging this war in the first place?? That was the real curiosity pointed-out in your post. At the end of the day, no one cares what UNITE HERE thinks of airline service quality anyway… so even if their data analysis WAS valid (which it clearly is not), UNITE HERE pointing it out to the world is not likely to have any impact on the airline anyway. The whole thing, while annoying for you, CF, I am sure, is entirely moot. I say, don’t waste… Read more »
Don
Guest
I know that the TWU has already organized Allegiant flight attendants. Now they are also going after Virgin America, Jetblue, and SkyWest flight attendants. In the meantime the pilots at Jetblue have filed (again) with the NMB to have a vote if ALPA should represent them. I also know that US Airways pilots are going on 4 years with no contract with their union (at least that’s what the big truck with sign had. And AA is having tons of negotiation problems with their contract. Plus, I remember that Southwest flight attendants union (the TWU) had a bitter battle in… Read more »
Don
Guest

Please excuse all the grammatical errors. Between me typing and doing research for my findings I hit the “Leave comment” button before proof reading.

nostephenson
Member

I’m glad to hear the union is spending their dues dollars in such a resourceful way as analyzing these Lufthansa complaints. Great use of resources & time UNITE HERE!

And in case you can’t understand American humor, that was dripping with sarcasm.

I would recommend UNITE HERE workers demand the a partial refund in their dues because of this waste of money.

Doug Swalen
Guest

I’m not sure but I think these guys were the people who came out and handed fliers to all the spotters at the Lufthansa A380 inaugural landing at SFO. The fliers questioned the safety of the Trent 900 engine and by implication Lufthansa A380 air worthiness. We read them and said, “Wha?” Some of us even laughed. We didn’t get what the reason was and the flier seemed non-sequitorial. Now I think I may have a clue why those fliers appeared in our hands.

Ron
Guest

Speaking of A380 engines, was there a resolution to the Qantas problem? I see the jets are back at LAX, so I assume they can once again operate at maximum thrust?

wjboll
Member

UNITE’s response is hilarious. Almost as funny as Laura Glading and the APFA with their buttons and lanyards telling us their union. :-)

tjfulkerson
Member

just to be sure, this is the same union that wants to hold all people at bay whey they feel that it will effect them the most. they are almost a terrorist group. they don’t care about anyone but the idiot dues paying members. the higher ups live like kings on the back of the working man.

t.j.

Stephen Dutton
Guest

Oh dear, oh dear…LH (Lufthansa) use the derated version of the Rolls Royce Trent, the Trent 970 on their A380-841’s, Qantas use the Trent 972 Higher thrust variant,
There are currently 7 A380-841’s in the Lufthansa Fleet all with upgraded engines.

gobluetwo
Guest
A few observations (not even really analysis) about the data: 1) They pointed out that the complaints increased in 7 of the top 8 categories. 2011 YTD (through April), LH has complaints in 7 categories (out of 12 total), whereas AF has complaints in 10 categories and BA in 9 categories. Misleading. 2) 2011 YTD, AF and BA have 42 total complaints, whereas LH has 41 total complaints. LH compared to AF and BA, respectively, “leads” in 2 categories: Res/Tktg/Boarding (7-5-5) and Fares (2-1-0). 3) AF had 97 complaints in 2010, but are on pace for 126 in 2011. Will… Read more »