According to USAPA’s website, here’s what happened that day:

• When pushing back from the gate, the auxiliary power unit (a backup source of electrical power) and the Hot Battery Bus (a critical source of primary electrical power) both failed – in other words, the plane had no electrical power and no radio communications. None.
• After opening a window to verbally call down to the ground crew (the Captain’s only option), US Airways maintenance was able to restart the power unit, but offered no explanation as to why it failed or any reasonable assurance that it wouldn’t fail again.

So we have a couple issues here. US Airways did a thorough review to see what happened. According to the letter from the US Airways VP of Flight Ops that I’ve obtained, the airline tried to meet with the captain in person to discuss the review but she was unwilling. So they put the results in writing to her.

On the issue of the APU failing, no problem was ever found. The APU (auxiliary power unit) is like a mini-engine which provides power when the engines aren’t running. This is actually used to start the engines on the ground, but it’s not necessarily required for flight. If you’ve ever seen an engine started while you’re at the gate with a big generator nearby, then that’s probably a case where the engines needed an external start because the APU wasn’t working.

Back to this incident, the first entry in the logbook from the captain said “APU failed at gate, unable to restart.” According to US Airways, the mechanics did a test that found no fault with the APU and then proceeded to restart it at the gate without any trouble at all. The captain said she wouldn’t accept the aircraft unless the APU was fixed, but since the mechanics couldn’t actually find any problems that needed fixing, they deferred the issue as permitted by the maintenance program.

That was followed by a second entry from the captain saying:

After APU Auto Shutdown on gate with no external power connected, battery power lasted 3-4 minutes. Unable to restart APU or communicate on VHF [radio] #1.

So the APU failed and wouldn’t restart. US Airways pulled the Electronic Control Box (ECB), which the airline describes as the “brains of the APU.” The ECB records any abnormal events to help with maintenance. So the airline sent the ECB to Diehl Aerospace, which is described as “the repair subcontractor for Honeywell (the manufacturer of the APU).”

Upon review, no auto shutdown nor any failed attempt to restart the APU was recorded. Could the ECB have failed to record the incident? Unlikely. It recorded minor issues before and after the event so it was functioning properly. Diehl sent the ECB back saying no problems were found, and that ECB went back into service with no further issues. The APU apparently was working just fine.

But what about the radio not working? Was there an issue preventing that from functioning properly? Well, that assumes that the APU did shutdown. All we know for sure is that the APU didn’t automatically shut down. But as US Airways notes in the letter, “the ECB does not record a ‘commanded’ shutdown.” So maybe there was a commanded shutdown by someone on the aircraft? I have no clue because it’s not spelled out, but let’s assume that the APU was in fact shut down for one reason or another. What about those other issues that followed?

Apparently the airplane worked as designed. When on the ground, if the engines aren’t on, the APU isn’t on, and there is no ground power (basically, when the airplane is plugged in to a source at the airport), then the batteries will power many of the electronics. But there’s a catch.

To avoid completely draining the batteries while on the ground, the system will cut battery power when voltage drops below 23V for 16 seconds and the airplane is on the ground. When that happens, that VHF radio #1 won’t work. So that’s probably what happened here, and it worked as designed. Was the battery draining too quickly? Not according to US Airways. The airline says “the two main batteries were . . . tested by an outside source and have also been returned to the Company with no faults found.”

So the APU worked, the batteries worked, and the airplane in general seemed to work as advertised. The only thing I don’t see mentioned is anything about the “Hot Battery Bus” not working. The only mention of the hot battery bus in the letter is this:

. . . The [automatic battery shut off on the ground] will not disconnect the batteries when either one is discharged below 23V while in the air; they will continue to power the hot battery buses, DC BAT bus, DC ESSENTIAL bus, and the AC ESSENTIAL bus as long as possible.

So that seems to say that the bus (or buses), which provides continuous power to vital systems, was working. But no further mention is made beyond that. In short, US Airways says “this aircraft performed exactly as it was designed.”

Does that mean the captain fabricated what happened? I wasn’t there, so I don’t know. The only clue we have is from the US Airways letter which says “there is no discipline contemplated” after reviewing this incident. US Airways says it just wants to “put closure on the incident.”

It seems to me that if a pilot fabricated a maintenance problem, and it was proven conclusively, then that would be grounds for discipline of some sort, right? So since there is no disciplinary action here, maybe they’re chalking it up to a misunderstanding or confusion about the situation. Either way, it puts a serious hole in USAPA’s ongoing campaign to question the airline’s safety procedures.

[777 (not an A330) APU exhaust photo via Flickr user Robbie 1/CC 2.0]

Today, let’s start with the mammoth Delta TechOps maintenance facility. With 63 acres, this place now serves 150 airlines worldwide. My question was simple. How could an airline which outsources some maintenance to other countries create a growing, profitable maintenance business inside the US as well? The answer actually makes a lot of sense.

I first met with Tony Charaf, President of Delta TechOps. Tony told me how the Northwest merger had brought the airline the capability to work on Airbus aircraft, and that meant Delta could now service most of the world’s fleet. While Northwest had worked to outsource nearly all its maintenance, Delta has gone the other way and has built a strong business. But how does that work when it’s generally so much cheaper for other airlines to outsource to  countries with lower labor costs? This requires a deeper view of what is actually involved in maintenance. Delta has decided to build a highly-skilled technical workforce that focuses on the more complex and niche-market tasks.

I was taken through the maze that makes up the airline’s engine shop. Delta mechanics overhaul around 600 engines per year. I saw a very active shop working on all kinds of different engines from airlines around the world. For this kind of maintenance, Delta does nearly all of the work on its own fleet in-house. What doesn’t it do? Some engines are on such a small fleet that it makes little sense to do it themselves. For example, Delta doesn’t do the engines that power its small 767-400 fleet.

Why not just do the work on a small fleet as well? It’s incredibly expensive. And that’s one reason Delta’s in this business at all. Starting an engine shop requires a very serious amount of capital, a high barrier-to-entry that makes the business even more attractive for existing organizations. But even some engines don’t make sense.

On my tour, I was handed a small piece of metal to review. It looked like junk, but it alone was worth about $20,000. Like I said, a lot of money is required and a very skilled workforce is as well. Because of these things, the engine business actually has a much larger margin than the airframe business. It’s some of that airframe business that Delta does outsource for its own fleet.

Maybe I should back up and explain what an airframe business is. Airplanes are required to go through certain checks during their lives. You check for problems that might be lurking under the surface, like corrosion. The heavier checks can take a couple weeks, but these are not nearly as skilled as the engine work, so Tony explained that Delta outsources some of that to make room for additional high-margin engine work.

I asked why Tony wouldn’t just do it all in-house. Why not grow the business further? The airframe margins are apparently razor-thin and it’s too hard for Delta to compete with places that have lower labor costs. The lower barriers to entry mean it’s just not a great business for Delta.

But Delta does do some airframe work, and it focuses on specialized work there as well. For example, while I was there I saw a LAN 767 getting retro-fitted with winglets. Delta has so much experience doing that work that it can do it very quickly compared to others who haven’t done as many. And while the airplane is there, Delta can do regular airframe checks as well. You’ll also see Delta doing specialized work on things like the landing gear and auxiliary power units.

Delta even has some reciprocal agreements. Aeromexico, for instance, will do some airframe work on Delta’s fleet while Delta does the engine work for Aeromexico. It’s a strategy that has been working.

The result is, as mentioned, 150 customers around the world. And that’s just the beginning. Tony said that beyond Atlanta, Delta expects its maintenance facility in Minneapolis to get governmental approval to do work for other carriers by the end of the this year. Detroit will follow in 2011.

This strategy, now working, has taken some time to refine. The IT integration was critical and took up a great deal of time. Now that the dust has settled and the business is growing and profitable, things are humming along.

Tony explained that success can be described this way.

We have succeeded in this organization by taking our strategy on the second floor [the offices] and building buy-in on the first floor [the shop floor].

It sounds like management fluff, right? But it is true. And it’s also pretty easy to achieve when you have a growing, profitable business that pays good wages. Then everyone’s happy. So for now, things are working out quite nicely.

Real Estate Developer Rick Caruso Eyes Expansion Into Airports – BNET
The man behind The Grove development in LA is now eyeing airports to improve shops and dining.

using miles for wedding travel – AntiBride.com
If you’re having your wedding, it’s a great time to try and use miles.

FAA Slaps American with $787,500 Fine For Lax Maintenance – BNET
The FAA is trying to step things up when it comes to enforcement. That latest is an American fine.

Pittsburgh’s European Connection Will Cost the City $5 Million – BNET
Pittsburgh is paying through the nose to keep Delta’s struggling flight to Paris at the airport.

The Cranky Flier: Booking Beyond Price – Sparksheet
I guest posted on the Sparksheet about the difficulty of encouraging people to book beyond price. (Hint: distribution makes life hard)

My name is Jeff Pinneo–I’m the CEO at Horizon Air and a pretty frequent reader of your blog. My compliments to you on the good work you do ‘drilling down’ on many aspects of our business that your readers are interested in and want to know more about.

The subject of regional airline safety has certainly been one of those topics in the year following the tragic accident at Colgan, and I think you’ve done a really good job of helping folks take an objective look at the matter. Your post last week was a good example–in it you bring much needed perspective to the picture without minimizing the overriding importance of safety or of the need for the industry to do everything it can to further improve it’s already strong record. Regarding the question posed in the headline, I’m in general agreement with your conclusion–that being wholly owned by a major airline is not in itself a predictor of a higher level of safety. There are many independent regionals with excellent safety records and solid underlying programs. Having said that, I’ve observed our own evolution since the acquisition of Horizon by Alaska Air Group [AAG] in 1986 (I was at Alaska from 1981-1990 and have been at Horizon ever since), and I can attest to many positive influences and outcomes that have stemmed from our being wholly owned by AAG and a sister company to Alaska Airlines. It all starts with having one board of directors and one chairman (Bill Ayer) who are responsible for the whole enterprise and their obligations for ensuring a consistently safe and dependable experience across the brands. This structure, coupled with their strong personal conviction about the importance of safety, led both board and management to a ‘single standard of safety’ mindset and practices at Alaska and Horizon long before such things were legislated. As a result, both companies have moved virtually in parallel on safety programs from technology (e.g. introduction of heads-up-guidance system (HGS) low-vis technology in early ’90′s, Required Navigational Performance (RNP) and WAAS [Wide Area Augmentation System] program development, etc.) to audit and self-reporting programs such as ASAP [Aviation Safety Action Program), LOSA [Line Operations Safety Audit], FOQA [Flight Operational Quality Assurance] and IOSA [IATA Operational Safety Audit] certification. Our board formed a dedicated board safety committee a decade ago to focus on and reinforce the importance of all these safety improvements. It was the first committee of it’s kind and to this day one of the only, if not THE only, such committee of an airline board of directors.

As a further enhancement to safety oversight, the board in 2008 directed that an Alaska Air Group Vice President-Safety position–one that would be responsible for safety programs at both airlines and report directly to the AAG Chairman and the board safety committee–be established. Tom Nunn, most recently the CEO at Frontier’s Lynx subsidiary, was selected to fill that role late in 2008. Prior to that time, each company had individual safety programs and processes.

So while I agree that the ownership structure of a regional airline is not directly correlated to safety, I can say from our experience that we’ve been distinctly advantaged by our structure and relationship with Alaska Airlines over many years with respect to safety and many other matters. The fact is that many of the structural changes and investments in safety noted above emanated from having a common board and a single chairman who’ve been consistently committed to ensuring nothing less than the highest levels of safety at both operating companies, and to supporting their management teams efforts to that end.

I thought you’d be interested in this background as it relates to what is likely to be a matter of continued public interest in the months ahead. I’ve also attached a fact sheet on Horizon’s flight operations and safety programs that illustrates how our story differs substantially from the many broad-brush characterizations that have been applied–often inaccurately–to the regional airline sector. I’d be happy to discuss all of this in further detail if you wish–I can be reached at xxx-xxx-xxxx. Thanks for your time and interest in these matters.

Sincerely,

Jeff

Jeff Pinneo
President and CEO
Horizon Air

As I wrote back to Jeff, 10 years ago, Alaska Airlines was found to have serious maintenance issues after the accident of Alaska 261 shined a light on the airline’s practices. That likely negatively impacted Horizon back then, just as they are benefiting from their enhanced attention to safety now.

In short, I think Jeff offers a great perspective from inside a regional, and I thank him for sharing it with me and all of you.

United’s $3.8 million fine was actually more massive than the one levied on US Airways on a per incident basis because United’s was for a single violation on one airplane. What the heck did they do? At least one enterprising mechanic apparently decided to use a shop towel as an airplane part. This is even worse than it sounds.

In December 2007, a single 737 went into the shop for maintenance. Someone decided to use “two shop towels instead of required protective caps . . . to cover openings in the oil sump area.” That’s just unreal and frightening.

[After speaking to United spokesperson Megan McCarthy, I think this needs to be clarified. The caps are only supposed to be used during the maintenance procedure and then removed. So this was not meant to be a flying replacement. This does mean that two things went wrong.

But these two things together are far less frightening than what I originally thought.]

What’s more frightening is how they found out about it. The plane kept flying in this condition for more than 200 flights until April 28, 2008. On that day, the airplane left Denver and then promptly returned after shutting down an engine due to low oil pressure. That’s when maintenance found the problem.

Holy crap.

As soon as United management found out about it, they self-reported it to the feds, as they should, but there are still so many unanswered questions. Who did this? Was it on purpose as an act of revenge against the airline or was it simply a mistake? If it was a mistake, shouldn’t someone else have caught it somehow?

[Now that I have a clearer understanding of what happened, it seems highly unlikely that this was intentional. Maybe using towels instead of caps was intentional but that seems relatively minor. Leaving the towel in there? My guess is that someone screwed up. United says that it has made some procedural changes to make sure these things get caught, but they wouldn't go in to details.

What I don't understand is why they haven't broadcasted this more publicly, because I've heard a lot of people concerned about the incident.]

During an interview with Houston’s NewsRadio 740 yesterday, I was asked whether I thought additional governmental oversight was necessary due to some of these issues. I just can’t see how this would have been caught by more inspectors. They can’t be everywhere, all the time. I just hope United has hunted down the guy who did this and fired him (or them).

For US Airways, it was a different story. They were fined for a bunch of different things regarding maintenance lapses during the merger integration. Most significantly, they had a single Embraer 190 that flew 19 flights without inspecting the cargo door to make sure it wouldn’t open during flight. They also had a couple of A320s the flew some flights without being inspected for potential landing gear cracks. Both of these were required by Airworthiness Directives and that’s a big deal.

They also had a bunch of other problems here and there on a variety of aircraft. Read here for all the details. Just the simple fact there are enough violations here that I don’t even list them all shows that there could be some major systemic issue, right? It appears that way, but US Airways says it has been fixed.

As usual, US Airways was very quick to publicly respond about this problem. I’ll let them explain . . .

Today’s proposed penalty dates back to challenges we experienced during the integration of maintenance systems and processes on flights that occurred in 2008 and January 2009. Our team worked cooperatively with the FAA to investigate and correct any discrepancies to the FAA’s satisfaction.

Over the past nine months, we and the FAA have completed a formal review of our aircraft maintenance tracking systems as well as a comprehensive review of our maintenance program. This collaborative process included efforts to identify the issues, drill down to find the root cause and develop comprehensive fixes.

So this has been known for quite awhile, obviously, and clearly the FAA is satisfied that the problem is solved or they wouldn’t let them continue flying. I imagine that they will be kept on a very, very short leash for awhile. That’s a good thing.

But it does point to issues with mergers. Combining maintenance programs is no simple task. I hope the FAA is keeping that in mind as it watches the Delta/Northwest integration proceed.

Both of these are some very serious fines. That being said, I still wouldn’t hesitate to fly either of them today.

[Updated 10/15 @ 320p to clarify the United incident]