The email last week came as a surprise. I was invited along with three others to chat with Southwest CEO Bob Jordan about the airline’s failures during the holidays. Considering Southwest had been very quiet throughout the meltdown, I was pleased to finally — emphasis on “finally” — see the airline putting itself out there, assuming more small calls like this were being arranged with others. What I learned is that Southwest isn’t willing to say it knows exactly what went wrong just yet, but it has a basic idea and will be working with consultants to reconstruct its failures quickly.
Bob did not mince words in understanding the impact of this event. He repeatedly talked about how the airline “messed up” and how it couldn’t do that again. That sounds easy to say, but not every company would admit it.
What’s harder to say is exactly what happened and how the airline could make sure it wouldn’t happen again. Bob spoke for 20 minutes and then the four of us were each allowed to ask one question. I asked Bob if he’d comment on the SWAPA union’s detailed dive into what it says went wrong. He wouldn’t, saying it was too early to know everything. He started to say he was sure that some of it was right and some was not, but he stopped himself saying he didn’t even know that. The airline wants to do a full autopsy before commenting, but it is finally at least speaking about what it does know.
SWAPA’s piece was detailed and full of conclusions immediately after the event. It had plenty of grenades to throw at management… which is one of the reasons I tend to discount union communications. There’s always some kind of negotiating motive behind them. That’s not to say that what comes out is always wrong, but it’s important to always keep the source in mind. Without question, comparing what the union had to say and what Bob had to say gave me a better sense of what happened.
Though Southwest will admit to having some idea of the problem, it believes that there is more to uncover. The airline has contracted with Oliver Wyman — where, it should be noted, COO Andrew Watterson worked for a dozen years — to do a full reconstruction and review of the mess. Though I have a general disdain for using consultants, this is actually a use I support. It’s a specific, limited task that could benefit from an outside perspective. Bob said this will not take months and will happen quickly. I really hope that’s the case, because the airline seems to want to reserve full judgment until this is done, and that just can’t keep waiting forever.
Fortunately, Bob didn’t just completely defer to this review before commenting on what happened. Here’s what everyone seems to agree on so far.
The Storm was Bad
The storm that rolled throgh on December 22 was very cold, very windy, and problematic in many ways. According to Bob, “we saw things that we just have never seen before with the temperatures. You had jetways that froze, you had deicing fluid that froze. You had aircraft that had ice on the engines in the morning.”
One of the questions I’ve had along with many others is why this was so much worse for Southwest than it was for, say, United which has hubs in Chicago and Denver. There has been talk about network design and all that, but one of Bob’s comments about the storm grabbed my attention. He said, “we had crosswinds that shut us down in Chicago for awhile.”
A look a historical wind data for Chicago shows that starting just after noon, the winds started to ramp up with gusts consistently in the mid-30 kt range. The wind was coming out of the WNW for most of this time, which is just slightly north of due west.
At O’Hare, where United has its hub, the six primary runways are all now east-west oriented which meant there was very little crosswind component. Midway’s runways that would have been in use run NW and depending upon the actual wind degree, would have faced a much greater impact. Little bits of bad luck like that can add up quickly to snarl the operation for one airline and not another.
Overwhelming the NOC
Southwest’s network operations control (NOC) is the operational heart of the airline. As the mess progressed, this drove what Bob called “an historic level of work into the NOC” to get the aircraft in place. And that turned into another “historic” level of work on crew re-scheduling.
Bob made it clear that the technology did not fail. Instead, it just wasn’t up to the task to be able to deal with the crushing amount of work that it needed to do. I liken it to shoveling in a driving snowstorm where just as you clear the snow, another layer falls on top, creating more work and you never get fully caught up. (Keep in mind I live in Southern California, so my metaphor may be woefully inaccurate.)
Cancellation upon cancellation happened close-in, and the tool that SWAPA referred to as SkySolver in its note — now apparently called GE Crew Optimization according to Bob — is meant to be able to run solutions to re-crew future flights. The key word here is “future,” because if there are problems in the past that haven’t been solved, Crew Optimization can’t handle it. And Southwest had many close-in cancels that Crew Optimization couldn’t touch.
Because of this, Southwest needed to bring in an army of people to manually solve all those previous problems, which could then be fed into Crew Optimization to create future assignments. But… there was no manual workforce available to actually deal with this, so Southwest quickly trained up over 100 employees in other functions to be able to work the issues and get good data into Crew Optimization so it could do its job. The airline then cleared out two-thirds of its schedule so that it could get all the pieces in place over a three day period, ready to restart on Friday, December 30.
If this sounds crazy, it is. There needs to be software to handle past problems, and… there is. Bob says that there is a release of Crew Optimization that the airline will start testing this week that will actually solve past problems. I followed up with Southwest to understand whether this is an actual new release or it’s just new to Southwest, and a spokesperson told me “we are working with GE on an update to the software that is tailored to our needs. It wasn’t previously available….”
Begging Customers to Return
Once the operation was up and running again, Southwest turned to figuring out how to deal with the hordes of rightfully-angry customers. As we all know by now, the airline put out vague guidelines that it would process refunds quickly, offer reimbursements, and then it pushed out a 25,000 point bonus to everyone stuck in the mess.
The communications were chaotic at best, and if I’d been given a second question I would have asked Bob if he was pleased with how communications were handled and if not, what he would have changed. Unfortunately, I didn’t have that opportunity. What I do know is that now that this is over, at least customer compensation has been humming along quickly.
Bob says they’ve processed 93 percent of refunds, turning them around within 3 days,. The remaining refunds are those which have come in during the previous three days, so it’s a quick turnaround. Reimbursements for expenses are tougher because they involve reviewing receipts through a more manual process. But Bob says Southwest is now processing 30,000 of those requests a day.
The 25,000 points were deposited almost immediately, and while Bob couldn’t give specifics with earnings coming up soon, he did say that many of the points handed out have already been redeemed, so people are coming back. Of course, this is nearly free travel we’re talking about, so it’s not a surprise. But for Southwest, it just wants to get people back on board any way it can.
The airline has also returned all but one percent of the mountain of bags that were stuck during the event, and what remains are the really hard ones with no tag or no address.
In other words, it sounds like Southwest is doing what it can on the customer side now that the deed is done, but how does it make sure this whole trainwreck doesn’t happen again? On the one hand, Bob really wants to wait for that Oliver Wyman report to come out, but on the other, he knows there are things that don’t need to wait.
What Southwest is Putting in Place Today
Here is what Bob outlined as already being part of the plan to make sure this doesn’t happen again. Until the Oliver Wyman report is done, consider this a partial list.
- Improving the internal “advanced warning dashboard” to include more metrics like number of open crew lines so they can get a better sense of when something bad is happening.
- Keep this new temporary workforce they built to manually process crew schedule changes. They activated 25 of them during the FA A NOTAM outage but they didn’t need them. At least they’re more ready to deal with disruption considering the seemingly-inadequate system they have in place.
- Moving forward on implementing the new release of Crew Optimization that will allow the system to process past problems
- Looking at every process top to bottom, for example, found they didn’t have enough engine covers in the outstations, so they’ll fix that.
- Doing a full deicing review, because it was bad. Denver they use open baskets because it gives better visibility but when it’s cold, that isn’t worth it since frostbite concerns require much more frequent breaks and reduce productivity.
- The Board of Directors has set up an Operations Review Committee, which seems like something it should have done after one of the many previous failures. I don’t see how this is going to help do anything except cover the board’s asses.
Some things are harder to fix than others. For example, we heard all about how Southwest had to manually call crews to get them reassigned. But Bob explained, “we have automatic notifcation in place [but] it takes a contractual change, I believe to require acknolwedgement.” I followed up and asked for clarification on exactly what contract is the issue, and a spokesperson explained “the collective bargaining agreements for Pilots and Flight Attendants contain scheduling rules that, if altered, would require negotiation with those respective unions.”
It is somewhat frustrating to not have more answers, but at least the initial steps seem to make sense based on what we publicly know so far. And regardless of what happens in the Oliver Wyman report, Bob said, “I can’t imagine we don’t boost our [technology] investment based on what we find.” He spent time defending what Southwest spends on tech today, but it sounds like he has an appetite for more.
Could this result in bigger changes around the airline? It’s possible, but I’m guessing probably not. In response to a question about whether a network rework should be considered, Bob left us with this.
I don’t think it’s the network. But there could be things about how we flow crews or how we assign crews or how we schedule trips… you know, those kinds of things.
Just how much Southwest changes because of this remains to be seen.
31 comments on ““We Really Messed Up Here” – Speaking With CEO Bob Jordan about Southwest’s Meltdown”
Who were the other three invitees?
Miss – I was apparently in the “influencers” group. There was Henry Harteveldt from Atmosphere Research, Scott McCartney who retired from WSJ, and someone named Kathleen Bangs who I don’t know but apparently is a former pilot who does a lot of tv appearances?
Thanks for the transparency on the “influencer” role. Also noted that you have NOT awarded Southwest a well-earned “Cranky Jackass Award” not just for losing operational control, not just for the complete communication failure during the crisis, and not even for damaging the ONE thing they hold sacrosanct: their BRAND.
But you have a special relationship with SWA that goes back to sitting in Herb’s chair. As I recollect, you also won a contest on aircraft routing they sponsored. And then your epic California “fly everywhere” gig that they ensured ran smoothly.
So I (seriously) understand that you are giving them somewhat of a pass in exchange for past favors and continued access. Given Southwest’s “culture”, I think it’s safe to say if you run anything truly damning on them, then game over. Just ask Mr. Neeleman about that. Plus, it’s your blog and you can write (or not write) anything you darn well please. And I’m grateful for this venue to exchange ideas.
Miss – Though I would normally just ignore this comment, for some reason I feel compelled to respond to all your back-handed compliments. I take great issue with your comment, as if anything has ever stopped me from saying exactly what I think about any issue in order to protect a relationship.
If a company decides to stop talking to me or giving me access because I say something true, then that’s their problem, not mine. Sure, that has happened over the 16+ year history of this site, and it’s usually to the detriment of that airline since in a vacuum where the airline doesn’t provide any info, it usually ends up being a more negative story about them based on the sources that are more widely available. Some comms professionals realize that and may actually give more access after seeing a negative story so they can help me understand their side in more detail.
Some go the other direction. I can’t worry about that. I just write my opinion and do my best to ensure that I’m not misrepresenting the facts that are known.
For someone to drag my name through the mud just because they’ve decided something is worthy of a Cranky Jackass Award even though I don’t (at least not yet) is a pretty dirty thing to do, and I don’t appreciate it.
My compliments were sincere, Mr. Snyder. I have a lot of respect for your knowledge, particularly your worldwide knowledge of airlines. I was not trying to demean you or drag your name through the mud. I’m grateful for the time and effort you devote to your blog. And I’m also grateful for the opportunity to comment here.
Despite that, I have inadvertently upset you. No one likes that. I don’t. You don’t. The other readers don’t. Let’s agree then, that I should refrain from future participation. That path probably best serves everyone’s best interests.
I saw the email from Bob Jordan to customers last night, and I’m glad I wasn’t the only one who noticed that Mr. Watterson’s former employer is the consulting firm that WN brought in to do the assessment. Not necessarily a bad thing, but worth noting.
I suspect that bringing in outside consultants isn’t just for a fresh/different/outside perspective, but also for reasons of internal politics, given the finger pointing that happens in organizations after such a large and public disaster. The consultants’ report/recommendations will be biased by the information they receive, the hope to sell in additional consulting work after the initial project, and a desire to not bite the hand that feeds them (i.e., don’t expect the consultants to be overly critical of the execs who hired the consultants).
However, both inside and outside the organization, the reports & recommendations from the consultants will be PERCEIVED as a more independent and less biased, and that’s what matters as much as anything in terms of getting buy-in and actually getting the fixes implemented. Finally, Mr. Watterson did consulting for long enough that he likely did some projects that are at least somewhat similar to this one. As such, he should know how to keep the consultants, many of whom he likely worked with and has presumably kept up with, (mostly) honest.
About two weeks before the meltdown occurred there were reports of FAA related issues with WN. That seems all but forgotten now but it mentioned maintenance lapses. I hope that is resolved as part of this review.
WN announced they will be building a maintenance facility at BWI – their first on the east coast – I hope this comes to fruition and that they also center more assets around less weather influenced airports like MDW and DEN.
It also wouldn’t surprise me if they are actively trying to poach a senior level operations exec from another airline as we speak.
I believe they have begun construction of the BWI structure. And, technically, you are correct that it’s the first one they have built on the east coast. However, they also have maintenance hangar facilities in both ATL and MCO, courtesy of the AirTran acquisition.
Having interline agreements is the most basic of basic redundancies for an airline of WN’s size. They have the most powerful, connected passenger service system in the world, USE IT!
As someone who works with glycol-based solutions quite often, I am a bit surprised that they had issues with the deicing solution freezing. You really need sustained -15-degree temperatures if its undiluted. This should not be out of the operational range for Denver or Chicago, and while it was cold it wasn’t a record setting sustained cold pattern (a 10 year event? Maybe?). Denver was in the 60s prior the real 36-hour cold snap.
Denver was a fiasco! 7-9 rampers went to the hospital from frostbite. SWA fails to issue proper protective clothing. In addition, they uses open bucket de-ice trucks which none of the other airlines use in Denver. Before the storm hit, they left ground equipment exposed and unprotected. In addition, they cold soaked the aircraft instead of running the APUs like United did. They were understaffed before the storm, failing to unload the baggage compartments the night before of the terminating aircraft putting them behind to start the day. Over 20% of the employees are new and this is their first winter. When the staffing problem became apparent, the VP of ground ops sent out a threatening email to terminate any absent employee without a physical doctor’s visit and note (illegal under Colorado law), creating large resignation of ground personnel. Followed by an offer to extend bonus pay in an effort to keep the remaining employees… in the end, hubris is the trigger of this disaster. Hope SWA can figure it out, but so far no accountability or responsibility for anything that went wrong…
What is “cold soaking” of the aircraft?
Cold soaking is putting a plane (or any other vehicle) in a cold environment, turning it off, and not providing any other way to keep it warm.
Hi Nick, In my almost 10 years with Continental Airlines, at DEN, (Stapleton Airport) we always plugged the airplanes into ground power overnight. The ground power must provide the electric heat for the airplanes, sitting there at the gate. Right ? The APU was not on. Sometimes the APU would provide the heat, or air conditioning, different system, than ground power. Ground power and APU are two very different systems, how did each provide heat. ? Peter Richards Boulder, CO.
Hi Peter, I don’t know the specific ways to avoid a cold soak. I just know of it because it is one of the things manufacturers have to do to have their planes certified.
I’d guess that airplanes, with perhaps the exception of the 787, don’t have electric heaters onboard since engines produce a significant amount of excess heat. (Excluding things like coffee makers and ovens.) Sent from my computer that moonlights as a phone.
“you had deicing fluid that froze”. I was very surprised at this comment as well. I checked the lowest temperature in Minneapolis and Chicago. MSP had -12F on the morning of 12/23/22. MDW had -9F on the morning of 12/23/22. All previous and subsequent days were warmer than that. The above temperatures are not even close to record lows for this time of the year or lows that are normally experienced by northern airports many mornings during the winter. I know deicing fluid is similar to automotive antifreeze. Automotive antifreeze can be mixed to remain in a fluid state down to -30F. I am guessing that deicing fluid mixtures are formulated to meet forecast low temperatures and not below to save money. If that is the case, it is very shortsighted to try to save money that way when temperatures can and often do fall below forecast lows during the winter.
And what did our elected representatives in Washington do?
They sent a letter.
Wow, thanks for representing the public.
Southwest has done a herculean job of processing refunds and working down the reimbursement requests and they should be commended for the speed at which they are working to take care of their customers. It is clear a bunch of people – probably from headquarters – have been reassigned to clean up duty.
It is certain that people are using their mileage credits quickly so WN’s revenue will be depressed for several months but people seem to be willing to give them another chance.
LUV has said that the whole mess will cost them $725-825 million which makes this by far the most expensive single airline non-labor operational crisis in US airline history but LUV is better able to absorb it than other airlines – but it will leave a mark.
The real indictment is that they could have expanded their automation to have prevented this from happening but didn’t move fast enough. The good news is that they will spend the money now and will be super cautious in allowing events to deteriorate until they are fully confident they can withstand a major event. And, yes, there is basic winterization that WN should have done that other airlines did which they simply have to spend the money to do. Southwest has significant operations in the northern tier of the US now. And alot of the improved processes and automation will help during summer weather problems as well.
Southwest has adapted well to alot of crises in the past and they will do the same here -but at a cost that didn’t need to happen.
Something I’ve noticed the past few times I’ve been on the Costco app is one of their currently bestselling items is a $500 SWA gift card for $450. Perhaps it’s a silly anecdotal data point that the public is mostly unfazed by the meltdown.
I still don’t see any good explanation as to why Southwest’s holiday issues were so much worse than everyone else’s. I understand it snowballed… And the combination screw them. But other carriers had to deal with cold and snow and labor issues also.
And “Making up for it” by giving out 25,000 points is it going to have the effect they hope it will. In the first place, with the let the points fit the demand model of that the airlines use now, the average person has no idea what that will buy. In reality, 25,000 points won’t actually get you a flight anywhere on most of their system. Most of their flights are more than that.
The public will want to know what they are going to do so this doesn’t happen again, and they are nowhere near providing that information.
Southwest has lived on the reputation that we are a happy happy fun fun airline to fly for years when the reality is different. In many ways, they are a full fare airline that you don’t get as much on as you do with the other legacies. In other words, their cost structure is similar to the legacies, while they are trying to sell a product that is closer to the low cost guys. It’s Beth them in the ass this time.
End it will be a long time before anyone thinks flying on Southwest is fun again.
“In reality, 25,000 points won’t actually get you a flight anywhere on most of their system. Most of their flights are more than that.”
Really? I am pulling up random routes for travel in March and April…. depending on the dates and times…. 25,000 point on Southwest can get you a free roundtrip between:
Oakland and Kahului
Los Angeles and Honolulu
Baltimore and Orlando, Tampa and Fort Lauderdale
Seattle/Tacoma and Chicago
Portland, OR and Dallas
San Diego and Austin
Denver and Orlando
Etc… etc… etc…
The list can go on and on…. 25,000 points on Southwest CAN get you a roundtrip ticket on many Southwest routes.
2 main reasons it was much worse for Southwest –
1. Think about how many more domestic travelers they carried than the other OAs. Most leisure travelers are cost conscious and were flying Southwest Airlines – would luv to see if someone could pull data points on numbers of Customers they were serving v the OAs. More Customers = more impacted.
Also, 2. Southwest Airlines runs a Point to Point system v Hub and Spoke like the OAs (not sure I saw that mentioned in the article) but that had a significant impact on the operation.
Not sure if that helps give a little more light on why this affected them over OAs.
I find these Southwest apologist lines tiring.
1. The belief that “Southwest carries more domestic passengers” is a byproduct of the other airlines choosing to use regional carriers, so the available seat miles for the regional carriers do not show up under the major they were flown for, although from a customer perspective it doesn’t matter if American Airlines or Envoy Air lets you down, its still American Airlines’s fault. For December 2019 AA and Envoy flew 161.1 billion seat miles, Southwest flew 152 billion seat miles. Yes, some of those AA (and some of the Envoy) seat miles are international. I briefly tried to find the ASM details broken out by domestic versus international, but they weren’t immediately handy. So yes, if you’re being pedantic on operating carrier Southwest carries more domestic passengers than any other airline is technically true, but its a hand wavy statistic. I’m really curious if there is a quick way to get that information based on marketing carrier. In any case, even if by marketing carrier Southwest is larger than its next competitor, it isn’t significantly larger. So it isn’t as if any of the other carriers failed as spectacularly as Southwest did.
2. Re: running “Point to point vs hub system” This is one of those things that translates into “We’re not willing to design our network for resiliency, because it might cost more.” Eh, thats a poor excuse.
Maybe I’m not understanding the P2P v Hub & Spoke correctly. But my understanding is the P2P helps swa get quicker turnarounds to make more flights available enabling them to have a more profitable business model in the way they’re handling it when there aren’t operational challenges (hence they have the most profitable history of any Airline in their 51 years of rich history). Also w hub and spoke models, OAs can choose to cancel a flight from LAX and their crew is still available to take another flight. Whereas if an airline does it w P2P, crews aren’t where they’re supposed to be.
I’m pretty sure w the epic amount of revenue loss they’ll be making changes to things that impact their business w keeping the P2P in tact for now while also trying to make things monetarily right for their customers; just won’t happen over night unfortunately. Time will tell.
Perhaps an expert on this model could comment here to give clarity and speak to this piece on how it added to the “meltdown.” I’m not an expert. I’m just a fan of Southwest and for the 95% of times I’ve flown them for the past few decades, I’ve never had non-weather related cancelation issues and this was the one time. Nay sayers and those who want to weigh in on how they failed and say how ridiculous this airline is and wants to put their feet to the fire probably doesn’t have a long history of flying w them to offset their opinion. . .
And that’s ok! We are all entitled to our opinions.
Guess I am one of the 7%of refunds who will never see my refund for a flight I had to pay for due to this mess… Really would love to stick by southwest on this but losing faith quickly.
Why wouldn’t your flight be reimbursed? If you have your receipt and you flew within Dec 22-Jan 2. SWA is reimbursing flights, meals, accommodations, cancellation fees at resorts, hotels, personal items u had to buy for lost baggage… the list goes on. They’re going above and beyond to try to make it right. The only way you wouldn’t be reimbursed is if you somehow don’t have the receipt for your flight. But you can also send them a snippet of your bank statement w the purchase.
NYT interview, Bob Jordan, 13 Jan 2023
‘We Just Couldn’t Keep Up With the Volume’
Southwest Airlines’ chief executive, Bob Jordan, said frigid temperatures and mistakes by the company caused its meltdown around Christmas.
https://www.nytimes.com/2023/01/13/business/southwest-airlines-bob-jordan.html
Too bad Brett could not get a real interview with Mr. Jordan.
This NYT reporter did.
Mr. Jordan is not transparent, sounds like Dennis Muilenburg, now fired CEO of Boeing.
I would like to know who is on the Board of Southwest Airlines.
I cannot find it.
Safari tells me the website is impersonating ‘southwestairlinesinvestorrelations.com’, and I cannot proceed.
I have been looking for the board, and have gotten this message for weeks.
Lol. That’s pretty horrible especially since it looks like they just failed to get their TLS certificate properly signed. That’s basic web server admin stuff.
I can tell you it’s safe to bypass that message. Just don’t give it any of your personal information.
(HTTPS everywhere is great.. until things aren’t setup right and it breaks.)
Pilots are moving forward with a strike vote. Apparently they are unimpressed with management’s response.
Did not someone warn them of this happening,Holl?;y Heagemen?
“I don’t think it’s the network. But there could be things about how we flow crews or how we assign crews or how we schedule trips… you know, those kinds of things.” (Bob Jordan)
How can their network not be a factor? A hub & spoke system, like the other majors run, isn’t perfect but provides backup planes, crews, staffing, and maintenance resources that Southwest’s point-to-point system just can’t support, particularly in far flung smaller markets. The point-to-point margins they normally run are just too risky during the holidays with elevated passenger volume, weather events, and staffing issues.