* Several institutional investors back airline to bounce back
* Confidence hit by staff holiday mess-up, union recognition
* Big investors say cost advantage won't disappear
* Upcoming union negotiations will be crucial
LONDON, Jan 10 (Reuters) - Several institutional investors are backing Ryanair to bounce back from a pilot rostering mess-up and a decision to recognize unions that jolted confidence in Europe's most profitable airline.
The mishandling of staff holidays caused thousands of flight cancellations, angered passengers and helped pilots to secure union recognition for the first time in Ryanair's 32-year history, sending the share price down 9 percent in one day in December.
The holiday rota errors were a rare serious slip-up by Chief Executive Michael O'Leary, whose formula of dirt cheap, no-frills travel has transformed the continent's airline sector and handed his shareholders big returns.
O'Leary had said hell would freeze over before he would welcome unions into Ryanair. The company now says dialog with organized labor will help the business.
Analysts say it will inflate its costs and could reduce the flexibility that helped it steal chunks of business from traditional airlines such as Lufthansa and Air France-KLM.
Top shareholder HSBC Global Asset Managers has cut a quarter of its Ryanair stake, according to a filing last month. HSBC declined to comment on the decision.
But other big Ryanair shareholders are maintaining or adding to their holdings.
"Because the share price has fallen, it has become, in our opinion, compellingly attractive," said Rory Powe, who manages European growth stocks at Man GLG and added to his Ryanair holdings in December. "I dont see Ryanairs cost advantage being eroded by anything more than an immaterial amount."
Ryanair's longer-term record as an investment may help explain why top shareholders appear reluctant to drop it.
The shares are up 150 percent over four years as it tackled a reputation for shoddy services and wooed higher-paying business passengers. Shares in its most successful rival, Easyjet, have barely moved in that time.
Analysts agree upcoming union negotiations will be crucial, amid questions over whether O'Leary's pugnacious management style is still appropriate for a company that was once an upstart challenger but now dominates European short-haul air travel.
"A unionized Ryanair will still be profitable, cash generative and value-creating in our view but will be a markedly different company," HSBC analysts said last week, rating the shares "reduce."
HSBC believes Ryanair unit costs could rise by 20 percent by 2020, with unionization and the localisation of contracts, and that Ryanair's talks with continental unions could be tough, ."..and airline strikes in these markets are commonplace."
Bernstein sees a short-term rebound in the stock but believes Ryanair is "cornered" - profitability per passenger will fall if it continues to expand at the current pace but a slowdown in expansion will cause earnings to decline.
Other big Ryanair institutional investors acknowledge the challenges but believe that, with its cost base still below rivals, it can still make bumper profits as the euro zone economy picks up steam, industry capacity growth eases and prices recover.
Ryanair has said it does not expect its costs to increase significantly.
Jonathan Fearon, who runs the Aberdeen Standard European Growth fund, said the market seemed to be pricing in the idea that Ryanair's cost advantage will disappear.
But he said staff costs were only one of many advantages in Ryanair's broad "ultra-low-cost" business model.
Fearon indicated Ryanair was still a favorite: it was a "category killer" in short-haul that stood to gain from Alitalia's troubles and as industry capacity growth slows, boosting fares.
Tony Gibb, investment director at another Ryanair shareholder, Fidelity, said the company's significant cost advantage, even over EasyJet, was "sustainable" and last month's share slump was "probably overplayed."
"There is negative press and the share price has come off quite meaningfully but that is probably more of an opportunity than anything else," said Gibb.
(Additional reporting by Conor Humphries and Victoria Bryan Graphics by Ritvik Carvalho; Editing by Susan Fenton)