When Grant Rawdin launched his fee-only financial planning practice in 1987, the Black Monday crash in October of that year might have dashed his entrepreneurial dreams. Instead, it fueled them.
"It was actually helpful in building the firm," said Rawdin, a former lawyer and accountant who at age 27 founded Wescott Financial Advisory Group—ranked third in CNBC's inaugural list of the top-rated fee-only wealth management firms in the country. "I learned about volatility and, working with the few clients, that I had to get through it."
The experience has come in handy. While the recovering markets have helped financial advisors of all stripes lately, the crises that have regularly rocked investors over the last few decades have played to the advantage of registered investment advisors.
"We've been through the biggest financial crisis since the Depression and witnessed the biggest Ponzi scheme in history," said Peter Mallouk, president of Creative Planning, the top-ranked fee-only wealth management firm in the country, according to CNBC. "It's caused people to consider where their money is and who they're dealing with."
Bernie Madoff notwithstanding, investors continue to gravitate toward RIAs, who at least nominally promise to always act in the best interests of their clients. The first generation of registered investment advisors—many of whom launched their firms in the late 1980s and early '90s—have transformed wealth management, shifting the focus of the industry from investment sales to financial planning and from product commissions to fiduciary fees.
The combination of rising compliance and technology costs, increasing competition and changing demographics is posing serious challenges to RIAs—in particular, small solo practitioners—and the ranks of fee-only advisors is continuing to swell, causing their share of the retail advisory market to increase. Leaders in the industry expect that will continue.
"We're no longer the underdog, and we're in the laser sights of those we're taking market share from," Rawdin said. "But we still have the wind at our backs."
The thousands of small RIAs that have mushroomed across the country, however, will be hard-pressed in an environment of rising costs and aging baby boomer clients.
"It's getting much harder to be small in this industry," said Russell Hill, CEO of Halbert Hargrove, which has nine financial advisors managing $4.1 billion in assets. The company is ranked second on the CNBC list of top fee-only wealth management firms. "We haven't even hit the big increases in compliance costs yet."
Hill's firm is focused on the "low end of the high-net-worth market," he said, with the typical client having a couple of million dollars in assets. It advises several small endowments and foundations, as well as a few corporate retirement plans.
Last year Halbert Hargrove launched a separate investment management consulting business targeting institutional clients that will share the firm's technology platform with the retail business.
"There are multiple paths to success in this industry, but the problems come when you try to straddle more than one," Hill said. "You can have a successful lifestyle practice, you can acquire other firms, or you can find a partner to help you grow, and the earlier you figure it out, the better your financial life will be."
The firms now trying to build capacity will face the most pressure, Rawdin explained. As they make the necessary investments in staff and technology, profits will fall and risks will increase.
"Those who stay small can continue to make a nice living, but they can't grow outside that band," he said, referring to firms with $100 million to $150 million in assets under management. "The midsize firms have to ask themselves [whether] they want to create all the systems that large RIAs already have and [whether] they have the expertise to execute mergers to grow."
Wescott Financial Advisory Group has acquired several smaller firms in the last few years but has had far more discussions than actual deals, according to Rawdin.
"Small firms are not profitable enough, and their owners usually have unrealistic notions of the value of their firms," he said. Rawdin said he expects to see more consolidation activity in the market for firms with $500 million to $1 billion in AUM, which are typically run as sustainable businesses rather than solo practices.
The firm has three partners, four offices—with the biggest part of its business in Philadelphia and southern Florida—and 14 advisors managing $2 billion in assets for about 400 clients. Like the other two RIAs that placed at the top of CNBC's list of leading fee-only wealth management firms, Wescott Financial Advisory Group has invested in young talent.
Read MoreAdvisors must rethink their roles
Four of the firm's advisors are currently in their 20s. The firm regularly looks to recruit individuals out of graduate school or early in their careers, ensuring that the next generation is well represented at the firm.
Creative Planning's Mallouk said he also anticipates stepped-up consolidation as the industry matures.
"Firms have to create economies of scale," he said. "Even at $500 million in assets, the compliance burden is so much greater." His firm currently manages just over $10 billion in assets.
Mallouk doesn't worry about the competitive response of the large brokerage houses that have shifted gears toward fee-based financial planning in recent years. He expects the movement of advisors out of brokerages to independent RIAs will continue apace and thinks it could take five to 10 years to resolve the issue of a universal fiduciary standard for all financial advisors—something the staff of the Securities and Exchange Commission recommended four years ago.
"The brokerages are trying to look more like the independent model without being fiduciaries," Mallouk said. "They're talking about fee-based accounts and financial planning, and they're lobbying to fight the fiduciary standard."
The next financial crisis might clarify the situation for investors.
"Whenever we get a bear market or a big Ponzi scheme is revealed, people get more educated about their advisory relationships, and money moves to the lower-cost, more transparent accounts," Mallouk said.
That will likely mean the RIA channel.