Investing in a rising rate world

Even if the Fed doesn't hike rates now, it will soon and portfolios should be adjusted, investment pros say.

"I think if you're thinking 12-to-18 months or longer, it's not a question of if, it's when. And the market is overly preoccupied with September versus December," said Leo Grohowski, chief investment officer at BNY Mellon Wealth Management.

As a result, Grohowski has been recommending whittling back on fixed income in portfolios as the Fed moves into a rate hiking cycle. "We've been under-weighting fixed income in general. Within fixed income, we're obviously staying short duration. We're using floating rate high yield as an example of one tool in the toolkit that will help portfolios in a rising rate environment," he said.

JPMorgan's Stephen Parker said investors have been too concerned about the timing of the Fed rate hike and too worried about the impact.

Read MoreRising interest rates: What you need to know

"We think the growth dynamic, while muted, is on a positive track," said Parker, U.S. head of multi-asset strategies for global access portfolios at J.P. Morgan Private Bank. "It's like being in a doctor's office, waiting for a shot, and I think the anxiety you feel, the anticipation of getting a shot, is a little worse than the shot itself. I think once we can get through this first rate hike and realize it's not a major change to the fundamentals of the economy, we're talking about going from all time lows in rates to almost lows."

Parker said equities is his favorite asset class, and the recent selloff has created a buying opportunity, but the U.S. market is a bit pricy. Sectors he prefers in the U.S. market are consumer discretionary, banks, health care and technology. As the Fed rate hike cycle begins, he is looking more at developed markets overseas for opportunities, where central bankers are still moving toward easy policies.

"When we're thinking about how we build a portfolio, the thinking is evolving. There are two drivers we are looking for—that's markets that have policy tailwinds and for markets where we think earnings growth is going to deliver," he said. Europe and Japan fit that criteria. Emerging markets are still too iffy, but he is watching them for opportunities since they are reaching extremes.

Read MoreThis might be the worst Fed option

Grohowski said as the rate hiking cycle gets underway, the traditional rules of balancing stocks versus bonds may no longer work. In its fully diversified growth with income model portfolio, BNY currently has 27.8 percent fixed income, 47 percent equities and the rest in diversifiers, such as real estate and commodities. In its growth with income portfolio, it recommends about 64 percent stocks, 30 percent bonds and about 6 percent diversifiers.

"The old 60/40 portfolio of stocks and bonds. a lot of that is based on 20 to 30 years of historical data. That's where I think you have to ask is the next 10 to 20 years likely to be as rewarding for fixed income investors. Too many investors are overstaying their welcome in fixed income," he said.

Other diversifiers include absolute return hedge funds, long-short hedge funds, as well as managed futures, which are volatile and should total only about 3 percent of a portfolio, Grohowski said.

"I think the market volatility that we've witnessed will end up being a proof statement behind why investors should be incorporating alternatives. A lot of them are meant to provide diversification and provide a smoother ride," he said. "Those types of tools in the toolkit are for more volatile markets and I think that ties into this market that is uncertain about the future direction of interest rates and Fed policy. ... We're in for a volatile couple of weeks here and probably a more volatile period for the next 12 to 18 months."

Grohowski said investors may not be prepared for the a period of coupon-minus returns, where some bonds issued with low yields become worth less than par as rates rise. "It takes on importance for investors that have liquidity needs or if you're a pension fund that has to mark to market. I think this coupon-minus challenge is one we haven't been confronted with for 30 years," he said.

Read MoreHow top money managers are playing rising rates

But "bonds will always be an important anchor in portfolios. Right now stocks and bonds are more correlated than they've been historically which is a bit dangerous, too," he said.