Investors are feeling skittish, after the Dow Jones industrial average and hit their lowest levels since March.
But Scott Hanson, senior partner and founding principal at Hanson McClain, told CNBC's "Brexit: Facing the Fallout" special report that investors should take a "step back and realize that if they have a balanced portfolio, not all their money is getting hammered in the stock markets."
He explained that even if a portfolio was 50 percent invested in stocks, there was still another 50 percent that was protected from the perceived volatility in the market. Despite the move in equities Monday, the CBOE Volatility Index fell more than 7 percent.
Hanson said, however, that money that was intended to be used within the next five years shouldn't be in stocks or other long-term investments at all.
Manisha Thakor of The Bam Alliance said that while was was difficult, "doing nothing is doing something — it's an active decision to stay put."
If investors felt that the current market action was "destroying" their retirement, that should be a red flag, according to Thakor.
"What that tells me is you either have the wrong asset allocation, an inappropriate level of of diversification or an inappropriate financial advisor who hasn't explained to you why doing nothing is the correct active action to be taking in a market like this," she said.
The luxury real estate impact
Senada Adzem of Douglas Elliman said that real estate investors were waiting for a steep decline in London property values before they got back into the market. Her firm had seen property values decline by 20 percent in the past two weeks, which Adzem said wasn't enough.
She added that luxury properties in other urban European markets would also see declines, but not as steep as those experienced in London.
"The political instability in the U.K. is impacting mainland Europe because many people now are wondering whether there are some other countries, such as Greece, such as France, will consider exiting the EU," she said.
Adzem said, however, that she had also seen investors pull out of the London market and park their cash in U.S. real estate markets such as New York, Los Angeles and Miami.
"They're going to buy trophy properties in the United States because those are the properties that retain their value regardless of the economy," Adzem said.