After a run toward record highs in stocks last week, there may be better clues on why the rally faltered: the market hadn't priced in Brexit. In fact, it's probably still unprepared for the event if it happens.
"I think stocks on the whole are feeling that concern. They (were) pricing in none of the risk," said Jason Pride, director of investment strategy at Glenmede Trust. As stocks in the last few days have moved lower under increasing concern, he would expect "more volatility and anxiety" heading into the vote as Brexit becomes a stronger factor for the market.
Two-thirds of fund managers think Brexit is "unlikely" or "not at all likely," according to the Bank of America Merrill Lynch June fund manager survey. The survey was completed from June 3 to 9, meaning those optimistic results show just how much Wall Street was not pricing in Brexit — until the latest slew of polls.
The S&P is down more than 2 percent since it hit a 2016 high on June 8, after several polls released in the last few days showed increased support for leaving the European Union. Late Monday, The Sun newspaper also endorsed Brexit and urged readers to vote "leave" next Thursday.
"I think the leave vote is surprising people that it's gaining momentum," said Quincy Krosby, market strategist at Prudential Financial.
At the same time, fund managers in the monthly BofAML survey said Brexit is the biggest tail risk, topping even concerns about a Chinese currency devaluation that was a significant factor in the stock market corrections last summer and earlier this year.
Market complacency about a top risk event with increasing chances of occurring will likely lead to further near-term price fluctuations. The CBOE Volatility Index, considered the best gauge of fear in the market, topped 22 for the first time since Feb. 24 on Tuesday.
"I would not be surprised to see further downside in the case of leaving the U.K.," Pride said. He expects a "strong snapback" if Britain stays.
Some analysts believe there will be a bounce back anyway because the volatility from Brexit will be short term.
"I think it's going to cause a lot of volatility around the U.K. and a lot in Europe but I don't see the longer-term effects," said Scott Wren, equity strategist at Wells Fargo Investment Institute. "If we get some downside here from it in stocks, I think it's a buying opportunity."
To be sure, most market analysts don't think Britain will actually leave the European Union.
"We don't think the base case is the U.K. will vote to leave the EU," said Gene Tannuzzo, senior portfolio manager of Columbia Threadneedle's Strategic Income Fund. He said he expects the roughly 10 percent of poll respondents who say they're undecided to vote to remain.
Analysts also generally put more weight on betting odds than poll results. The bookies see a two-thirds probability Britain stays in the European Union, according to oddschecker.com.
"There is concern (about Brexit) but to me it's more manifested in the cash on the sidelines," Krosby said.
The BofAML survey showed June cash levels rose to 5.7 percent, the highest since November 2001.
— CNBC's Patti Domm contributed to this report.