Stocks could have further downside from here, but Ritholtz Wealth Management's Josh Brown has some good news for investors: Markets are never down three years after a recession. "We studied every post-World War II recession, and the effects of the stock market before, during and after," Brown said Wednesday on CNBC's " Closing Bell: Overtime ." "Do you know how many times three years after a recession has been officially declared, the stock market has been down? Zero times. It has never happened." What's more, stocks posted a triple-digit return after five years in "every single instance," he said. Brown made the remarks even as he believes that Wall Street will face greater challenges ahead, such as data from June that showed inflation reaching its highest level since 1981. Investors are also assessing how rising prices will hit corporate earnings forecasts this month. "It's possible that earnings don't fall apart right away, but it's just a different environment with a different set of challenges. And these V-shaped bottoms that everybody had grown accustomed to for so long — some people their entire career of investing, that's all they ever saw. This is not that. We're in a downtrend," he said. Still, Brown maintained that investors will be fine so long as they keep a long-term perspective on markets. "If you understand that in the back of your head, the question is not, 'Do I buy stocks, do I sell stocks?' The question is, 'What am I expecting from the stocks that I buy now?' If you can stay calm and understand that this is not about where the S & P ends 2022, you're going to be fine," he said.