The stock markets could be hurt by rising oil prices, said James Paulsen, Wells Capital Management's chief investment strategist.

"When you go back to 1980, the one time when you have a negative correlation between oil prices and stock prices, is when you fall below 6 percent unemployment," Paulsen told CNBC's "Power Lunch" on Thursday.

"I think the reason for that is because, at that point, people start noticing the labor market is tightening up. ... So, when oil goes up from relatively tight resource markets, sometimes the stock market starts looking at it as a negative."

U.S. crude prices were up about 4 percent Thursday afternoon, while Brent was up about 5 percent. The Bureau of Labor Statistics reported last Friday that the unemployment rate ticked up 0.1 percent to 5.7 percent.

Read More Summers: Why the Fed should not raise rates

Paulsen added he believes this current scenario could lead to a number of problems, including concerns about the Federal Reserve raising rates, as oil is starting to hit bottom. "This is just another challenge for the stock market this year," he said. "I think [the Fed is] going to start that process, and I think we're going to struggle with that."

Nevertheless, some experts disagree in part with Paulsen's assessment. "I don't think the Fed is going to be compelled to do much," Ron Insana, a market analyst and CNBC contributor, said on "Power Lunch" Thursday. "If anything, [the Fed] will raise a token eighth of a point to show they're maybe thinking about this."