The sector is now at its lowest point relative to the S&P 500 since 1931, according to Oppenheimer head of technical analysis Ari Wald.
"The sector has really been troubled not only for the last year, but really for the last five years. And there's a few things going against this sector that we think that it's too early to really make the call for a turn here," Wald told CNBC's "Trading Nation" on Monday.
Over the past 20 years, for example, the S&P 500 has risen more than 130% while the XLE energy ETF has fallen 3%.
"The trend is still bearish whether you're looking at it versus itself, versus the market. Point two, I think the broad-based internal weakness is a negative still weighing down on the group. Point three, I think you have the risk of a rollover in the price of oil. And point four, you've got the headwind of poor seasonals that the sector has to face as well. Typically November to January, the sector has been the worst performing of the 10, now 11, S&P sectors since 1990," said Wald.
Chad Morganlander, portfolio manager at Washington Crossing Advisors, is also urging caution in the energy patch.
"You have to be extremely nimble," Morganlander said during the same "Trading Nation" segment. "We have an underweight on the energy sector currently. We believe that over the next three months we would be somewhat more circumspect on investing in energy."
However, Morganlander struck a more optimistic tone about the longer term.
"If you look out over the course of the next 18 to 24 months, energy, we believe, should be part of a well-diversified portfolio. With the passing of Covid hopefully, going into 2021, into 2022, you could see better growth dynamics, better realignment between supply and demand when it comes to oil. So that could potentially take oil prices up, and as well bring the whole sector higher," said Morganlander.