"Gold miners are as hated as anything I've seen," said Arnold Espe, USAA's VP of mutual fund portfolios.
Indeed, after money poured into gold on fears of what central bank easing will mean down the road for inflation, investors since the beginning of the year have been moving into other investment opportunities as they position for better U.S. economic growth ahead.
According to Bank of America Merrill Lynch's June global fund manager survey, investors' allocation to commodities fell to the lowest level on record, with 32 percent underweight.
(Read More: How Gold And Silver Bears May Actually Help Silver)
One reason gold and gold miners have been abandoned has been the prospects that quantitative easing will eventually be coming to an end in the United States.
(Read More: Taper Tipoff? Bernanke Hints Easing End Is Nearing)
"One way or another, the U.S. is getting toward the end of its liquidity cycle. And that is supporting U.S. yields and it's supporting the U.S. dollar, and that's going to be a headwind for the gold price and all commodity prices going forward," Erik Wytenus, head of foreign exchange and commodities at JP Morgan Private Bank, told CNBC.
But while there may be headwinds, Espe expects the precious metal to benefit from ongoing global "monetary debasement" as central banks like the Bank of Japan print money in hopes of stoking inflation and reducing the value of their currencies and as developed countries continue to run large budget deficits.