That is concerning because it may mean shipments actually swing negative in the later months of 2014, potentially ending a multiyear run of market share gains in the key U.S. market. Already, the company said its share of large motorcycle sales in the U.S. has fallen 1.8 percentage points so far this year to 52.3 percent.
Read MoreHarley Davidson trims forecasts, stock putters
Such a loss is tough to manage because the overall motorcycle market has begun to level off, thanks in part to an aging customer. Harley-Davidson has responded by introducing some smaller, less expensive models that appeal to a new generation of enthusiasts.
The company didn't respond to a request from CNBC.com for comment.
Harley-Davidson still has serious hurdles to overcome. First, smaller bikes tend to have lower profit margins than the hogs that made the company famous. Even if the company successfully transitions into a new categories, they may not be big moneymakers.
There are also plenty of other choices on the market. Polaris, which also reported earnings Tuesday, is in the process of relaunching its so-called Indian bikes. The company said its motorcycle sales more than doubled in the second quarter to $103 million, thanks to demand for Indian models. Indian models still only have a small fraction of the dealership representation Harley-Davidson enjoys, indicating there is plenty of scope to keep taking share.
The other major threat to Harley-Davidson is its financing division, which has thrived since the crisis as consumers had fewer places to find loans. The financing division accounts for a full quarter of Harley-Davidson's operating profit, but it's likely to begin shrinking.
Read MoreHarley-Davidson recalls 66,000 Touring cycles
One reason is that financing for assets like motorcycles has begun to return as the financial markets heal. The amount of asset-backed securities based on auto loans, for instance, already looks on track to return to pre-crisis peaks, according to the Securities Industry and Financial Markets Association.
So while Harley-Davidson may have been able to enjoy very wide spreads on loans to customers, competition is bound to squeeze margins. The company acknowledges that operating profit from the financing division may fall slightly in 2014.
The bull case on investing in hogs: Harley-Davidson has endured plenty of tough times and kept on running. But the stock has nearly quadrupled since the start of 2009 and trades at 16 times consensus 2014 earnings.
Buffett, who loaned Harley-Davidson about $300 million in 2009, was recently paid back. It may serve investors well to follow the smart money's lead.
—By CNBC.com's John Jannarone