Abercrombie's problems aren't all about the logo

Having lost its "cool kid" factor in the United States several years ago, Abercrombie & Fitch's international business was the beacon on which many analysts who stuck with the company had pinned their hopes.

But after 10 consecutive quarters of double-digit same-store sales declines and sales growth made even more challenging by a stronger U.S. dollar, that argument is beginning to lose some steam.

ABERCROMBIE SHARES
Chris Ratcliffe | Bloomberg | Getty Images

Over the past few weeks, several investment banks have lowered their ratings on the company, in part due to its slowed international trends and foreign exchange headwinds.

Read MoreWhy Macy's latest bid for growth may struggle

These woes are the latest set of challenges facing the retailer, which saw the exit of controversial CEO Mike Jeffries in December. It's also fighting a secular change that's denting sales for all of the traditional teen retailers, as their target shopper shifts their spending to lower-cost, fast-fashion stores.

"The Abercrombie bull case used to rest on the international business improving," Morgan Stanley analyst Kimberly Greenberger wrote in a note to investors on Thursday. "We are skeptical Abercrombie & Fitch can turn the international business around."

Read MoreWill this lingerie shop join the retail graveyard?

Greenberger downgraded Abercrombie's rating from "equal weight" to "underweight," assigning it a price target of $18. The company's shares were trading about 3 percent lower on Thursday, near $24.

Big shifts seen in retail
Big shifts seen in retail   

This month, Credit Suisse analyst Christian Buss downgraded Abercrombie to "underperform," noting the revenue pressure that weighed on brands with a heavy international presence in the fourth quarter. Given that 35 percent of Abercrombie's business comes from overseas, and that its stores there are more profitable than its U.S. locations, this trend doesn't bode well for the company.

What's more, additional trouble is brewing for the teen sector, as the U.S. entry of British value retailer Primark looms. The fast-fashion category has already caused a dent in prices and margins among the teen set, which should only get worse after Primark opens this year, Buss said. Whereas prices are already low at Forever 21, H&M and Uniqlo, they're even lower at Primark.

Read MoreMake money off your selfies with this new app

A recent study by Credit Suisse compared the cost of nine "core product" categories, including T-shirts and skinny jeans, across fast-fashion stores in the U.K. The total basket price at Primark came in at 99 pounds, significantly lower than the average of 212 pounds.

"Primark's entrance to the USA in '15 could be a key event in the growth of 'deep value fashion' and will prove disruptive for the apparel sector as a whole," Buss wrote in a note to investors.

"The retailer's extreme low prices on fashion basics will help reset consumer expectations for pricing across apparel retail, eroding pricing premiums and taking profits from mid-value specialty retailers, department stores and discounters."

For its part, Abercrombie has made a number of tweaks to its business model, in an effort to cut costs and boost sales. The biggest change has been its shift away from logoed products.

Abercrombie will release its fourth-quarter results on Wednesday. Thomson Reuters forecasts call for the retailer to earn $1.16 a share on revenues of $1.168 billion.