Chen downgraded the stock from "buy" to "hold," with a price target of $40 per share.
On CNBC's "Fast Money," he said that a larger apparel company could help Lululemon with supply-chain and distribution issues.
Read MoreLululemon a potential target for VF Corp: Analyst
Chen said that Lululemon founder Chip Wilson's 27 percent stake could be an "obstacle" in a takeover attempt and added that the former CEO could launch a bid to take the company private.
"It's possible because if you look at the free cash-flow profile – $150 million a year – as well as the net cash position, it's a company with a lot of growth potential," Chen said.
Lululemon operates in an attractive sector, which could continue to pressure on the company, Chen said.
"Performance sportswear is probably growing at high single to low double, if not better. It's a hot market," he said. "That's why we see so many entrants, from the Gap. Macy's has a private-label brand. You can get some great product at Target, so the Lululemon Wunder Under is over $100. So, it's a premium-price product, and there's a lot of opportunity in this space."
—By CNBC's Bruno J. Navarro. Follow him on Twitter @Bruno_J_Navarro.