The Retail Industry Leaders Association is calling the West Coast ports situation a "catastrophic shutdown of America's supply chain."
Founder and CEO of Talmage Advisors, Liz Dunn, told CNBC's "Closing Bell" on Wednesday that for now retailers are the ones to suffer.
"Retailers don't have the power to mark up and pass on to the consumer, so I think it's going to impact the retailers in a lot of ways that won't necessarily be as visible to the consumer," she said.
Read More Port fight costing us $400,000 a day: CEO
Dunn said that retailers will start planning ahead for similar scenarios by crafting contingency plans. The companies biggest hit so far are Perry Ellis, Ann Taylor and Lululemon.
"As we head into earnings season, I think we'll hear about it from a pretty large swath of retailers," she added.
One potential long-term impact could be production moving back to the U.S., Dunn said. It could also move to Mexico and Central America, where port problems would be a non-issue.