Part of that empire was predicated on Baker's shrewd approach to real estate and belief in its value, even if the retail business wobbles. The company's willingness to part with its classic Fifth Avenue location stands in contrast with retailers like Macy's, which remains steadfastly loyal to its iconic Herald Square store in New York.
Hudson's Bay last year struck a deal to sell its flagship building to WeWork Property Advisors and Rhone Capital to give the company much-needed liquidity. At the time, Hudson's Bay said it would continue to operate out of the building, but at a smaller scale.
Still, Lord & Taylor's presence at the location would have been diminished if it continued on. In April, Lord & Taylor's longtime president, Liz Rodbell said she was stepping down after decades at the company.
In a filing Tuesday, Hudson's Bay said, "After evaluating best use scenarios for its New York City Fifth Avenue location, the Company has decided not to maintain a presence at this location following the turnover of the building to WeWork."
Hudson's Bay also said it expects to close up to 10 Lord & Taylor stores through 2019.
The company on Tuesday reported it had a net loss of $314 million Canadian dollars ($241 million) for the fiscal first quarter, widening from a loss of CA$214 million a year earlier. Quarterly revenue was up 1 percent, while comparable sales were down 0.7 percent.
Its stock is down 9 percent year to date.
Hudson's Bay underlined its efforts Tuesday to revive Lord & Taylor which, like its competitor J.C. Penney, appeals mostly to aging shoppers. Younger consumers are heading to trendier alternatives, including an array of web-based brands. As part of those efforts, Lord & Taylor launched a partnership with Walmart whereby it will sell its merchandise on the discounter's website.
Hudson's Bay also said this week it is selling flash sale website Gilt, roughly two years after paying $250 million for the business, in an e-commerce push to support its discount store Saks Off Fifth. It had already written down the acquisition by $116 million. Rue La La has agreed to buy the website.
Gilt came to prominence after the Great Recession, when high-end companies had excess goods they needed to get rid of. In the years since the recession's end, that merchandise became harder to buy. Meantime, the flash-sale websites are seeing increased competition from affordable fast-fashion brands like Zara and luxury consignment websites like the Real Real.
Some of the brands that sold in Saks, meantime, balked at seeing their goods on Gilt, sources tell CNBC.