A Men's Wearhouse in New York

Men's Wearhouse said Monday that it won't give rival Jos. A Bank access to nonpublic information that it could use to assess whether to potentially raise its $2.3 billion buyout offer.

Shares of both companies fell in afternoon trading.