Read MoreIs Spotify cannibalizing the music industry?
Ken Parks, Spotify's chief content officer, said he was optimistic about his company's prospects, and noted that the negotiating process was slow wherever it went. Spotify, which has more than 10 million customers in 57 markets around the world, negotiated with labels for almost two years before it arrived in the United States in 2011, for example.
"When the decision makers finally feel that the heat is intense enough that they have to do something different, they will," Mr. Parks said. "I think we are approaching that moment in Japan."
Others have doubts, pointing to the Japanese market's devotion to the CD, which remains a primary source of revenue for record labels in the country, and an indispensable promotional tool.
Peculiarities of Japan's business climate have shaped its attachment to the CD, but cultural factors may also be at play, like Japanese consumers' love for collectible goods. Greatest hits albums, for example, do particularly well in Japan,perhaps because of the elaborate, artist-focused packaging. The hugely popular girl group AKB48 pioneered the sale of CDs containing tickets that can be redeemed for access to live events — a strategy credited with propping up CD sales, because it can lead the biggest fans to buy multiple copies of an album.
Tower Records closed its 89 American outlets in 2006, but the Japanese branch of the chain — controlled by NTT DoCoMo, Japan's largest phone carrier — still has 85 outlets, doing $500 million in business a year.
More from the New York Times:
MacArthur Awards Go to 21 Diverse Fellows
Report Cites Forced Labor in Electronics Industry in Malaysia
In Performance | Lance Coadie Williams
At Tower's flagship store, in the heart of the skyscraper-lined shopping district of Shibuya, a group of preteen girls called Kokepiyo performed for fans and autographed CDs one afternoon last month, while their mother-managers watched protectively. Outside, an 18-year-old student who gave her name as Yuria had come to Tower to see her favorite band, the Lotus. She carried a bag full of merchandise she had bought at the store, and said that she frequently buys multiple collectible copies of CDs.
"Each store has its own freebies to give away to sell more CDs," Yuria said. "So itall depends on how good they are."
In the United States, digital sales have long since overtaken physical ones. But CDs still account for 41 percent of the $15 billion recorded music market worldwide, and, in addition to Japan, some big markets like Germany remain reliant on CD sales. That attachment worries some analysts, who contend that if those countries do not embrace online music, an inevitable decline in CD sales will further damage the industry.
"If Japan sneezes and Germany catches a cold,that's it — we're done," said Alice Enders, a media analyst with Enders Analysis in London.
A distinctive business ecosystem in Japan has kept CD sales lucrative for music companies. Pricing restrictions on retailers keep the cost of most new CDs at more than $20. In the mid-2000s, a nascent download service, Recochoku, was tethered to Japan's expansive cellphone market, but that system collapsed once the country moved on to smartphones like the iPhone.
Read MoreSony CEO: Why we need to stay in smartphones
Part of the problem, executives say, is the complex array of companies that control rights to the most popular music in Japan, which have been very slow to license new services.
Sony's Music Unlimited, for example, is the largest available streaming service in Japan, but it lacks the most popular hits there. (Sony declines to say how many subscribers it has to Music Unlimited, in Japan or elsewhere.) Apple's iTunes store arrived in Japan in 2005, but only in 2012 did it begin to sell the Japanese music titles of its hardware rival Sony.
Executives in Japan and the West blame an overly cautious Japanese music industry for not adapting, and serious worries remain about Japan's ability to recover from its losses last year.
"A substantial amount of senior management is worried about what happens on their watch, but not necessarily worried about what happens after that," Shigeo Maruyama, the former president of Sony Music Entertainment Japan, said in an interview.
This year, things in Japan are looking slightly better. In 2013, there were no million-selling albums, but this year there have been two: a Japanese version of Disney's "Frozen" soundtrack and the latest release by AKB48. Yet in the first half of the year sales were still down an additional 3 percent compared with a year earlier.
Read MoreApple on the brink of another 'super cycle'
"The Japanese record companies' hope is to maintain the current size of the physical market, and to try to make the digital market grow again by licensing new digital services," said Yoichiro Hata a director of the Recording Industry Association of Japan.
For the rest of the struggling global recording industry, that growth cannot come soon enough.
"It's inevitable that this market comes back to growth," said Mr. Grainge, of Universal. "What I'm not going to predict is when."