Steel and nonferrous metals, electronics and precision machinery companies were the most affected, with around half saying business had suffered to some extent.
The poll, conducted for Reuters by Nikkei Research, canvassed 487 firms of which 280 replied. Companies participate on condition of anonymity.
Failure to communicate
Weaker sales was the most often cited complaint, while some also noted difficulties in procuring materials or goods.
Read MoreChina's tourist spends $1000-plus a day on average
Others said it was hard to proceed with projects as prospective or former business partners refused to talk to them. This seemed to be particularly so when dealing with state-owned companies.
"Our business relationship with a Chinese broadcaster has come to a complete halt," responded one manager at information and communications-related company.
Concerns over the safety of their expatriate employees and a belief that goods are moving more slowly through customs than previously were also mentioned.
Read MoreWe are friends with China and Japan: Australia
Since coming to office in December 2012, Abe's government has halted a decade of declines in military spending, promoted the country's defense industry and dropped a ban on aiding a friendly country under attack.
Beijing and Seoul, major victims of Japan's wartime aggression, accuse the hawkish Abe of wanting to revive militarism and of whitewashing the nation's past. Japan's competing claims with both countries over tiny islands have also flared into acrimony.
The diplomatic tensions have been borne out in foreign investment data.
Read MoreJapan firms near crisis as labor shortage deepens
In 2013, investment by Japanese companies in China slid by a third and investment in South Korea fell 18 percent, according to the Japan External Trade Organization. That compares with a 21 percent climb for Asia, including a 70 percent jump in the Philippines.
In the first quarter of this year, investment in China tumbled by more than half to $1.3 billion, although South Korea has seen a rebound with investment up 43 percent to $1.1 billion.
In late June, prominent academics, including Abe economic adviser Koichi Hamada tried to give the premier a "peace and security" proposal to improve relations with China and Korea, but their report was rejected, people familiar with the matter told Reuters.
Read MoreWhat happened to Japan's yen-driven export boom?
"If tensions were to reach a boiling point, the worries are that the Chinese government would do things like restrict store openings and create hurdles for factories to operate," said Tetsuya Inoue, a senior researcher at Nomura Research Institute, who reviewed the survey results.
Abe has met with the leaders of all Southeast Asian nations but has not been able to secure a summit with the presidents of China or South Korea. This week he said he wants to meet with Chinese President Xi Jinping at a Pacific Rim summit in Beijing in November to improve relations.