There is mounting pressure to postpone the Olympics for the first time in their 124-year history.
Monday evening, the United States Olympic and Paralympic Committee reversed its course after surveying its athletes.
"It's more clear than ever that the path towards postponement is the most promising," the committee said in a statement.
The USOPC says 1,780 Team USA athletes shared their input on the games and the issues related to COVID-19.
"Our most important conclusion from this broad athlete response is that even if the current significant health concerns could be alleviated by late summer, the enormous disruptions to the training environment, doping controls and qualification process can't be overcome in a satisfactory manner," it adds.
The U.S. becomes one of several countries calling for the postponement.
Billions of dollars are at stake. Japan has pumped at least $12 billion of investment into the games, not to mention the losses that would be incurred by sponsors, broadcasters and others. Rescheduling would also be logistically complicated, as organizers would have to make sure the new dates didn't conflict with other major events.
The International Olympic Committee, the Olympics' governing body, provided an update to its athletes and the public on Sunday, announcing it is stepping up scenario planning and will have a decision on the games within the next four weeks.
"On the one hand, there are significant improvements in Japan...on the other hand there are a dramatic increase in cases and new outbreaks of COVID_19 in different countries on different continents," the group said.
Sources familiar with the matter tell CNBC that pressure is mounting and there could be a decision even sooner. One person told CNBC that postponing the games is the most likely scenario, as opposed to an Olympics without spectators which would "drain excitement" from the games.
One IOC member, Dick Pound, told USA Today on Monday afternoon that the games would be postponed, but the IOC has not made an official announcement yet.
"It is the right of every IOC Member to interpret the decision of the IOC EB which was announced yesterday," an IOC spokeperson said in response.
In a letter to Olympic athletes on Sunday, IOC Chair Thomas Bach ruled out canceling the Olympics. "Cancellation would not solve any problem and would help nobody. Therefore it is not on our agenda."
The Olympics have only been canceled three times: In 1916, during World War I, and in 1940 and 1944 during World War II.
On Sunday, Canada became the first major country to speak out and withdraw its athletes from going to Tokyo this year due to COVID-19 risks.
"While we recognize the inherent complexities around a postponement, nothing is more important than the health and safety of our athletes and the world community" the Canadian Olympic Committee said in a statement.
Team Australia says the stress has been difficult on its athletes, it told its team to prepare for Tokyo in 2021.
"Our athletes have been magnificent in their positive attitude to training and preparing, but the stress and uncertainty have been extremely challenging for them," said Australia's Olympics Chef de Mission, Ian Chesterman.
Other countries have begun to follow suit. "My clear advice to the board (of the Norwegian Olympics committee) is that you should ask not to send any athletes to the Paralympics or Olympics in Tokyo 2020 in light of the serious situation," Norway's Olympic Committee president, Berit Kjoell, told reporters on Monday.
Austria's sports minister is also urging postponement.
"The closure of sports and training facilities in many (but not all) countries has already raised doubts about the equal opportunities for participants in the 2020 Summer Games," Werner Kogler, who is also Austria's vice chancellor, said in a statement.
Britain is expected to follow suit shortly according to the BBC.
With more than 300,000 cases worldwide, the World Health Organization says it has confidence the Japanese government and the International Olympic Committee would not proceed with any games if it were to be dangers to athletes and spectators.
Japanese Prime Minister Shinzo Abe spoke out about the the possibility of postponement for the first time on Monday. Abe says if holding the games in their complete form became impossible, "we may have to consider postponing the Games, given the Olympic principle of putting the health of athletes first," he said.
Even the athletes are speaking out.
"We have heard from hundreds of athletes and today we are calling on both the International Olympic Committee (IOC) and the International Paralympic Committee (IPC) to place athlete safety and welfare first by postponing the Games until the COVID-19 global pandemic is under control," Global Athlete, a worldwide athletes group, said in a statement.
Disclosure: CNBC parent NBCUniversal owns NBC Sports and NBC Olympics. NBC Olympics is the U.S. broadcast rights holder to all Summer and Winter Games through the year 2032.
Correction: The 1916 Olympics were canceled during World War I. An earlier version misstated when the war began.