Trump and Abe played under mostly clear skies at Trump National Golf Club in Jupiter, Florida. The White House did not immediately say whether anyone else joined them and reporters were not allowed on the course to watch the pair.
After four hours at the club, Trump tweeted a picture of himself and the prime minister on the golf course.
"Having a great time hosting Prime Minister Shinzo Abe in the United States," Trump said in the tweet.
Shortly after, Trump and Abe left the club to head to a private lunch.
Abe traveled to Florida on Air Force One with the new U.S. president after a more formal meeting at the White House on Friday. The weekend excursion is Trump's first use of his Mar-a-Lago resort to entertain an international leader.
Washington and Tokyo have said the weekend trip is a sign of the importance that both leaders place on the U.S.-Japan relationship.
Abe, in particular, sought to ease concerns in Japan about the fate of the decades-long alliance under Trump, who had raised concerns about U.S. military spending in Japan during the rancorous 2016 presidential election campaign.
At a news conference on Friday, Trump and Abe focused on areas of common ground, with Trump setting aside campaign pledges to force Tokyo to pay more for U.S. defense aid and Abe promising to help the United States to create jobs.
Trump and Abe are set to meet again for dinner Saturday evening, along with some members of the Japanese delegation. The two leaders are set to depart Florida on Sunday.