Foreign bond prices also fell Thursday. The yield on U.K. 10-year bonds climbed to 1.318 percent, while the 10-year German bund yield hit a high of 0.569 percent, its highest level since January 2016.
In economic news, 158,000 jobs were created in the month of June according to this morning's ADP private-sector payrolls report. The number of new jobs is below the expected 185,000, prefiguring the Bureau of Labor Statistic's monthly jobs report this Friday.
The US weekly jobless claims totaled 248,000 versus the 243,000 estimate.
Economic activity as reported by the ISM non-manufacturing index hit 57.4 in June beating the 56.5 estimate.
There are also three Federal Reserve speakers Thursday. They include San Francisco Fed President John Williams, who speaks at 3:45 a.m. ET in Hobart, Australia, at the Economic Society of Australia. Fed Gov. Jerome Powell speaks at 10 a.m. at the American Enterprise Institute, and Vice Chairman Stanley Fischer speaks at a 7:30 p.m. event at the Martha's Vineyard Hebrew Center Summer Institute on Government Policy and Labor Productivity.
Oil prices surged on Thursday morning with a larger-than-expected 6.3 million barrel drop in US stockpiles, but later pared those gains after a two percent drop.
Brent crude traded around $47.85 a barrel Thursday afternoon, down 0.54 percent, while U.S. crude was around $45.34 a barrel, up 0.47 percent.
—CNBC's Patti Domm and Gina Francolla contributed to this report.