By a vote of 87-12, the Republican-led Senate passed and sent to the House of Representatives the "First Step Act," which would ease the way for some prisoners to win early release to halfway houses or home confinement.
The legislation also aims to establish programs to head off repeat offenders and protect first-time non-violent offenders from harsh mandatory minimum sentences.
Earlier this year, the House passed a bipartisan bill focusing on prison reforms, which did not include sentencing reforms.
With little time as Congress tries to wrap up its session this month, Senate proponents are hoping their broader version is accepted by the Republican-controlled House.
The United States leads the world in prison population, with about 2.2 million people incarcerated at the end of 2016.
During Senate debate of the bill, Democratic Senator Dick Durbin noted the United States had 5 percent of the world's population, but 25 percent of the world's prisoners.
He added that minorities bore the brunt of tough minimum sentences that judges have been directed to impose as a result of a decades-old law that has exploded the numbers of incarcerated people.
"The majority of illegal drug users and dealers in America are white. But three-quarters of the people serving time in prison for drug offenses are African-American or Latino," Durbin said.
In response to criticism from some conservatives that the legislation could prompt the release of violent criminals into society, the bipartisan measure was reworked to scale back the discretion judges would have in some sentencing cases.
Before passing the bill, the Senate defeated amendments by Republican Senators Tom Cotton and John Kennedy that would have further tightened requirements.
Those amendments would have excluded child molesters and other violent felons from early release, required notification of victims before offenders are let out of prison early and included a plan to track the effectiveness of anti-recidivism programs.
The push for the legislation gained momentum as progressive Democrats were joined by fiscal conservatives, who saw the potential for savings if the U.S. prison population was reduced, along with religious conservatives who preached the importance of giving people a second chance.