Read MoreBank of England's Carney: rate rise 'getting close'
"Looking forward, we expect U.K. GDP (gross domestic product) growth and the pace of the decline in U.K. unemployment to moderate in the quarters ahead, both in absolute terms and relative to the U.S. All else equal, this would imply the need for a slower pace of tightening in the U.K.," Pill said.
Britain's economy grew 0.9 percent in the three months from April to June this year, the quickest increase since the third quarter of 2013. The U.S. economy, meanwhile, expanded at a 4.6 percent annual rate over the same period, its fastest pace in 2-1/2 years.
Looking back over the past quarter-century, the BoE has typically tightened more slowly than the Fed, according to Goldman.
Read MoreThe Federal Reserve is making a major shift in interest rate policy
On the basis of five tightening cycles for the U.K. and three for the U.S. over this period, the Fed has raised rates on average by 2.1 percentage points in the first year, while the BoE has raised rates by just 1.3 percentage points.
One reason they have risen more gradually is that the U.K.'s private sector has been more highly leveraged compared with the U.S. In addition, the average duration of debt held by British households is shorter than their American counterparts.
"This has meant that changes in policy rates in the U.K. have provided more 'bang for their buck' than has been the case for changes in policy rates in the U.S.," Pill said.
Read MoreBoE asks government for more powers over housing
One risk to Goldman's forecast that the BoE will hike rates first is that the growth slowdown in the euro zone may produce a bigger-than-expected drag on the U.K. economy.
"In this eventuality, the BoE could well delay the timing of its first interest rate increase," Pill said.
"For this reason, we have less confidence in our expectation that the U.K. will move first than we do in our expectation that, when rates to begin to rise, they will do so more gradually in the U.K. than in the U.S.," he said.