Paul is insisting on a vote on his Fed transparency bill, and has informed Senate leadership of his intentions, the source said.
"As part of Senate consideration of the Janet Yellen nomination to be Chair of the Federal Reserve, I will request a vote on my bipartisan Federal Reserve Transparency Act, S. 209. The American people deserve transparency from the federal reserve and the federal government as a whole," Paul said in a statement following the reports.
A Senate Democratic aide told CNBC on Friday morning that the ability of Paul to single-handedly block the nomination "should not be overstated." Paul would need 40 other senators to back his effort to block Yellen's nomination; otherwise, Senate leadership could defeat his blocking effort.
Although hearings have not yet been scheduled, the aide said the leadership at this point is confident the nomination will succeed.
Paul intends to formally put the hold in place next week, once the Senate is back in session, the source added.
Markets gave back some of their earlier gains on the report. (Click here for the latest on the markets).
The senator's bill would mandate a complete audit of the Federal Reserve.
According to Paul's website, his bill is a version of legislation advocated by his father, the former Texas congressman and presidential candidate Ron Paul.
(Watch: Rand Paul says 'audit the Fed')
—By CNBC's Steve Liesman. Follow him on Twitter