"You don't want just huge amounts of cash in these places," Holder told the audience at the University of Virginia. "They want to be able to use the banking system. And so we will be issuing some regulations I think very soon to deal with that issue."
Holder's comments echoed remarks by his deputy, James Cole, in September during a Senate Judiciary Committee hearing on Capitol Hill.
(Read more: How cannabis could be the next great US industry)
Colorado this month became the first state to open retail outlets legally permitted to sell marijuana to adults for recreational purposes, in a system similar to what many states have long had in place for alcohol sales.
Washington state is slated to launch its own marijuana retail network later this year, and several other states, including California, Oregon, and Alaska, are expected to consider legalizing recreational weed in 2014.
The number of states approving marijuana for medical purposes has also been growing. California was the first in 1996, and has since been followed by about 20 other states and the District of Columbia.
But the fledgling recreational pot markets in Colorado and Washington state have sent a new wave of cannabis proprietors clamoring to obtain loans and make deposits in banks and credit unions.
The Justice Department announced in August that the administration would give new latitude to states experimenting with taxation and regulation of marijuana.
But with the drug still outlawed at the federal level, banks are barred under money-laundering rules from handling proceeds from marijuana sales even in states where pot sales have been made legal.
The lack of credit for marijuana businesses, however, poses its own criminal justice concerns, Holder said.
(Read more: World leaders slam war on drugs as 'a disaster')
"There's a public safety component to this," he said. "Huge amounts of cash—substantial amounts of cash just kind of lying around with no place for it to be appropriately deposited—is something that would worry me just from a law enforcement perspective."
Holder did not offer any specifics on a timeline for action on banking services for marijuana. Cole in September said the Justice Department was working on the issue with the Treasury Department's financial crimes enforcement network.
Critics of liberalized marijuana laws have said the lack of credit faced by pot retailers was beside the point.
"We are in the midst of creating a corporate, for-profit marijuana industry that has to rely on addiction for profit, and that's a much bigger issue than whether these stores take American Express," said Kevin Sabet, co-founder of the anti-legalization group Smart Approaches to Marijuana.