They say that the amount of trading being done in the "dark" means that publicly quoted prices for stocks on exchanges may no longer properly reflect where the market is, meaning that investors may not be getting the best prices for their trades.
The measure under consideration, known as a "trade-at" rule, has long been sought after by exchanges like Nasdaq OMX and the New York Stock Exchange as a way to win back market share against off-exchange competitors such as Credit Suisse's Crossfinder, one of the largest dark pools in the United States.
Read More Goldman considers draining its dark pool, Sigma X: WSJ
The talks within the SEC are at an early stage, and the pilot program would need to be approved by the full five-member commission, as part of an order instructing the public exchanges to carry out the study.
The initial trade-at test would only apply to a small number of thinly traded stocks, as part of a broader study exploring ways to incentivize more active trading in smaller and mid-sized company stocks, the sources said.
The results of any pilot would have to be effective enough to prompt the SEC to change its market rules.
SEC spokesman John Nester declined to comment on the agency's plans.
Meaningfully better price
Currently, SEC rules require trades to be executed at the best available bid or offer, whether it be on a public exchange or in a dark pool, which doesn't publicly disclose bids or offers.
A trade-at rule would force brokerages and dark pools to route trades to public exchanges, unless they can execute the trades at a meaningfully better price than available in a public market.
Read MoreFormer SEC Chair: Markets don't need more regulation
It is unclear how the SEC would define a meaningfully better price.
Dark pools were created to allow large investors to trade big blocks of trades without tipping off the broader market. Brokerages also offer similar dark markets internally to their clients.
Such venues have come under fresh scrutiny, along with high-speed traders, after bestselling author Michael Lewis recently published a book alleging the U.S. stock market is rigged in favor of high-frequency traders.
Dark pools, which are regulated more lightly than exchanges, have gained popularity for trades of all sizes because they allow firms to avoid paying exchange fees.
Around 40 percent of all U.S. stock trades, including almost all orders from "mom and pop" investors that go through brokerages, now happen "off exchange," up from around 16 percent six years ago.
Regulators in Canada and Australia have already taken steps to curb the growth of dark trading by requiring, for instance, that off-exchange trades be of a minimum size or have a significantly better price than can be found on an exchange. Authorities in Europe and Hong Kong are eying similar rules.
The SEC for years has been studying market structure issues, an effort that intensified after the May 2010 "flash crash" when $1 trillion in shareholder equity was briefly wiped out of the market in a matter of minutes.
The agency has taken a cautious approach, especially because its last overhaul of market rules in the early-to mid-2000s fueled the market fragmentation that gave rise to high-speed trading and dark pools.
Read MoreCramer: Why SEC won't crack down on HFT
If the SEC ultimately proposes a "trade-at" test, it would be incorporated into a broader "tick size" pilot that is being developed to study whether widening the increments, or ticks, at which stocks are priced could incentivize more trading in small and mid-sized companies, the people said.
The idea for a tick-size pilot originated in the 2012 law known as the Jumpstart Our Business Startups (JOBS) Act. It required the SEC to consider a pilot to study whether allowing smaller company stocks to trade in wider increments might help spur more trading. Today, all listed stocks for companies big and small are quoted in one-penny increments - a pricing scheme that has been around since 2001.
Previously, stocks were traded in fractional increments, such as one-sixteenth of a dollar.
While decimal pricing has helped reduce costs for investors in large companies, critics say smaller companies have suffered because brokers have a harder time turning a profit. As such, there is little incentive to produce market research on these stocks, or even to trade them, so they languish and become illiquid.
Exchanges, including the NYSE and Nasdaq, have lobbied the SEC to include a trade-at provision in any such pilot.
Without it, they fear that increasing the quote size will only drive more trades away from the exchanges because spreads will be even higher and dark pools will have a greater incentive to pull in more business, according to two other sources familiar with the exchanges' thinking.
Now, SEC staff are thinking about dividing up the pilot into three groups of similar types of stocks, according to the people familiar with the agency's thinking.
One group would trade with a wider tick size, most likely at nickel increments instead of a penny. Another group would continue to trade in penny increments as they do today. And the third group would trade at the wider tick size and also be subject to the trade-at rule.
SEC staff are also considering whether to exempt orders for mom-and-pop investors from the trade-at requirement, the sources said. That's because these kinds of investors tend to be more worried about getting their trades executed, and less concerned about the price of the stock.
Sang Lee, a co-founder of the market structure consultancy Aite Group, said it's not clear if expanding a trade-at rule beyond a pilot would have a positive impact. "I don't know if just throwing in another set of rules would not just lead to another set of unintended consequences," he said.
Long time coming
The SEC in 2009 floated a plan that would require dark pools to make information about an investor's interest in buying or selling a stock available to the public instead of only sharing it with a select group of investors.
However, the plan has failed to gain traction and has not been formally adopted.
The people familiar with the agency's thinking said the move to include a trade-at test in the pilot has some support among the SEC staff.
Republican SEC Commissioner Michael Piwowar has been among the most vocal agency members pushing for a tick-size pilot.
In February, SEC Chair Mary Jo White also endorsed the idea, saying the pilot would become a reality.
Neither have publicly discussed including a trade-at provision, and it is unclear how the five-member commission would vote on such a proposal.