The BATS talks follow the SEC's broad review of equity markets, which has resulted in investigations into whether exchanges and traders are following the rules, and not giving advantages to certain clients. One line of inquiry has concerned the proliferation of order types that exchanges offer to their clients, including high-frequency traders.
The SEC is investigating several exchanges, alternative trading systems, and anonymous dark pool trading venues, including one operated by Barclays. Other cases or settlements could follow this year, a person familiar with the matter said.
More from the Financial Times:
William O'Brien departs BATS with immediate effect
BATS-Direct Edge investors to split $235m
BATS-Direct Edge merger creates NYSE rival
The BATS settlement talks were first reported by the Wall Street Journal, and came to light just two weeks after the exchange group's chief executive, Bill O'Brien, abruptly left the company. He had joined BATS as part of its merger with Direct Edge, where he was chief executive. Mr O'Brien is not expected to face any personal liability from the investigation, a person close to the matter said.
BATS and the SEC declined to comment.
Read MoreHigh-frequency trading sees a planet divided
At the time, BATS did not give any reason for Mr O'Brien's departure. It followed a period of intense scrutiny of the industry from regulators, law enforcement officials and lawmakers following the publication of Michael Lewis's book Flash Boys, which alleged the markets were rigged.
After the publication of the book, Mr O'Brien was involved in a heated and widely watched exchange on CNBC with Mr Lewis and Brad Katsuyama, chief executive of the IEX equity trading venue, which is featured in Flash Boys as a crusader against high-frequency traders.
BATS later corrected comments by Mr O'Brien regarding the way certain BATS exchanges price trades.