Officially announced in May, the "bond connect" program follows similar stock connect programs launched with Shanghai in 2014 and Shenzhen in 2016.
Those programs were the "game changer" for index company MSCI to add 222 mainland Chinese stocks to its benchmark emerging markets index in an annual review in June. MSCI rejected the shares in the last three reviews.
However, the bond connect could be more significant for global asset managers. While the MSCI Emerging Markets Index is tracked by only $1.6 trillion, Standard Chartered estimates three major bond indexes combined are tracked by more than $4 trillion.
Becky Liu, head of China macro strategy for the firm, expects JPMorgan's Government Bond Index – Emerging Markets (GBI-EM) index to be the first to include mainland Chinese bonds, as early as 2018. "It is likely to be followed by outright inclusion in the Bloomberg-Barclays Global Aggregate Index and Citibank's World Government Bond Index (WGBI) in the next few years," she said in a Thursday report.
Liu expects initial passive inflows of about $20 billion from the JPMorgan index inclusion and about $100 billion to $110 billion each for the Citi and Bloomberg-Barclays indexes.
JPMorgan declined to comment. Citi did not respond to a CNBC request for comment.
"The Bond Connect Program is another good step in the right direction and will help move Chinese securities closer to inclusion in the major bond indices," Steve Berkley, global head of Bloomberg's Indices business, said in an emailed statement to CNBC.
Some of the firms have already taken steps toward adding Chinese bonds to their global indexes. In March, Bloomberg launched two fixed income indexes with yuan-denominated Chinese bonds, and Citi said it will consider including Chinese bonds in three government bond indexes.
In addition, Reuters-owned Tradeweb Markets announced on June 25 it will be the main trading platform for foreign investors on bond connect. Tradeweb has more than 2,000 clients globally, according to a release.
The bond connect also fits into China's long-term effort to attract foreign investment and promote the yuan as an international currency. Also known as the renminbi, or RMB, the currency became one of the International Monetary Fund's reserve currencies in October 2016.
"If you want to internationalize the RMB, then the RMB-denominated assets must be available to investors. Otherwise when they hold the currency, they can only hold cash," Moody's Chung said.
To be sure, Chung noted that the connect is just launching and may have unknown trading issues to resolve. In addition, if the yuan weakens or doesn't strengthen much against the U.S. dollar, foreign investors may be less inclined to buy Chinese bonds.
Foreigners own less than 2 percent of China's total bond market, but are growing investors in China's sovereign bond market — at 3.93 percent at the end of last year, up from 2.62 percent the prior year, according to an HKEX April report.
If foreign access improves, a search for yield could drive inflows to China. Its 10-year government bond yield is near 3.56 percent, more than 100 basis points above the U.S. 10-year Treasury yield of around 2.30 percent.
The bond connect "fundamentally changes the way foreigners can access China's bond market," Gary Lam, analyst at Citigroup Global Markets Asia, told CNBC. "The onshore Chinese bond market can become an attractive [asset] class."