The important decision is to choose only one and stick with it.
"Any one of them could outperform the others, depending on the market environment. But I would warn many investors that attempting to jump back and forth with these is probably going to result in worse long-term returns," Goldberg said.
Most of the major index fund and ETF companies offer this choice between an S&P 500 index fund or ETF and total stock market portfolio, including BlackRock's iShares, Charles Schwab and Fidelity Investments. When Fidelity recently made the decision to offer the industry's first-ever no-fee index fund, it chose its total stock market portfolio, not an S&P 500 one.
The benefit of questioning the S&P 500's ubiquity in portfolios is, ultimately, to reinforce the importance of core, long-term investing anchors at a time when there are thousands of ETFs.
"It's possible to create asset-allocation models that are exactly matched to a particular standard deviation, alpha and other metrics, because the ETF industry has sliced and diced stocks into so many categories, from broad-based indices to niche sectors. But is it worth it for an investor?" Goldberg said. "I think this makes investing overly complicated, which is why a broad-based ETF makes sense as an anchor or core position in a portfolio."
Selecting the one stock fund that best suits an investor's needs and sticking with it over the long term is the first step in creating an asset-allocation plan that also can include satellite holdings to gain diversification that a core holding lacks.
Global investments such as VT are a good way to get low-cost exposure to thousands of developed and emerging markets stocks, but probably are too broad to serve as a core holding for most U.S.-based investors, Mishra said, because of the significant weighting to overseas stocks. She advises between 15 percent to 25 percent of a core portfolio be invested in foreign equities and noted that VT has more than 45 percent of its assets invested in foreign equities, making it slightly aggressive for the core portfolio if held in place of SPY or VTI.
"Given higher volatility and foreign-exchange risks associated with global stocks, VT is more suitable as a satellite holding," she said.
—By Joe D'Allegro, special to CNBC.com