Understanding total cost
David Mazza, head of research for State Street Global Advisors' SPDR ETFs—which offers some of the largest, lowest-cost ETFs in the market—said that while an ETF may look cheaper on paper or at first glance, it could be a lot more expensive when it comes time to trade it.
ETFs trade on an exchange, and investors must pay a commission if not using one of the newer commission-free discount brokerage trading accounts, as well as a bid-ask spread. (The bid is the price that someone is willing to pay for a security at a specific point in time; the ask is the price at which someone is willing to sell. The difference between the two prices is called the bid-ask spread.) The lighter the volume in a particular ETF, the higher the potential for a most costly bid-ask spread.
"If you hone in on one particular measure, such as expense ratio, when doing due diligence and homework, you may miss out on the opportunity to have a less expensive total-cost-of-ownership experience," Mazza said.
Another component ETF investors should be aware of is tracking error, which is often overlooked but should not be, since most ETFs are passively managed and seeking to track the performance of a benchmark. "A fund at first glance may be very inexpensive, but it may be quite small, so the portfolio managers can not buy all the securities the index represents, and that could introduce tracking error," Mazza said.
(Read more: Fitting a star fund manager inside an ETF)
Dave Nadig, chief investment officer at ETF.com, provided an example of how even index ETFs covering the same sector can diverge to a significant degree. In 2013 the best-performing U.S. financials ETF was the RevenueShares Financials ETF (RWW), which returned 42.04 percent, while the worst performer was the Powershares KBW High Dividend Yield Financials ETF (KBWD), with a return of 18.35 percent. That's a whopping 23.69 percent difference.
"No one can tell you exactly which portfolio will outperform, but you can't assume you are making the right choice by just looking for the one with the cheapest expense ratio," Nadig said. RevenueShares ETFs are weighted based on top-line revenue. The Powershares KBW High Dividend Yield Financials ETF is weighted based on dividend yield.
Buying the biggest or most liquid ETF is no panacea either. Nadig used an example from one of the biggest ETF providers, the SPDRs, to make his point. The largest financial sector ETF, with roughly $17 billion in assets, is the Financial Select Sector SPDR Fund (XLF). It saw $4 billion in flows last year alone, but it has consistently underperformed the $1.3 billion iShares Dow Jones U.S. Financials ETF (IYF), which has less assets and lower trading volume, according to Morningstar data.
(Read more: It's a good time to be an ETF bad boy)
For the three years ended on Dec. 31, the cumulative return for IYF was 46.5 percent, while the XLF came in at 44 percent. "Of course, time periods matter here, and you can find windows where the opposite is true," Nadig said. But bigger is not always better. "I don't believe the biggest funds always provide the best performance. It is all about strategy and not about fund size," said Emerging Global Advisors' Holderith.
The SPDR financial ETF tracks the S&P Financial Select Sector Index, while the iShares financials ETF uses the Dow Jones U.S. Financials as its benchmark.
Nadig said he wouldn't suggest that IYF will always outperform—recently it hasn't—but the difference in their portfolios is primarily in diversity: IYF has 266 stocks compared to XLF's 80, which pull just from the S&P 500. The iShares financials ETF has a significantly lower average market cap as a result: $84 billion vs. $103 billion. IYF also skews a bit "growthier", Nadig said, with a higher P/E and has some slight sector skews: it underweights banks and insurers a bit in favor of REITs. But IYF is the ETF.com pick largely because of its significantly broader portfolio.
There are big notable among the top 10 holdings of the indexes—the Dow Jones index includes Visa and MasterCard among its top 10 holdings, while the S&P Financial Select Sector Index does not.