Harvard, Yale, Stanford and other elite universities may help shape some of the brightest minds in the country, but some of their endowment investment returns can look something less than brilliant.

Harvard, Brown, Cornell, Stanford and Yale all under-performed a classic allocation of 60 percent stocks and 40 percent bonds and even benchmark returns for hundreds of other colleges and universities, according to a new ranking of five-year returns compiled by recruitment firm Charles A. Skorina & Co.

Of course, such an analysis isn't perfect. Comparing complex, multi-billion dollar portfolios that attempt to balance risk management with strong performance is inherently difficult and much of the underlying structure of the endowment investments are kept private.

That doesn't mean the relatively crude analysis isn't interesting.

For example, Harvard's $30 billion endowment—the largest in the country—performed worst among the 12 schools ranked, which included all the Ivy League universities plus MIT, University of Chicago, Notre Dame and Stanford. (See chart above)

One common trait among the various endowments is their use of so-called alternative investments like hedge and private equity funds, real estate and more. Those can diversify and help juice returns, but they can come at a high cost in terms of fees paid to third party managers. In short, they don't always improve returns.

The Cambridge, Mass.-based endowment gained 11.3 percent over the 2012 fiscal year but only 1.7 percent annualized from fiscal year 2009 through June 30, 2013. That's 4.5 percentage points worst than a 60/40 portfolio of stocks and bonds (up 6.2 percent) and 2.1 percentage points worse than preliminary average returns for endowments of more than $1 billion (up 3.8 percent) compiled by the National Association of College and University Business Officers.

(Read more: )

The best was $7.6 billion Columbia, which beat both the classic stock-bond mix and the NACUBO average with its five-year 6.8 percent annualized return.

"While our results for the past fiscal year were excellent, the important story here is that over the past decade, our investment performance has consistently outpaced both the market and our peers, helping Columbia compete academically with other great universities that have far larger endowments," University President Lee Bollinger said in a statement.

(Read more: )

The Skorina report also notes that the head of Harvard's endowment, Jane Mendillo, made the most money of the 12 chief investment officers despite her relatively poor performance.

Mendillo made $5.27 million in 2011, the most recently available data. The top performer, Nirmal Narvekar of Columbia, made the second most: $3.08 million. Cornell's A.J. Edwards made the least at $673,973 in total compensation.

Spokesmen for Yale, Harvard and Cornell declined to comment. The press offices of the other schools in the report did not respond to a request for comment.

—By CNBC's Lawrence Delevingne. Follow him on Twitter @ldelevingne.