I know what you're thinking: Who's the crowd? What if someone tries to come in with a wild number to skew the results?
There is a reliability algorithm they use every time an estimate is sent in. Say you go onto the site and you are looking at XYZ stock, which currently has a consensus of, say, $1.10. If you put in an estimate at, say, $1.80 and it's the first time you are on the site, it will likely flag you and say, you're estimate is too far outside the trusted range, and is being reviewed manually.
Deutsche Bank concluded: "We found that some of the value-added in the Estimize dataset was due to the 'wisdom of crowds' effect as more predictions give way to greater accuracy."
Read MoreRich get richer: Largest hedge funds suck in cash
This makes sense.Think about it: Apple may have almost 50 analysts (Estimize will have about 250 in the final week leading up to the earnings report), but once you get past the S&P 100 most stocks have a dozen at most, with many at half a dozen or so. That makes it more likely you will get a consensus number that is far from the reported number.
But when you have, say, 30 or 40 reasonably well-informed people contributing estimates instead of 10 or 12, the probability that you can get closer to the reported number increases. It is a simple fact of the law of larger numbers.
Is this enough to get me to use Estimize consensus numbers over, say Thomson Reuters or Factset or S&P Capital IQ, all of whom aggregate sell-side analyst data? Not yet, but I will be watching. I'd like to see more follow-up studies using a larger data base.
Read More Stay far, far away from this country: Dennis Gartman
A lot of money is at stake for anyone who can get the forecasts right! It certainly merits a closer analysis as the database gets larger.
By the way, Wall Street seems to have discovered the value of data mining in general. Recently, another company—TipRanks—was started to gather sell-side analyst estimates, but instead of going to the analyst company's directly, the information comes from sweeping the internet for publicly available information! They have end-run the analysts and their companies, but are still using their numbers!