But most importantly, the 1904 St. Louis Olympics featured the fewest number of athletes: It's scarcity value that makes this medal the most valuable of all, Greensfelder said. (Among medals without a particular Olympic personality attached to them: e.g., Jesse Owens, "Miracle on Ice" U.S. hockey team star Mike Eruzione.)
Bill Mallon, an orthopedic surgeon and former professional golfer who is also one of the world's foremost Olympic historians, pointed out that the first Winter Games in 1924 had only 17 events. This year in Sochi there are 98. The 1896 Summer Games had 43 events. There were 302 events in the London Games of 2012.
The collector's cheat sheet
It's easy enough to look past the precious metals chart and get a sense for how to value an Olympic medal as collectible, in addition to the scarcity value.
(Read more: Socio-economic data claims the gold)
No. 1: What condition is it in? An Olympic medal that has been worn or that an athlete let their kids play with, "We call that a burial piece," Greensfelder said.
No. 2: Does it come in the original presentation and with all the original components? This is a fact that is lost on most people, Greensfelder said: Medals only started being awarded with the ribbon around the neck in 1960, but that adds value, as does the decorative box or wrapping that each medal, both winner and participation versions, come in. "It's like a Matchbox Car," Greensfelder said. "In the box versus out of it makes a difference; the box can be as valuable as the medal ... Lilehammer was a mark of excellence in presentation."
The pawn shop
Still, the exponential growth in the number of Olympic events through the modern era hasn't resulted in a glut in recent years. Fewer medals make it to market now than in previous Olympics.
When O'Neil first started to see Olympic medals showing up at coin shows, she didn't know it at the time but it was a byproduct of Soviet bloc economic desperation. "They wanted to go home with U.S. dollars," O'Neil said.
At Lake Placid, Lithuanian bobsledders brought pillowcases with stuff to sell for cash, and it was the same in Calgary, Greensfelder remembered. "If you could get around the Russian handlers, they wanted to sell stuff."
"That's all changed," O'Neil said. "There are no medals coming out of Eastern European countries anymore."
Now, "It's tip money for them," Mallon said.
For the countries still not well off, they also tend to lack athletes winning medals. In the last 10 years, most winners' medals that have gone into collections were from Communist countries with a history of athletic achievement that did not transition successfully to capitalism after the fall of the Soviet Union. A $5,000 sale of a medal can make a person relatively wealthy in a country like Cuba.
(Read more: Russian Olympic broadcasters have a way with words (about Americans)
About a decade ago, the sea change in availability of medals dawned on Greensfelder in Norway: Russians were there with tons of money buying, not selling. Today, there are a significant number of deep-pocketed Olympic memorabilia collectors in Russia.
It's considered an exception if a medal from recent games comes to market, and if any new winners medals come up, the values are going up, she said.
"It's almost always in the families, and it only gets out of the families when people need the money," Mallon said. Greensfelder, who serves as a medal authentication expert for auctions—including verifying that the Jesse Owens 1936 gold which sold for the highest price ever actually came from the 1936 games—said, "In bankruptcy cases, I hear from their attorneys."
And then there are the items that even Olympic collectible experts dream about, but doubt they will ever see: the winners medals from the first U.S Olympic men's basketball Dream Team, or the torch Muhammad Ali used to light the Atlanta Summer Games flame.
The closest O'Neill came? "Ali signed some of the Atlanta Olympic relay torches and those sell for $5,000 to $10,000. He can't sign anything anymore and since he lit the cauldron in Atlanta, people pay special attention."
Olympic torches follow the classic law of supply and demand. There were 16,000 torches in Atlanta. And a total of 14,000 torches were made for Sochi. For the Helsinki 1952 Summer Olympics, by comparison, there were a few dozen torches in the relay, according to O'Neil, who says one sold for approximately $742,000 two years ago.
"There are quite a number of torch collectors, but the Sochi torch is not something to invest in, for sure. You will always find a Sochi torch in the future."