So what makes a good burger?
The bun is important, said John Callea, a financial advisor for a Wall Street firm. "If it's too soggy or breaks apart, that can ruin everything."
Read MoreA year's supply of burgers for only $144?
Fellow member John Faron, regional vice president at a large ETF provider, says a burger "should not be too tricked up," meaning the toppings should not take away from the patty itself.
The top three burgers for the group have been (in order): the "Black Label" burger at Minetta Tavern, Burger & Barrel's "Bash Burger," and the lamb burger from The Breslin, Puma said.
The group does not cherry pick toppings to customize their burgers. Instead, they opt for the chef's recommendation.
"We want to see what he created," Puma said. "We don't want to monkey with his craft. If he says it's his best burger, that's the one we want to try."
The Chester's hanger steak burger with pimento cheese, miso-Russian dressing, handmade bun and steak fries ranked 7.15 out of 10. That is a good score—but lower than the 8.75 out of 10 at Minetta Tavern.