Shares of the company have also performed well, rising 136 percent since the start of 2010.
Read MoreThis media company aims to come roaring back
While Derby wagering has held up well, it only accounts for about 20 percent of profits for the weekend. So even if the race is hit by a rainstorm and fewer gamblers put down money, investors probably have little reason to worry.
The real key to the Derby's performance is its status as the Super Bowl of horse racing. The event attracts a similar crowd of the rich and famous for a three-day weekend of revelry in Louisville, Ky., where the local airport becomes crowded with private jets.
Such an audience has been willing to pay top dollar for premium tickets in enclaves with names like "Mansion" and "Millionaires Row." Those sales are usually booked well in advance of Derby Weekend, some several years beforehand. They also contribute to the majority of profits for the event.
Read MoreWhich costs more? Kentucky Derby vs Super Bowl
With so many seats regularly selling out, it's easy for the company to increase revenue by simply adding space and amenities. Churchill Downs is in the process of adding 2,400 seats and upgrading 20,000 in a venue that holds more people than any NFL stadium.
Another steady profit driver for the Derby: media rights. The company recently renewed an exclusive agreement for rights through 2025 with Comcast's NBCUniversal, parent company of CNBC.
While the terms weren't disclosed, Churchill Downs says it negotiated better economics than the previous deal, and there's reason to believe the improvement was substantial. Marquee sports events have become increasingly valuable to networks in recent years because people still tend to watch them live, which is appealing to advertisers.
Read MoreTV execs: Video-on-demand ads worth more than live
The Derby's television audience has remained large and loyal. Over the last five years, each 48-minute telecast averaged about 16 million viewers, according to Nielsen. It also attracts more women than men, which is extremely rare for a major sports event.
Read MoreAt the Kentucky Derby, big hats mean big bucks
Altogether, the 2014 Derby will generate about $75 million of earnings before interest, taxes, depreciation and amortization (EBITDA), Wells Fargo's McKnight estimates. That compares with about $200 million for the entire company.
Another large contributor to profits is Churchill's online division, which contributes about a quarter of total earnings before interest, taxes, depreciation and amortization. It has benefited as gamblers shifted their wagers onto computers and smartphones. Indeed, continued growth in smartphone usage should support that growth further over time, especially as many physical off-track betting venues have shuttered in recent years.
One area of potential concern is Churchill's gaming business, which accounts for about half of the company's profits. It owns casinos in regional locations like Mississippi and Maine, which have struggled to grow in recent years. Many of their core customers remain cash strapped, and the industry appears to simply have too many venues.
Fortunately, that dynamic presents an opportunity for Churchill to roll up other regional operators and find meaningful cost savings. The company will likely have net debt of just 1.2 times EBITDA at the end of 2014, according to consensus estimates, giving it plenty of firepower to go on the hunt.
Churchill isn't cheap, trading at nine times 2014 consensus EBITDA. But with those earnings likely to grow at a double-digit percentage pace, the stock should wind up more lucrative than a typical day at the races.