Some racegoers are even willing to spend up to $1,000 to land the perfect accessory to their outfit.
The iconic horse race takes place the first weekend of May and, when I attended for the first time last year, I wanted to fit in. As a Derby and a headpiece rookie, I headed to a millinery in Downtown Manhattan, to consult an expert: Lisa Shaub. She's been making hats for 30 years and has even worked with celebs like Taylor Swift and Britney Spears.
The first thing Shaub asked me is what color I'd be wearing. I had a white dress, so that left us with an array of options.
Here's her shop on Orchard Street in NYC:
When hat shopping, you want something "that is working with you and not overwhelming you," she told me. That looks different for everyone: "Some people are really tall or really dramatic. Some people are really small with a tiny little face. Different people have different needs, so I make different kinds of shapes for everybody."
The traditional derby hat is a big, wide-brimmed hat with a big flower, but Shaub is known for her fascinators and that's what she recommended I wear. A fascinator, I learned, is a smaller, more lightweight headpiece that fits on the head with an elastic.
Shaub starts working on Derby-specific hats starting in January. While most of her clients set up appointments for custom-made hats, she always has walk-ins the week leading up to the Derby and even on race day.
"Everybody is a procrastinator," she said, and while she'll get calls for derby hats as early as February, "most people are coming in after April 15."
Each hat takes a couple of weeks to make but the extravagant ones are even more time consuming. One of the most complex ones Shaub has made for the Derby was for comedian Tracy Morgan's wife, Megan Morgan, pictured below.
Another memorable Derby hat, said Shaub, is the one she made last year for Maryellen Bonomo, owner of the 2017 Kentucky Derby winner, Always Dreaming. Shaub, who wasn't watching the race, remembered getting a call from her mom right after Always Dreaming clinched the race: "She calls me and says, 'You have to turn on the TV!' And there [Bonomo] is, in my hat, being interviewed!"
The Kentucky Derby is like Shaub's Super Bowl, or "millinery Christmas," as she put it. For some hat shops, the annual event is the backbone of their business. "It's fun to see everyone dressed up. I love to see people being really creative and expressing themselves, so it's nice when people have a license to do that."
And it's inspiring to see how the super rich do the Derby, she said: "People that are really going, they've spent thousands of dollars on their outfit. "
Shaub's pieces range from $95 to $700, and she said the average Derby racegoer will drop $475 on one hat. And those who also attend The Kentucky Oaks, the sister race of the Kentucky Derby held at Churchill Downs on Friday, are often buying two different hats, one for each event.
They're all out of my price range, but I couldn't resist … and I needed something. I shelled out a couple hundred dollars on a piece that Shaub called "modest and sweet." She added: "You're a new hat wearer, so you may feel more daring next time."
Here's what I went with:
I felt pretty daring in the bright pink fascinator, and a bit fiscally irresponsible, but I was also as ready as I ever would be for the Run of the Roses.
And if I ever find myself at the Kentucky Derby again, you can bet I'll be wearing my Lisa Shaub fascinator.
Don't miss: 14 money lessons rich parents teach their kids
Like this story? Subscribe to CNBC Make It on YouTube!