Will buying coffee every day keep you from getting rich? Here's what Kevin O'Leary and other experts say

Coffee isn't always worth the cost, experts say–here's why

If you're like many Americans, you enjoy a nice cup of coffee each morning before work. And if you're like me and you buy that coffee instead of brewing drinks at home, the habit can get pretty costly.

When I crunched the numbers, I realized I spent more than $2,300 this year at Starbucks.

Some experts think that's a problem. But not all of them do. Here's a round up of three — "Shark Tank" investor Kevin O'Leary, wealth manager and bestselling author David Bach and self-made millionaire Ramit Sethi — on the pros and cons of spending your money on daily coffee.

'That is such a waste of money'

O'Leary is not a fan of buying coffee every day. He prefers to make it at home. "Do I pay $2.50 for a coffee? Never do I do that," he tells CNBC Make It. "That is such a waste of money for something that costs 20 cents. I never buy a frape-latte-blah-blah-blah-woof-woof-woof."

He still drinks one cup every morning: "It costs 18 cents to make it and I invest the rest." Before making any purchase, in fact, he asks himself, "Do I really need this?" Because, he says, "if I don't buy it, the money is going to be invested and make money every year while I'm sleeping."

Why Kevin O'Leary refuses to spend his money on fancy coffees

Bach agrees: "We all throw away too much hard-earned money on unnecessary 'little' expenditures without realizing they add up," he writes in "The Automatic Millionaire."

If you ditch a $5 latte, or another small luxury, like fast food or soft drinks, he explains, you'll have a considerable amount of money to work with.

I ran the numbers on my own coffee habit and was shocked by what I found. If I skipped a year of Starbucks and invested the money instead, with compound interest, it could grow to $4,551 in 10 years assuming a 7 percent annual return, according to CNBC calculations. In 20 years, it could hit $8,952 and, in 30 years, it could reach a whopping $17,611.

That's a lot of money for not much effort.

Ultimately, Bach writes, "you've got to find something you can give up to get up."

CNBC | Shawn M. Carter

But 'there's a limit to how much you can cut'

Not all money experts agree with O'Leary and Bach, though. Sethi argues that "you can't cut your way to growth," and gives the example of the way millennials supposedly overspend on avocado toast.

"You may have seen an article going around telling you the only way to be a millionaire is to cut back on avocado toast," he tells CNBC Make It. But the idea of getting rich by buying less, he says, is "horrible" advice.

"Do you know how many avocado toasts you'd have to forego to get just a 20 percent down payment" on a median-priced home? "Is it 100? Is it 1,000? No, it's over 2,500 avocado toasts."

In other words, eliminating small, regular purchases is unlikely to help you save enough money to become wealthy. "There's a limit to how much you can cut," Sethi says, "but there is no limit to how much you can earn. You've got your salary. Negotiate it.

"Think about how much you can grow versus how much you can protect, and that is how you start crafting your own rich life."

Self-made millionaire: Cutting out avocado toast and lattes won't make you rich—here's what will

Everyone agrees: Be responsible

Whatever your stance on coffee, it's useful to track your spending so that you know where your money is going. If you're buying coffee every day, be sure you can actually afford to because you've already covered other vital expenses, like monthly student-loan or credit-card payments. That's what I do, and it's why I feel comfortable continuing to buy coffee: It makes me happy, and I can swing it.

In addition, Sethi recommends automatically putting money aside for savings and investing so you can stay on track with your long-term financial goals. Try to dedicate 5 to 10 percent of your take-home pay to savings and invest another 10 percent. he says.

If you can't quite make 10 percent, don't fret: Save whatever you can, even if that's "5 percent or 1 percent," says Tom Corley, author of "Rich Kids: How to Raise Our Children to Be Happy and Successful in Life."

"The point is to get into the habit of saving money," he says. "You can always increase the percentage of savings down the road."

Once you've taken care of what's important, Sethi says, you can relax a bit and enjoy yourself. "Everywhere you turn, you hear people telling you what you can't do with your money: No lattes, no jeans, no vacations," he says. "While everybody else is worrying about how little they can spend and how much they can cut back on, you know you've automated your investing."

Like this story? Subscribe to CNBC Make It on YouTube!

Don't miss: My daily Starbucks habit costs me $2,300 a year—here's why I refuse to stop

Video by Jonathan Fazio

Self-made millionaire: Don't buy a house before you ask yourself this question