Early retiree Chris Reining, who built a $1 million portfolio by age 35, has a similar perspective.
If you want to save big, start with small changes, he tells CNBC Make It: "I know there are some people out there that say you shouldn't worry about the $5 latte, but the more I think about it, cutting out the $5 latte was a good place to start. Because if you try to downsize your house, get rid of all yours cars and make all of these drastic changes, it's so overwhelming and you're not going to do any of it."
After cutting out his morning coffee, Reining eliminated the $15 lunches he bought every day. Next, he cut out the bigger things, such as the $1,000 a month he spent flying airplanes.
"The small changes will lead you to be able to make the big changes," Reining says. Plus, "you can always go back and add the small changes in later. Now I buy a $5 latte and it's no big deal, but I think it was really important to get the ball rolling by starting with something small."
Like this story? Like CNBC Make It on Facebook.
Don't miss: 26-year-old who banked $100,000 in 3 years says these strategies helped him do it