3. Making the investments too complicated
They don't have to be. You can put your money into an age-based plan, which, like a target-date fund, will automatically adjust the investments. Age-based portfolios typically are designed to shift to all cash as high school graduation day approaches. Fully half of the $50 billion Vanguard holds in 529 plans is in age-based plans, and that's where most of the growth is coming, Bruno said. If you want to be a more active investor in your 529 plan, keep in mind that you are allowed by federal law only to buy and sell within the plan once a year.
Read MoreCollege costs are soaring—but not all aid is
4. Not understanding just how much fees can cost you
Some state plans are sold by brokers and can have upfront commissions of 4 percent to 5 percent, Glassman said. If you want to use a broker-sold plan in your state, perhaps for the income-tax deduction, you should investigate whether a company with which you're affiliated, or an LLC you can form, can establish a plan without paying the commission, Glassman advised.
Investments within the plan can have damaging fees, too: Nordlicht analyzed two portfolios for CNBC, using an initial investment of $14,000 (the maximum allowed per person per child under federal law) and then not a penny more for 18 years. With an annual return assumption of 7 percent, an age-based index fund plan with inexpensive underlying mutual funds had a balance of $45,636.53 at age 18; an age-based actively managed fund had a balance of $37,014.88. That's more than $8,000 worth of books!
These numbers also make another good point: If you think that a one-time investment into a 529 plan at birth, even to the max contribution, is going to come close to the hundreds of thousands of dollars needed for college if you don't keep adding year after year, think again. This hypothetical exercise was good for roughly one year at a private university at today's prices.
5. Overestimating on student expenses
Clients often overestimate how much students will need for college bills. "Maybe some expenses don't materialize," Nordlicht said. "Or your child decides to live off campus—the IRS can question the excess."
The cleanest distribution: Wait until you have the bills in hand, then make a check out in the student's name—and make sure the student in question pays the bills.
And remember, assets in a 529 plan can be changed to a different beneficiary and the money isn't considered the child's money for federal financial aid purposes.