Credit scores matter. Good scores can make mortgages, auto loans and credit cards—really, your entire financial life—less expensive.

But many consumers still don't have a clear understanding of the impact their scores can have or what they need to do to improve them, according to a new survey by Consumer Federation of America and VantageScore Solutions, which makes credit-scoring models for credit bureaus.

Thirty-nine percent of Americans surveyed by the groups did not correctly identify all three actions that can help a consumer raise a low score or maintain a high one: making loan payments on time, using a credit card and keeping the balance below 25 percent of the credit limit, and not opening several credit card accounts at the same time. Forty-two percent of respondents incorrectly thought their age and marital status were among the factors used to calculate a credit score. And 72 percent did not correctly identify all of the creditors—from credit card issuers to mortgage lenders to landlords—who might use a credit score.

Some aspects of credit score knowledge are "distressingly low," federation Executive Director Stephen Brobeck said on a conference call announcing the results of the fifth annual survey, which was based on 1,007 phone interviews. For example, only 8 percent of people knew that multiple credit inquiries when getting a mortgage or auto loan wouldn't lower one's credit score if they were made within a one- to two-week window of each other.

The groups have updated their credit score quiz website to help consumers improve their understanding of what goes into their scores, and how they are used. You can also take CNBC's credit score quiz to test your knowledge. Just click on the link below.

Quiz: How much do you know about your credit score?

Still, few understand the full financial impact of a low score. Only 20 percent of those surveyed, for example, knew that low credit scores are likely to increase the finance charges on a $20,000, 60-month car loan by more than $5,000. Forty-one percent incorrectly thought that the additional charges would be less than $3,000.

There were some improvements this year over past surveys the groups have conducted, particularly among millennials. More than half of those ages 18 to 34 said they'd obtained their credit scores this year, compared with 44 percent last year. (Many credit card issuers including Citibank, Discover and Capital One, now offer free scores on select cardholders' monthly statements or through online access, and Credit Karma and Quizzle allow free looks at your score and the reports that influence it.)

Fifty-seven percent of millennials also said they'd obtained a free copy of their credit report in this year's survey, compared with 49 percent who said so last year. And nearly three-quarters of millennials said it was important to check their credit reports for accuracy. Credit reports can be downloaded for free once every 12 months from each of the three major credit bureaus at annualcreditreport.com—a step advisors recommend consumers take to stay on top of their credit and spot errors that could affect their scores.

Read More45 million Americans have no credit score

The Consumer Federation of America and VantageScore Solutions note that while there were slight improvements overall compared to last year, there's still a lot of room for improvement. "We know that if consumers are educated about credit scoring, they will feel empowered and are more likely to improve their score," said Barrett Burns, president and CEO of VantageScore Solutions.