Small business owners of America, the SBA has heard your complaints:

It wants to help pick up the slack from the big Wall Street banks that are ignoring America's entrepreneurs, and it knows that it needs to make government loans easier to get, easier to understand—and less expensive.

"Lamentably, some larger financial institutions have not been doing small business lending, so we have had to become more aggressive," said Maria Contreras Sweet, the head of the Small Business Administration, during an interview at the eMerge Conference in Miami.

Read MoreThe top Latino entrepreneurs in America

Small business is booming and owners are confident. Contreras cited the "dramatic change" in small business confidence which is now at all-time high. She also noted that small businesses are generating two out of three net jobs in the U.S.

"It's a movement; it's a new awakening [spreading] from young girls, millennials, to hard-core entrepreneurs. Everybody's thinking about starting their own business today," the SBA Administrator said.

The following three actions are the "aggressive" steps that the SBA Administrator said the agency has taken to spur the small business economy.

The SBA has recruited more credit unions and alternative online lenders so it can "open up the credit box."

To counter criticism that its loans are expensive there are now no fees on loans of $150,000 or less, which is "pushing more capital out the door."

And the SBA is working to streamline its forms so they are less complicated. She encouraged small business owners to verify these efforts at www.sba.gov.

SBA financing buoyed some of America's most successful companies when they were small, including Apple, Fed Ex, Nike, America Online, Intel, Ben and Jerry's, Tesla, Costco, I-Robot and Under Armour, Contreras Sweet said in additional written comments to CNBC.

Contreras Sweet also highlighted to CNBC five key SBA programs that she believes business owners may be overlooking.

LINC: SBA's online matchmaking service helps entrepreneurs get a date with a lender. Eighty percent of small business loan applications are rejected, according to some industry estimates, and more applications than that are never filed because of the difficulty of getting an appointment with a loan officer. This SBA program requires entrepreneurs to fill out a short form with their small business loan request and answer 20 questions, and interested lenders will contact them within 48 hours.

Free Business Counseling: SBA's business counselors and executive mentors provide free consultation and advice to entrepreneurs across the country, both online and in person. Services are tailored to help veterans, women, Native Americans, millennials and exporters, among others.

Win a Contract: This program helps small businesses learn the best approach for winning government contracts, joining corporate supply chains, and exporting beyond U.S. borders.

Innovation funding: The Small Business Innovation Research program (SBIR) awards $2.5 billion dollars in research grants to small business innovators each year. The Mars rover Curiosity is exploring the planet's surface with systems designed by five different SBIR recipients. The fingerprint scanning technology on iPhones, and 70 percent of the microprocessors inside, were funded through SBIR research. The SBA is one of 11 federal agencies participating in this program.

New Spanish-language web site: More Hispanics are starting businesses than any other immigrant group in America. Hispanic entrepreneurs contribute nearly half a trillion dollars to the nation's economy in each year. For National Small Business Week, SBA is rolling out its new Spanish-language web site. It has translated more than 2,000 pages of content, including information about how to write a business plan, get a loan, or qualify for a contract.