A patch of New Mexico desert where space dreams begin for Musk, Branson

In the middle of nowhere, New Mexico, is a structure which rises from the scrubby floor of the Southwest desert like something from Tatooine, Luke's home planet in "Star Wars." It's a hangar for spacecraft, complete with an astronaut lounge.

This is Spaceport America, New Mexico, Earth, ZIP code 87654-3210.

Here, someday, paying customers will ride to the edge of space in a Virgin Galactic spacecraft. The spaceport covers 27 square miles of state land, built at a cost of $220 million from New Mexico taxpayers betting on the future. "This is the world's first purpose-built commercial spaceport, the first spaceport anywhere in the world that was designed from the ground up to be a commercial facility," said William Gutman, vice president of aerospace operations.

It's pretty quiet here.

A 12,000 foot runway at Spaceport America.
Jodi Gralnick | CNBC
A 12,000 foot runway at Spaceport America.

Virgin Galactic flights were supposed to begin in 2009, and Richard Branson's company has been paying an annual lease of $1 million for three years. Deadlines kept floating, and with the fatal crash of a pilot in a 2014 Virgin test flight in California, there's no firm date for the first flight.

Elon Musk's SpaceX has also leased space to do vertical test launches, but that area, too, is quiet.

"Virgin has not had a launch yet, neither has SpaceX," said Gutman. "We're ready for both of them, but neither one of them has gotten to that point yet."

It illustrates the slow progress in launching the age of commercial space travel. Twelve years after Paul Allen and Burt Rutan won the $10 million Ansari Xprize, considered a seminal moment in privatizing space, taking people to space on an American craft is still TBD. It seems for every two launches forward, there's one crash back.

However, in the last year the positive momentum has picked up. SpaceX has had a stunning record of successes, especially in recovering boosters. Blue Origin, owned by Amazon's Jeff Bezos, has successfully launched and landed his New Shepard rocket and passenger capsule in tests in Texas. Real estate developer Robert Bigelow was finally able to send one of his inflatable space habitats to the International Space Station for testing, part of his goal to create floating hotels in space.

Down on the ground, the FAA has approved 10 spaceports around the country, but none may have the space and developed infrastructure like the one in New Mexico.

William Gutman, vice president of aerospace operations at Spaceport America.
Jodi Gralnick | CNBC
William Gutman, vice president of aerospace operations at Spaceport America.

In three years of operation, Spaceport America has hosted 28 vertical launches to suborbital space. Boeing is planning a landing test here for its new crew carrier which NASA wants to ferry astronauts to the space station. Authorities at the spaceport have worked with the U.S. Army's White Sands Missile Range to the east for access to airspace not used by commercial airlines. The spaceport has diversified its income, renting itself out as a backdrop for several car commercials and the upcoming Gary Oldman film, "The Space Between Us."

But Spaceport America continues to run on a deficit. Gutman said it costs about $5 million a year to operate the facility, and revenue will cover 75 percent of that this year. Revenue should rise to 90 percent of expenses next year, and the spaceport hopes to be cash flow positive by 2018. That's even without any launches by Virgin Galactic or SpaceX.

Gutman recognizes that some New Mexicans aren't pleased such a huge investment has yet to reap much reward. "Those people on the cutting edge always get pushback," he said. "In five years I'm confident that this is going to be a center of activity."