Spurred by rising global oil prices U.S. drillers learned to tap crude trapped in shale starting in the middle of last decade and brought about a surprising boom that made the U.S. the biggest oil and gas producer in the world. The increase alone in daily U.S. production since 2008—nearly 4.5 million barrels per day—is more than any OPEC country produces other than Saudi Arabia.
But as oil flowed out and revenue poured in, costs weren't the main concern. Drilling in shale, also known as "tight rock," is expensive because the rock must be fractured with high-pressure water and chemicals to get oil to flow. It became more expensive as the drilling frenzy pushed up costs for labor, material, equipment and services. In a dash to get to oil quickly, drillers didn't always take the time to use the best technology to analyze each well.
When oil collapsed from $100 to below $50, once-profitable projects turned into money losers. OPEC added to the pressure by keeping production high, saying it didn't want to lose customers to U.S. shale drillers. OPEC nations can still make good profits at low oil prices because their crude costs $10 or less per barrel to produce.
Now drillers and service companies are laying off tens of thousands of workers, smaller companies are looking for larger, more stable companies to buy them, and fears are rising of widespread loan defaults. OPEC said in a recent report that it expects U.S. production to begin to fall later this year, echoing the prediction of the U.S. Energy Department.
Read MoreWhen to start investing in energy again
To compete, drillers have to find ways to get more oil out of each well, pushing down the cost for each barrel. Experts estimate that shale drillers pull up just 5 percent to 8 percent of the oil in place.
"We're leaving behind a large amount of hydrocarbons, and that's quite unacceptable," Freitag says. "It requires different thinking now."
Engineers have adapted some of the best sensor technology and mathematical models, developed first for deep offshore drilling, to see into the rock better. As they drill, they use imaging technology to find natural cracks in the rock that they can then use as a target when they fracture the rock, to leverage natural highways for oil and gas.
After they fracture the rock, they can map the new cracks. That way they can know how close they can drill another well to target more oil without sapping production from the first well. EOG Resources, one of the pioneers of shale oil drilling, has reduced the space between wells in an area called the Leonard Shale, in Texas, to 560 feet from 1,030 in 2012.
Drillers are finding they can back into wells drilled only a few years ago to re-frack them or inject specially tailored fluids to get oil flowing again. That can return a well in some cases to peak output, without the expense of drilling a new well.
The companies are also getting much faster.