(John Kemp is a Reuters market analyst. The views expressed are his own)
* Chartbook: http://tmsnrt.rs/2DSvXTy
LONDON, Feb 1 (Reuters) - U.S. crude oil production topped 10 million barrels per day (bpd) in November, according to monthly estimates published by the Energy Information Administration on Wednesday.
Crude output was the highest in 47 years and just 6,000 bpd below the record set in November 1970 (Petroleum Supply Monthly, EIA, Jan. 31).
Production has doubled over the last 10 years, from a low of around 5 million bpd in 2007, reversing decades of decline since the 1970s.
The most recent surge in output confounded some observers, who had forecast production was about to level off, though the EIA had predicted it for some time.
Crude production increased by almost 850,000 bpd in just the three months to November, according to the EIA.
Nearly all the recent surge has come from the lower 48 states excluding federal waters in the Gulf of Mexico (http://tmsnrt.rs/2DSvXTy).
Most of that increase has come from onshore shale plays in Texas (where output has risen by 500,000 bpd since August) and North Dakota (where output is up by more than 100,000 bpd).
Production from the lower 48 excluding the Gulf of Mexico hit almost 7.9 million bpd in November, easily beating the previous peak of nearly 7.8 million bpd in March 2015.
From an output perspective, the U.S. shale sector has fully recovered from the price and output slump that started in 2014 and 2015.
Production has beaten its previous peak even though there are fewer than half the number of rigs drilling for oil compared with before the slump.
Producers have pulled back from marginal areas to the most productive core parts of shale plays such as the Permian in western Texas and eastern New Mexico.
Smaller, older rigs have been idled or replaced by newer, larger and more powerful equipment that can bore through rock faster and further.
Rigs are increasingly drilling multiple wells from a single pad (avoiding downtime for moving in and rigging up) and individual wells have much longer horizontal sections (exposing more rock to each well).
The result is that more oil is being produced with far fewer rigs and drilling teams.
Experience shows that rig counts generally follow changes in benchmark West Texas Intermediate futures prices with a lag of around four months.
In turn, production tends to follow changes in the rig count with a lag of up to six months, to allow for delays in completing wells and hooking them up to gathering systems.
So the total lag from a change in prices to a change in production can be six to 12 months especially once delays in reporting are taken into account.
WTI prices have been on an upward trend since February 2016. The rig count has been rising since June 2016. And lower 48 production has been increasing since December 2016.
The timeline is consistent with previous patterns for cyclical changes in prices, drilling and production.
Understanding the lags in the system between prices, drilling and production is absolutely critical to making sense of the data.
More recently, the rig count has been flat or lower since August 2017, which is why some analysts predicted output would also level off.
But the frenzied drilling activity in the second half of 2016 and especially in the first half of 2017 led to the build-up of a large number of wells that had been drilled but not yet completed.
As more of these wells have been completed and oil started to flow, production has surged even as the rig count has remained flat.
New completions of wells drilled earlier in 2017 should continue boosting production throughout the first half of 2018.
In recent months, WTI prices have climbed higher and the rig count has also started to turn up again, which should boost production in the second half of 2018.
The more oil prices rise, the bigger the eventual response from the shale drillers will be which is why OPEC and its allies need to start planning an exit strategy from production cuts before they become self-defeating.
U.S. oil and gas drilling costs rise as rig count climbs, Reuters, Jan. 30
U.S. shale producers renew challenge to OPEC, Reuters, Nov. 27
OPEC must think about exit strategy, Reuters, Oct. 25
Oil drillers, not forecasters, are responsible for WTI weakness, Reuters, Oct. 2 (Editing by Dale Hudson)