Why utility bills are not getting cheaper

It seems like all the pieces are in place for utilities to achieve a major IQ increase: Tech companies—from big-betting Google to the more staid Cisco and IBM—are sinking money into technologies that both augment and derive value from the smart grid. President Obama is pushing for new restrictions on carbon emissions from power utilities. And a 2013 incident at a California power substation, in which at least two individuals knocked out 16 of 17 transformers with semiautomatic gunfire, triggered a special Senate panel to review the nation's physical grid security last month.

Consumers want a smarter, more secure grid. Technology companies want a smarter, more secure grid. Lawmakers (at least ostensibly) want a smarter, more secure grid. Utilities could benefit from a smarter, more secure grid. So why isn't the grid, on the whole, getting a whole lot smarter?

It all comes down to money, but it doesn't stop at the fact that no one—not consumers, Congress or the utilities—wants to pay for sweeping upgrades to America's aging power grid. Institutionally, the American system of regulating utilities is stacked against such improvements, even when there is a clear and pressing need for them. The interests involved are way out of alignment; the incentives for utilities to pursue such upgrades are limited. It's not so much that utilities dislike the idea of a smarter grid, but the current system isn't friendly toward pursuing one.

Read MoreBiggest risk to utilities: You going solar

So who is left holding the bag? The consumer. While the technology and public will are seemingly in place, institutional inertia and disincentives to integrate new technologies mean consumers won't be getting the smart grid they want anytime soon. And while they're certainly paying to incrementally upgrade their power grids, they're doing so in a way that's as painfully inefficient as the grid itself.

"Very seldom do they provide services in the most efficient manner; they're usually provided in a very inefficient manner," Jon Wellinghoff, the immediate past chairman of the Federal Energy Regulatory Commission, said of the nation's utilities. "It's difficult, given the structure that we have to accomplish what we want to accomplish and provide low-cost, efficient services for consumers."

The system that feeds the grid's current inefficiency was originally devised to create order within a market-based grid ecosystem so fiercely competitive that America's rapidly industrializing and modernizing cities couldn't develop infrastructure in a coherent, efficient way. (Imagine a dozen electric companies all trying to bring customers onto proprietary grids one house at at time.)

To solve the problem, the government replaced free-market practices with heavily regulated monopolistic ones—one power company per geographic area would agree to serve all customers at rates overseen by public utility commissions (PUCs) that would stand in for free-market competition, ensuring fair pricing. It worked. The country was electrified, and the grid as we know it was born.

But regulators haven't always been able replicate the role of competition in the marketplace, said Karl Rábago, executive director of Pace University's Energy and Climate Center and a former public utility commissioner in Texas. "Regulators have the responsibility to drive more innovation than the monopoly would on its own," Rábago said, but they don't always do so, especially considering their other role: to keep utilities from overbuilding (big investments that don't pan out cause most major utility bankruptcies).

Utilities are entitled by law to receive return on investment from capital improvements to their infrastructure, and typically such investments result in a rate increase—the cost gets passed along to the consumer, including a small profit margin for the utility. To increase rates, utilities need the blessing of their local PUCs, which have to worry about utilities gold-plating the grid—making unneeded or incremental capital improvements that don't deliver much value—simply to increase their capital accounts. Utilities in turn have to worry about recovering the costs of any improvement. Most of this process is so arcane that consumers can't (or can't be bothered to) understand how the money they pay into the utility is being spent.

"In an optimal situation, a utility wants to do something in the public interest—a proposal is approved, the cost is built into rates, technology improves efficiency, the capital earns a return that goes to the bottom line for further reinvestment in the system ... In a good regulatory environment, this works," said Richard Sedano, principal and U.S. programs director for the Regulatory Assistance Project, a nonprofit team of industry experts focused on better aligning public interests with those of utilities and regulators. "But for all kinds of reasons, there are many states in which the clockwork is not reliable."

All that adds up to a situation in which utilities feel much more comfortable feeding their bottom lines with large capital improvements with reliable records of boosting profits. It's easier to make the rate case to PUCs, and it is a safer bet for the utility.

"Most traditional companies are oriented toward making very large investments in new kinds of infrastructure that they see as an asset that will lift substantially in their bottom lines," Wellinghoff said. "They're not interested in making incremental investments in efficiency."

All that, coupled with the fact that they have no real market competition, means utilities are extremely risk-averse, and their unwillingness to take risks feeds the inertia. They want to know that they are going to recover their costs via a rate increase before they go out on a limb investing in a new technology.

On the other side, regulators typically are skeptical of new technologies, making it difficult for utilities to successfully lobby them for a rate increase. Even assuming that consumers wouldn't mind tacking some pennies onto their monthly bills to help build out the smart grid, utilities have a difficult time getting regulators on board.

"There are limited things you can do by going halfway; there are some things you can only do in one big leap," Rábago said. "And it's tough for the regulators."

Without that vote of confidence from regulators, who are often unfamiliar with new technologies, utilities shy away from smart-grid upgrades or enhancements to cyber or physical grid security and instead fall back on what's worked in the past: major capital investments in new power plants or substations. "Not taking risks means not moving ahead with potentially valuable additions to the grid," Sedano said.

Read MoreA $15 trillion dream of a GE-Silicon Valley hybrid

There's also the fact that utilities make money by selling electricity, and reducing their sales volume isn't necessarily in their best interests. A rate increase based on a capital investment in grid infrastructure is good for the bottom line, but there's a built-in disincentive to take efficiency improvements too far.

For instance, a utility might be considering upgrading its meters, with a choice between automated meter reading (AMR) and advanced metering interface (AMI) technologies. The former allows a meter to beam its data to a passing truck, simplifying the job of meter reading. The latter consists of technologies like Google's recently acquired Nest smart meter, technology that opens two-way communication between the consumer and the utility. While AMR is less expensive, AMI offers better utility and efficiency for the consumer, ultimately offering customers more control over their energy usage (and energy bill).

Although AMI is in most cases the better long-term option, a utility often has stronger incentive to go with AMR, Rábago said. Aside from being less expensive, a last-generation technology locks other companies—technology-driven concerns like Google, Cisco, or IBM that work at the cutting edge of smart-grid technology—out of the ecosystem.

Integrating technologies a generation behind cutting-edge can erect a barrier to competition, keeping customers tied to services offered by the utility itself. And because it's less expensive, the utility can do so under the guise of responsible frugality, keeping rate increases minimal and electricity consumption even.

"At a progressive utility, they may say, 'We're willing to take the hit, but we have a responsibility to shareholders. We're willing to do this at no growth, but we need an earnings opportunity,'" Rábago said. "And that's often where things derail. What ends up driving the conversation: 'How will this hurt sales of electricity?'"

Consumers hate rate increases. Lawmakers don't want their names associated with rate increases. The only party that likes rate increases is a utility, and even utilities are wary of them (putting a rate case before a PUC is often an uphill and expensive legal battle).

"Many legislatures are silent about this; they leave it to the commission to figure out, and rates become the thing that drive these conversations," Sedano said. "'If you raise your rates, I'm going to be mad at you.' And that can be the way something is decided."

However, some states are moving in the right direction at the legislative level, working to institute smarter, more secure 21st-century grids by smoothing out the inefficiencies and incongruities that currently permeate the regulatory system.

"What sometimes needs to happen to break the logjam is for state legislatures to pass laws," Sedano said. "They have to say, 'We really want the smart grid.'"

Read MoreShale boom won't budge utility bills—or stocks

Some PUCs, like those in Massachusetts, California and New York, are testing a new kind of relationship among the grid, utilities and the consumer. It's creating significant changes, from the way rates are set to the way utilities are compensated. For instance, new institutions, like California's Electric Rate Adjustment Mechanism, act in various ways to decouple revenues from sales volumes, allowing utilities to collect on fixed costs not just when they sell a lot of electricity but also when they save it.

These piecemeal improvements indicate that there is a path forward, Sedano said, and that regulators, utilities and lawmakers are increasingly cognizant there's a general public interest in moving toward a technology-driven grid. It's just going to be a while before the consumer and the taxpayer—the one paying for all of this—is going to reap the rewards.

"Change is coming," Rábago said. "I meet an awful lot of people right below the C-suite who want to figure out how to tackle this stuff, and they'll be driving it in a few years."

—By Clay Dillow, special to CNBC.com