An IMEUC market design (articles this site) provides an effective solution by allowing an accurate aggregation of all customer's willingness to pay matched to all supplier's willingness to provide, balanced in advance of delivery at very short time intervals throughout each day.

Other problems aside from the "Free Rider" inherint in many alternative market designs include the "Verification" problem (how do we know that a customer has actually reduced their consumption in response to a request, and not simply claimed such for a machine not actually running at the time?) and the "Pre-design" problem (a savy building electrical engineer can design a new structure's electrical system specifically to exploit weaknesses in a market design when it doesn't really contribute anything to solving the target problem).

The actual response events could be communicated through a two-way AMI network back to the utility company. But unfortunately utility companies are not planning to exploit this AMI capability because the last thing they want to take on is the daunting task of tracking every customer’s load profile behaviors. They would rather focus on their largest customers like heavy industrial or commercial businesses, who are much fewer in number than residential customers, and more importantly where a single demand response results in a relatively large drop in grid demand as compared to a residential customer response.

However, price-regulated utilities will try hard to advance the strategy, because "the higher they can get their cost base, the more profits they make".

Mainly, I do not see any "free rider" problem. Your critique relies on the notion that the investor in DSM should reap all of the benefit. A major reason for having markets anywhere at all is that self interested actions produce social benefits.

The largest hurdle for "smart grid" technology is not time of use pricing, but the lack of it. Because of current market structures, consumers don't feel high prices. When consumers actually feel them, they will change their behavior. That, and only that, will lead to DSM investment. Some markets, NY and ERCOT in particular, have implemented some of the necessary retail market reforms. Granted, we have a long way to go. Keep in mind that, in addition to capacity, FTRs and ARRs provide revenue streams to fund improvements, whether 'smart' or not.

Your point about subsidies distorting decisions is well taken. Your point about careful financial analysis is also well taken. The very act of investing in DSM and "smart grid" may well eradicate many benefits. However, the power markets are not unique in this regard. Nevertheless, I agree that investment analysis should always consider the consequences of subsidy and market changes.

Transparency is essential to make sound decisions. We have understood for a long time that transparent markets are efficient and give good signals to gold miners, corn farmers and oil drillers. By illuminating power value, investors can arrive at a notion as to what an investment might yield, whether that is generation, DSM or transmission. LMP markets are designed to bind the value of power most tightly to the principle of the "Law of One Price" for a reason, to enhance transparency.

However, the flood of price data from the ISOs appears chaotic and of limited value for making investment decisions. My company, RisQuant Energy, is dedicated to modeling the flood of hourly LMPs into forward curves. Given current fuel curves and historical weather, what should the price and volatility curves for the next thirty-six months look like? Every day, in addition to well known markets, RisQuant models obscure markets like MISO/WAUE (MISO interface to WAPA), MISO/WUMS (Wisconsin/Upper Michigan), ISO-NE/NB (interface to New Brunswick), NY-D, PJM/New Jersey Hub, ERCOT/Houston, forty-five locations and growing. I am starting IESO/HOEP this afternoon.

James Carson, http://RisQuant.com/

The RisQuant company you have there is very fascinating, and sounds like it has much potential for considerable growth in the utility industry.

Your statement "A major reason for having markets anywhere at all is that self interested actions produce social benefits." I would argue that the financial benefits to investors investing in utility company DSM are too tiny compared to the social benefits. The investor benefits need to be far greater before seeing widespread investment in utility DSM.

Granted as consumers feel higher electricity prices down the road, there will generally be growing interest in investing in DSM. I suggest however that growing interest will come much more from residential consumers wanting to better manage their own individual demand, not from investors wanting to invest in utility company DSM programs. As it stands now for the vast majority of utilities under the current regulatory framework, they are only interested in implementing just enough DSM to keep the lights on reliably, or just enough to satisfy push from regulators. This totally excludes any widespread opportunity for individual investment by residential customers in their own demand management.

Presently residential customers have really only one option if they wish to invest in their own demand management: purchase home automation and/or energy monitoring technology that is completely independent of their utility company. And being divorced from the utility company is not what you really want for a future smart grid to work properly.

I agree that 'utility company DSM' is going to remain frustrating and stunted. That is primarily, in my view, the consequence of PUC straight jackets. Until they can quantify the benefits to rate-payers, utilities can't re-capture their investment. However, the benefits from technology investments are notoriously hard to quantify. They tend to be deferred and indirect. The ISOs seem to have a freer hand and at least seem to be more inclined to pursue techology upgrades. NY-ISO & PJM already use DSM for reliability.

My view, however, is that the primary drivers of DSM will be consumers, not utilities at all. Commercial and industrial consumers are already exploring cost reducing alternatives, especially in retail competitive states and provinces. Residential consumers will catch up in time. The primary impediment to this are the retail regimes.

I disagree that utility independent automation and monitoring technology is bad. On the contrary, I believe that an open architecture approach will yield results much more effectively. By publishing uniform standards, entrepreneurial companies can address the needs with innovative ideas. Vast experience in other technology development supports my view. For example, my computer is a hodge-podge of hardware and software from many companies that works just fine --- because of open standards. Would the internet have evolved into what it is today without open standards? No.

James Carson, http://risquant.com/

P.S. If anyone can instruct me regarding how to insert/embed a link....

Example:

James Carson

"Mainly, I do not see any "free rider" problem. Your critique relies on the notion that the investor in DSM should reap all of the benefit. A major reason for having markets anywhere at all is that self interested actions produce social benefits." -- The key to the "free rider problem" is that potential end-user investors in load-management technology and other equipment to manage their peaks to improve the utility's load curve don't see the full payback benefit from their investments and therefore can't cost-justify making as much investment as would benefit the utility, and thus all other customers.

As for the free rider 'problem', we will have to disagree. I think that end users will invest in energy management technology (DSM) if and only if they can reap the benefits of their decisions. They won't if they can't. This requires real time price exposure. They will invest if it is in their individual interest. My view is to let the consumer lead for a change.

James Carson, RisQuant Energy

You and Len do agree on some things like real-time price exposure, which is a crucial part of Len's IMEUC market reform proposal detailed on this website.

My opinion is that real-time price exposure historically is precisely what regulation of consumer prices is attempting to hide from consumers. Policymakers have always believed, I think, that the public could never deal with them efficiently, since who has the time and resources to constantly monitor them hour by hour and act accordingly. This is where I think Len has revolutionary ideas in his IMEUC proposals detailed on this website.

Computer and communications technology through smart meters designed with much more capabilities could easily handle real-time prices, and with slick cleverly designed software for consumers also manage their private energy uses accordingly. I can foresee products being developed under IMEUC that would allow consumers to "set-it-and-forget-it" if combined with smart appliances and Home Automation Networks. It would also permit consumers nice user interfaces on their PCs to occasionally review what their automated system has done for them in the past with detailed logs stored on the PC.

It's nice to dream a bit but I suspect it will be many years, perhaps decades, before we see smart grid deployed everywhere for all consumers to use and benefit from, and before anything like real-time prices are exposed at the consumer level.

Rather than focus on how long it may take for smart grid to be deployed everywhere, I prefer to look at the progress we have made already. Several states have the retail regimes already in place to implement home automation networks that are sensitive to market power prices. Commercial and industrial consumers in quite a few states are already using such technology. I find it interesting that it is the high cost states that are early adopters of competitive retail. PJM, NY-ISO and now ISO-NE are already using demand response resources for reliability and capacity.

James Carson, RisQuant Energy

I confess I tend to be biased towards residential consumer interests, which is what I think about the most in my comments. I also admit some states are actively pursuing the retail regimes to enable HAN networks and demand response programs, and are rolling out smart meters to handle them. And granted there are already some utilities actively providing real-time market pricing to industrial and commercial customers, even here in Ontario, but it is just beginning.

As you point out, consumers will have to drive demand and purchase the in-home devices and smart appliances to use them. This is where there are substantial business opportunities. The industrial and commercial applications are already being dominated by specialized engineering companies to design and set up a factory's demand response systems and monitoring of real-time prices using the internet. Consumers however will not be willing to pay hundreds or thousands of dollars to a third party to come in and set up their HANs and program them, so the in-home products will have to be relatively easy to use out-of-the-box as we say.

James, perhaps someday you can offer a similar type of information service to residential consumers that RisQuant does now. Think of the volume potential to offer a homeowner internet access to RisQuant's predictive data for a small monthly fee. This type of data could prove to be very useful if were input to HAN systems in making automated demand response decisions, or also simply for advising consumers on when the best time in the day or so to recharge their electric vehicles. (I might be just dreaming again, but hey, this is how innovation is created for new businesses.)

I do not believe we have gone far enough to support that conclusion. Thomas may well be right, maybe not. The most important first step, and this is where you and I agree, consumers need to feel the bite of high prices. Then, and only then, will they act. The current regulatory regime, with some important exceptions, blunts price signals deliberately. You advocate an IMEUC market whereas I think simpler LMP driven time of use (hourly?) pricing suffices.

Once an appropriate pricing regime is in place, we will see how consumers invest in DSM. If Thomas is right, we may need to provide incentives and subsidies to 'do the right thing'. However, until TOU pricing has run its course, I believe that incentives and subsidies are pointless. Furthermore, the experience in the commercial and industrial sectors in de-reg retail states is very encouraging. I see lots of interest from the c&i sectors.

Bob: Your idea on supplying analytics to consumers is well taken. C&I consumers are already very interested in quantifying the benefits of DSM investments. The complexity and volatility of the power markets makes it harder than you think. Once we learn on large consumers, we can apply similar concepts and algorithms to the residential sector and mass market them. While RisQuant Energy is focused on providing transparency to the wholesale sector, providing transparency to the C, I & eventually R sectors is definitely on our agenda.

One issue with IMEUC which I am thinking about is the possibility that it places too much emphasis on eliminating "spinning reserve", and rewards that action (accurately predicting consumption in advance) excessively, while under-rewarding peak shifting (responding to [real-time requests from the ISO for peak reductions / real-time price increases via some form of contract addition]). A difficult balancing act. IMEUC provides the tools to adjust the ratio of rewards between those two by adjusting the percentage (proposed 50%, probably too high) paid for unconsumed purchased options and how such collected funds are redistributed, but setting that percentage needs further study. The basis of that error was that at the time I wrote the design, I'd though that the amount paid by the ISO for (spinning and short-response) reserve was much higher than it has actually turned out to be on further study. An endless prospect for study, but at least the results of such study will be applicable accross a wide variety of ISO's.

Any new "market" design (and lets face it, all discussions are at core about market design, admitted or not) needs to properly address a number of such issues explicitly. IMEUC does, no others do.

You should re-visit your ideas about spinning reserve. It has nothing to do with forecasting load. Spin is for emergencies on the generation and transmission side. Prices for spin and reg services have indeed fallen over the past 5-6 years as the market has evolved.

As far as the micro-gen is concerned, I do not know the heat rates and capital costs, so I have formed no opinion yet.

"Smart grid" is largely a waste of money without a "smart market design" to match.

The key roadblock to implementing DR systems on a widespread basis, especially in a timely manner, is exactly the mandate of regulators cited by another article on this website - "State regulators are required by law to ensure that any new energy programs result in just, reasonable and affordable rates,". This is because the cost of implementing DR on a wide scale would result in draconian rate increases that are at least politically unpalatable, and likely far from affordable, unless of course it is rolled out painfully slowly over many years, perhaps even decades. The answer to this dilemma, in my humble opinion, is widespread regulatory reforms that would allow utility companies to make additional income over and above uniform rate base income. This might entail permitting utilities to become the ‘Home Depots’ of energy efficient appliances and new consumer demand response devices. Alternatively it could be simply offering consumers energy data services for an extra fee on the their utility bills where real-time prices or DR requests are fed through their AMI systems to communicate with consumer Home Automation Networks (HANs). Another scheme might be to reward individual consumers with rebates who regularly practice DRs, where each DR is verified by the technology communicating back through the utility's AMI network or the internet to the utility, e.g. reporting that a load has actually been shed by a customer instead of it simply not being turned on before the DR request was issued.

Another futuristic and revolutionary approach to solving this economic problem could be to adopt some form of the Independent Market for Every Utility Customer (IMEUC) proposed on this website by Len Gould. It would foster a public culture of widespread energy efficiency and conservation by including real-time prices exposed to all consumers, and consumer-controlled demand responses in HAN environments, among other things.

Until there are substantial regulatory reforms, utilities have no easy way to fund the widespread deployment of DR technology within their grids to serve all their customers without applying for unpalatable rate increases. And without the grid technology deployed, customers cannot implement any of the technologies within their premises to use it even if they are made aware of its existence elsewhere.

However, who should pay for DR at the wires level is another matter. RTOs & LDCs remain monopolies even in the most competitive retail markets. So, it stands to reason that these monopolies should be mandated to pay for upgrades such that those upgrades will enable the retail markets for energy to evolve towards effective and efficient competition.

One final point, if the whole point of DR is to save energy, especially expensive peaking energy, and thereby presumably save money, how can the net cost of the investment be so large? Shouldn't the savings be greater than the cost???

James Carson, RisQuant Energy

I'm sure everyone wants the savings from DR to be greater than the net cost, why bother if it's not. The critical point you make is "expensive peaking energy", but without exposure to real time prices, customers have no way of knowing when they are happening.

There's also much more to saving energy beyond DR curtailing consumption of expensive peaking energy. Efficiency and sustained conservation are being viewed by governments everywhere as the lowest cost approach by far because in addition to lowering demand at all times including peak times, it avoids building (some) future generation by reducing future demand growth. So even though DR and TOU pricing may today only target peak period reductions, simply getting consumers to participate in them sets up the public for developing a culture of conservation and buying into continuous efficiency improvements, the latter now being forced more and more onto manufacturers of consumer goods by governments.