Your asumption that greater open interest leads to greater market volatility stands opposite to most academic research. Notably, Jeff Fleming and Barabra Ostdiek in their 1999 paper, "The Impact of Energy Derivatives on the Crude Oil Market" showed that increased open interest reduces market volatility. Nobel Laurate Milton Friedman famously argued the case for futures speculation in his 1960 paper, "In Defense of Destabililizing Speculation". A wealth of other research shows that speculative traders bring new information about supply and demand to the markets that would otherwise not be available.

Tom: Thank you for bringing my attention to the academic research on the role of speculators, market information, and volatility. I will definitely review the papers that you cited, taking your comments into account as I do so!

My interpretation of the essence of his "Destabilizing Specualtion" paper is that that if destabilising speculation exists, it reflects dissatisfaction with the strategy of buying futures as insurance and instead reflects the willingness to pay for the right to gamble. As long as economists resist the urge to be normative and label gambling as "bad", there is no moral significance to losses incurred in the process of speculating.

Try explaining to a desperate consumer that its the "technicals" driving why they must choose between heating, eating, or buying prescription medications. From a marketers perspective it may be heaven -- to the consumer -- its hell without the heat!!

I applaud the author for trying to make some sense of it for those of us who care about the people it impacts...

The price of a natural gas futures contract is driven by the demand for natural gas futures contracts, not the demand for natural gas. The underlying connection between pricing of physical gas and the contract is from the physical to the paper, not the paper to the physical. That is, the physical buyers and sellers are tying their prices to NYMEX, the NYMEX traders are not tying their bids and offers to physical sales and purchases.

The vast majority of trading of the NYMEX Natural Gas Futures contract is done on technical analysis. That means the underlying commodity could be anything and the supply and demand for the underlying commodity could be anywhere.

The issue for consumers of natural gas is why do they continue to allow their physical product price to be tied to a futures contract that is technically traded and whose price may have no bearing to the underlying physical market forces of supply and demand. Futures contracts have failed in the past and they will fail in the future when they lose their relationship to what physical buyers are actually paying.

The fact is that most commodity buyers and sellers are "price takers", not "price makers" and will, therefore, get prices from some other mechanism than their own decision making. That is why they trade bacon futures (pork bellies) for "price takers" and not Hormel Select Maple Cured Bacon from a "price maker".

You and I finally agree on something.

Lori,

I agree with your point that until techinal traders and speculators find a more profitable market that natural gas will remain volatile. My fear is that the funds will move into the power markets as supply and demand again becomes out of tilt. As the financial contracts for power are barely traded, physical forwards will again be the financial instrument of choice due to liquidity. How many banks can be brought down with another Cinergy June 1998?

Rick: Natural gas futures are physically settled, therefore, futures prices must converge with cash fundamentals. Technicals can drive the day market, but this shoudn't impact the hedger who holds their position until expiration. I encourage you to read, Lester Telser, "Why There Are Organized Futures Markets" International Library of Critical Writings in Economics-Vol. 43, 1995.

Risk of delivery of the underlying is what helps to tie the futures contract to the physical. Without this, we in the market would find it difficult to find enough speculators and hedgers to make trading possible. We are gathering like minded folks under the unifying banner of natural gas trading...both physical and financial. Without this, we are little more than strangers hanging out in a hotel lobby in Las Vegas looking to get lucky.

I think the whole energy industry would suffer if futures hedging was replaced with fixed-price negotiated contracts. Futures provide the most complete picture of supply and demand for any of the traded commodities. Private transactions offer no public information. Even the most ardent opponent of futures trading will still point to the market as the best forecast of forward prices.

EIA Annual Energy Outlook 2003 with Projections to 2025 for one example, has projections which are completely at odds with current NYMEX markets (natural ges to remain very cheap through 2025, much below current trader prices). Their position appears to be well backed by eg. Southeastern Electrical Generators as a group, who project natural gas prices to remain stable around $3/MMBtu through 2020, and are making serious investment decisions on that basis. Who's right? Futures traders pushing contracts over $9/mmbtu? What gives here?

Second, I believe that all of the books and reference material on general commodity trading are irrelevant, except for technical traders. The gas contract is unique among contracts because of its volatility. I know, as I am a culprit in part of this, as I was a member of the Technical Committee that wrote the contract.

I would suggest that you go back to paper indices (NGCH, Inside FERC, NGI - 1987 through 1992)) before the contract was introduced and compare the volatility of the prices before and after the contract introduction. I think that what you will see will show you that there is no physical basis for the volatility that exists today.

The January contract, alone, has varied from $3.52 to $10.04 over the term of the contract. That high is equal to a crude contract high of over $66 or 2 oil over $2.40 or UL gasoline over $2.30 bosed on those contracts' lows.

The gas contract is unique for many reasons, but it is unique.

Energy Development in Africa

"What is remarkable about this region is the enormous quantity of natural gas (associated gas) which is being flared as there does not appear to be a market for it. Of the gross 2.6 billion cubic meters produced in Gabon in 1999, 1.8 billion cubic meters were flared and 0.6 re-injected. In Cameroon 2.1 billion cubic meters, the entire gross production, was flared, as was 0.9 of Equatorial Guinea's 1.1 billion cubic meters. The Democratic Republic of Congo and Congo (Brazzaville) have similar but lower figures. "

My opinion is that volatility is prevalent because the speculators and NYMEX want it to be, not because the physical buyers and sellers want it to be. In 1990, the contract specifications called for maximum daily limits of 10¢/MMBTU ($1000/contract) and no daily limit during the first nearby futures contract. No trading party could hold a speculative position of more than 5000 contracts in all months combined. Now, the contract has a maximum limit of $3.00/MMBTU ($30,000 per contract) for all months. If any contract is traded, bid, or offered at the limit for five minutes, trading is halted for five minutes. When trading resumes, the limit is expanded by $3.00/MMBTU in either direction. This continues if necessary. There is no maximum price fluctuation limit during any one trading session. Further, speculative positions can now be 12000 contracts in any one month for all months. Quite a difference and the intent was purposeful. This move let the investment banker and money manager into the game, with the result to further distance the physical buyer and seller from reality.

We had no problem with price discovery before the futures contract arrived on the scene. That was true price discovery. Even in the early days of the futures contract, price discovery was representative of the market, but only because a lackluster commodity and reasonable trading limits kept pricing in check. Now you have a market which is so hyped up on politics, oil concerns, dwindling supply concerns, technicals, etc. that the natural gas market sits in backwardation just as oil does. Yet we probably should be having the cheapest winter in recent years based on year long fundamentals in regards to natural gas. If everyone believes that natural gas supplies are in jeopardy, explain to me how winter 2005 can be valued higher than winter 2-5 years from now, unless it is not hype premium. What is that “wealth of research” about supply and demand brought to the market by speculators as described by Tom that tells us prices should be lower in the next few years? The LNG speculative savior card is years away. Based on today’s prices, the only thing that gets us closer to lower prices in the next few years is loss of load through loss of industry or through conservation.

Current prices surely don’t reflect where the producer and buyer think it should be. If NYMEX continues to let extreme volatility happen, they will drive buyers and sellers away from the market. Producers (and end users) know this environment does not help them. For the independent producer with variable yearly developmental flow rates and unknown future exploration costs and results, it is difficult to budget participation in a market that will not promise a predictable return. That being the case, supply suffers due to poor drilling activity. If a producer cannot provide long term stability with their product through price to its customer, or stable reserves, they will lose load. Those industries capable of switching to alternate fuels or alternate countries will do so. Stability is the desired product of both parties. Volatility is the product of speculative trade and unwarranted fear.

The market is never wrong, only the completeness of the supply and demand picture it presents. If the volatility persists, participation will increase. Public utility commissions and corporate shareholders will demand more hedging, not less.

There is nothing "wrong" with Nymex natural gas contract. I've spent a good portion of my career in the research department of a major futures exchange. I've written CFTC filings to create new futures markets or to modify existing contracts. The only problem, believe or not, with the natural gas contract, is a lack of speculative interest. Other physical commodity contracts such as cattle, corn and wheat have a much higher percentage of non-commercial participation and subsequently much lower volatility (Note that the Committment of Traders data is available on CFTC website). Why? Becuase speculation increases the flow of supply and demand information into the market. Increased information reduces volatility.

While I respect your opinions, I must firmly disagree. The science is too overwhelming.

First, "Why don't those with superior information step in and correct it.." is a joke, right? That is like saying about the O.J. Simpson trial, "Why don't those with superior information step in and correct it..." I also would note that many do. A great deal of physical gas moves at prices not dictated by NYMEX.

Second, "The market is never wrong.."? That's another joke, right? If it is hever wrong, then what exactly is the CFTC investigating and fining former gas traders over? Obviously, you have information unavailable to people like Alan Greenspan who thought he had seen "irrational exuberance" in a market. Jeesch, if he only was as sophisticated and well educated as the guys as NYMEX! Then he would know that a market is never wrong, irrationally exuberant, over bought, over sold, or anything other than perfect.

Thirdly, if "There is nothing wrong" with the gas contract, then why all of the noise? The volaitlty is not caused by poor liquidity, it is caused by NYMEX rules. If, by NYMEX rule, the prompt month contract couldn't move more than $0.30 per day and the outer months only $0.10 per day, do you honestly think that there would be a volatility issue? I sure as hell don't!

As for "speculation increases the flow of supply and demand information into the market", how can that be? Speculation increases the flow of disinformation into the market. For speculation to exist, both sides have to believe that they know more or better facts than the other. One is clearly wrong and holds disinformation, not information. Isn't it "sell the rumor, buy the fact"?

As for other commodities, want to talk about the perpetual FBI investigation into grain trading in Chicago (COMEX)?

Finally, there is a lot going on here, but most of it isn't science. I studied portfolio theory with a professor who called analysis of commodity trading "reading chicken bones cast on the ground by a witch doctor". As you, yourself, noted, if this stuff was science, why isn't everybody doing it successfully? By definition, the scientific method requires repeatability. If there is science here, then we can predict outcomes from that repeatbility. If we can repeat outcomes, we can consistently make money by being on the right side of every trade. But.......wait, if there is science and it is available to everyone, then everyone would be on the right side of every trade and there would only be one side available! That means that comodity trading requires a fundamental disagreement. The seller thinks the price is too high (how could that be in a perfect market?) and the buyer things that the price is too low (ditto) and .......only one is correct - the proverbial zero sum game. Now, back to making money......all we need is a sucker!

I know alot of these comments sound sarcasticr, but when good, reasonable people consistently raise concerns, the best answer is never that "the science is too overwhelming". My grandfather used to say, "Figures never lie, but liars can figure." The NYMEX gas contract is deeply flawed, as the collaspe of 90% of the gas trading desks in the last ten years shows. Its volatility is 80 times greater than contracts with half of its volume of daily trades. It may never resolve these problems, but anyone who believes that "science" dismisses these concerns is simply whistling past the cemetery.

These discussions are good for all concerned and we may agreed to disagree, but neither side has "science" on its side, only facts, thoughts, and opinions.

You prove my point in mentioning corn, wheat and cattle. All of those contracts have maximum move limits. In the case of corn and wheat that move is currently10% of the value based on the cost of a March contract, and cattle is 3% of the value for the Feb contract (limits of $0.20, $0.30, and $.03 respectively). Natural gas is effectively limitless. They have much lower trading limits and longer cooling-off periods thereby providing greater price stability. I am not certain what the allowed position limits of the contracts are, but I do know that in natural gas, the current position limits subject the market to manipulation. At 12,000 contracts, and current open interest levels, one player could currently control a large share of the market, especially in months past the prompt month. Further, that position could be more because additional positions can be established through banks and participation in other hedge funds. Additionally, the same player could buy any amount of contracts OTC. While we are at it, ACCESS trade is also problematic in that it allows larger players to drive prices higher with little volume traded. That price sets the opening bell the next day. So I too think that if you truly believe what you are saying, you need to get up from your desk and stick your head out the window to see what is going on out there in the real world with the natural gas contract. There is no need for this country’s productivity or the freezing poor in Detroit to be held ransom by the NYMEX.

Concerning the EIA reference for long term natural gas pricing, I spoke with the gentleman that publishes their index about 6 months ago to see where he was coming from. As it turns out, the prices published are a reflection of the price producers need to stay in business and make the historical average margin (I think he quoted 8%). EIA admited that their price forecast had nothing to do with the actual final market price/value. The price dip that occurs is a "reflection of new technology that enhances efficiency", but over time, costs overrun the gains.

You're pointed in the wrong direction (along with a few politicians, disgruntled procurement officers that didn't hedge, and your ill-informed professor). Starting with J.M. Keynes and Holbrook Working, there has been eighty years and countless volumes of research dedicated to the futures markets. Three or four Nobel prizes awarded in the wake. Nearly all of the research disagrees with you.

Since 1970, worldwide participation in futures markets has increased by a multiple of 10,000. CME Eurodollar open interest alone has a notional value in the hundreds of billions of dollars and is the backbone of international banking. The proliferation of mutual funds and 401K investing would be non-existent without stock-index futures. Per capita US commodity costs inversly correlates with the rise in open interest in agricultural commodities since the nineteen sixties. Finally, energy futures has prevented us from the nonsensical energy crisis' of the Carter and Nixon administrations.

The train has long since left the station. If you still don't hear anything, I'd caution you to stop putting your ear to the track and turn around.

Thanks for article Lori !

Donald: That's extremely interesting information which i've never seen before. Seems like quite an important item of data. "price forecast had nothing to do with the actual final market price/value.", given some discussions i've seen here in eg. other articles regarding what "sets the price" of gas now and in future, eg. LNG imports.

A typical transaction with an end user for an MMBtu of gas finds this sort of breakdown: (very general)

1) A producer can enter into a wide variety of deals for its gas, generally with a marketer, utility or pipeline, selling it daily, monthly, in packages, etc. Pricing may be on a daily basis, linked to a citygate published indices, or based on the NYMEX (any time) or typically at some number +or - the NYMEX last day settle. So the producer is getting the essentially 100% of the NYMEX price for the commodity you see traded on the board.

2) The end user typically will deal with a marketer to buy its gas, so the producer deal is generally behind the scenes. Two types of transactions generally occur here. If an end user wants to transport its own gas from the well head, it makes arrangements with a pipeline to get the gas to its facility. These costs are outside the NYMEX transaction, and in the area I work, are roughly in a 10-50 cent range plus the cost of fuel or loss at 3-7%. The end user can also buy his gas at the citygate (utility connect with the pipeline). In doing so, he negotiates a "basis" with the marketer. The basis is also traded OTC. The basis will contain any proportionate share of brokerage fees, broken contract fees, or margin fees that the marketer may have in doing the deal. If the end user deal is locked in on the NYMEX by the marketer, that may mean 100ths to 10ths of cents going to the floor trader per MMBtu for brokerage in the course of the trade, and maybe 1-4 cents/MMBtu to the marketer for margin, which covers his fee, cost of money, and other potential contract purchase costs. Basis is often equated to the cost of pipeline transport from the Henry Hub to the utility, although it may have little to do with it, and is more of a regional cost of obtaining gas. 3) Finally, utility costs vary widely for distribution depending on customer size, location, and utility. Utility charges in our area range roughly from $0.17 to $1.96 per MMBtu. Again this cost is outside the NYMEX transaction.

4) Now, the amount made through speculative trade on an MMBtu is totally based on the daring of the trader and the frequency of trades.

I've enjoyed the banter and the fence looks like it will remain firmly in place. I'll catch Rick's backside and watch for the train coming the other way, although I believe it is on the wrong track. Tom, your exhuberance with the infallability of the NYMEX contract is admirable. You remind me of the people I used to talk to at Enron.

Tom, the question that immediately jumps to mind when I hear statements that futures prices (i) are a forecast of future spot market prices or (ii) reflect the real cost of natural gas is: "On what day?" It takes me back to the corporate budget-setting environment, where trying to base budgets on futures market prices was to me ill-advised unless those prices were being locked in at that moment; otherwise, the whipsaw effect of volatile futures prices was relentless.

With respect to convergence, I have in the past simply compared the cash index for deals done on the day the futures contract expires to the futures contract expiration price. (The logic is that on that day, the prompt-month futures contract is, in essence, converted to a physical contract.) Although inexact due to the month-long delivery of the NYMEX contract volume that you pointed out, the two prices I cited usually converge quite well...with the possible exception of the December 2004 contract. I'd welcome your thoughts.

SPENCER JAKAB

Dow Jones Newswires Sat, Dec. 04, 2004

NEW YORK - The cause of a 22 percent spike in natural-gas prices last week appears to have been a typographical error.

Employees at ANR Pipeline Co., a unit of Houston-based El Paso Corp., said that they were the source of the error that was picked up by the Energy Information Administration in its Nov. 24 report. It was corrected Thursday.

"It was just someone who put in a typo," said Sandy Myers of ANR Pipelines, a large operator of gas storage facilities. "It was just when someone was posting our storage number, and it had a big effect."

The EIA's report, which coincided with the expiration of December options and came ahead of a long weekend when the market was heavily short, had an explosive effect on prices. Traders trying to roll over short positions were forced to buy at any price in the last two and a half hours of trading, incurring tens of millions of dollars in losses and spurring margin calls.

The EIA's policy of only releasing revisions once a week kept the market guessing until Thursday morning if the report was an error or if it showed a bona fide rise in storage demand. Thursday's storage report carried a revision which seems to match an apparent typo still archived on ANR's publicly accessible bulletin board.

EIA spokesman Jonathan Cogan confirmed that the reason for the revision was an erroneous submission. But he wouldn't give any more details, citing the agency's policy of confidentiality in order to encourage compliance in its various surveys of private companies.

Although El Paso could have theoretically benefited from the error, analysts said that an intentional move by the company was almost out of the question.

"We do not believe it was malicious. It was probably just an input error possibly by someone filling in over the Thanksgiving holidays," wrote Kyle Cooper, an energy analyst at Citigroup, in a Thursday afternoon report before ANR's involvement became clear."

I agree that budgeting with unhedged futures prices can be difficult. Conversly, if you didn't use the futures market what would you use. Long term average purchases don't reflect current market condtiions, short-term average purchases are inherently biased by the futures market. I think most boards and auditors woud rather have the futures prices rather than an alternative source.

Dave,

Futures markets are transparent and subject to daily audit. If Enron's special purpose entities were subject to the same transparency, people wouldn't have invested their life savings in a false hope of Enron stock. I sorry to be associated with the name even in jest.

Sorry you took my point the wrong way. What I was inferring was that the people I dealt with at Enron were continually striving to improve their product, and, in the process, never felt they were wrong about anything. They were the trendsetter, the golden boy. I was just laying out the possibility that perhaps it wouldn't be a bad idea to step back and take a look at the futures market in regards to natural gas, and see if the product might be improved. Nothing is perfect. I respect your opinion and learned from the discourse. Thanks.

As a business person, I spent 17 years of my business life in competitive markets before I entered into business relationships with utilities over 10 years ago. I can say with certainty that utilities' management and regulatory commissions believe that the world should be an orderly place where you can plan for five or ten years in advance and then simply execute that plan. I can also state with certainty from my international experience that most of the rest of the world believes that is the case for all business, not just utilities, and that is why the US markets consistently out perform most of the world. If one reads the Economist, you can see ads every week for economists to help plan and shape the future markets of the world. These are the same positions that planned the Thrid World economy five years ago to no great avail. The same is true for utilties, commisioners, and public advocates which seek analysts, planners, and economists to foretell the future for them.

Thus they often look to "science" to plan for the future and forecast results. However, has anyone ever seen this "science"t work in predicting market behavior? Is there a secret algorithm out there that only the few have access to? Have five and ten year forecasts for US electricity demand and/or capacity been accurate? How about five year or ten year price forecasts for natural gas or crude oil? What about "day ahead markets" and other perfectly planned ideas from engineers and economists using "science"?

My point is that all of the number crunching is outstanding at predicting the past and offering explanations (with correlations) of what happened, but, because we are dealing with a world of uncertainties in the future, these studies are usually so flawed as to be valuless. Discussing the past markets can give us some valuable insights into the future, but nothing, nothing, can predict the future and absolutely nothing can model a vibrant and unregulated economy. To say that "but we have to do something" is to waste resources that, if applied to today's markets, would earn a greater return than all of the modelling that could ever be done of those future markets.

As Alfred Sloan said, "It the long run we are all dead."

So, at least from my point of view, this was a great discussion and if even one person came away wondering about how, with these markets that are so vibrant and unpredictable, anyone can make money, then the discussion was all worthwhile. And if even one utility executive or one commissioner came away saying "Nobody can predict this stuff...", then the outcome was even better.

Just saying, "nothing, nothing, can predict the future"