Making Smarter Investing Easy.

Exploring the mechanics behind the behavior:

1/17/07 - Why are SP500 premiums so high? Sorry, I'm not going to answer that question directly because answering it would be pure speculation. I have an opinion and so do you. Everyone has an opinion as to why the stock market is rising, but that would be irrelevant. However it would be more useful and more relevant to determine whether or not futures premiums have any predictive value.

For those of you who don't know, the SP500 premium is the difference between the futures price and the index's price. This fluctuates throughout the day but generally the premium decreases a small amount each day until the the third Thursday of the Quarter (contract expiration day), when it finally reaches zero. So the premium for yesterday, the 59th day until expiration, was 7.8 and it was around 12.3 the day after the December 2006 contract expired on Dec. 15, 2006 (the 15th was the cash settlement date while the last trading day was the 14th). These premium values represent significantly higher values than at any other time since the year 2000. So naturally, the question that arises is whether or not the futures premium is a good predictor of the future? So here's a graph that shows you the price action of the last three years and you be the judge.

Below is a graph that depicts the price action of the SP500 index daily price (not futures prices) relative to the closing price on the 59th day to contract expiration. This graph shows that in 2004, the expiration price was lower than the price in Jan.; the expiration price for 2005, was also lower, but only slightly lower than the Jan. price; and in 2006, the expiration price was higher than the Jan. price. The graph also shows you that prices fluctuated in the Spring and that prices were both lower and higher than the 59th day until expiration. It also shows you that the maximum amount of profit that you could have realized during this period was 19 points in 2004; 35 points in 2005; and 25 points in 2006. While this is good for investors (any profit of any kind), this doesn't correlate so well with the trends in premiums.

What the graph doesn't show you is that the premium on the 59th day-until-expiration was -1.1 in 2004; +1.2 in 2005; +5.2 in 2006; and +7.8 this year. The significance here is that the premium has risen by 2.3 points in 2005, 4.0 points in 2006, and 2.3 points in 2007 over the previous year. However, these facts don't correlate to the maximum obtainable profit. And coincidentally, neither do they correlate to the maximum loss. In 2004, the maximum loss was -30 points; in 2005 it was -21 points; and in 2006 is was -24 points. The point is that the rising trend in futures premium isn't providing investors with any protection from potential losses and it isn't signaling greater gains. But instead, if you look at the end result on expiration day, there does seem to be some kind of correlation. In 2004, the negative premium is correlated to a lower expiration price. In 2005, the positive premium is correlated to a marginal difference, and in 2006, the large premium is correlated to a higher priced expiration. So perhaps this means that in March 2007 we'll see higher prices, but not without first seeing lower prices. Just note that prices in the Spring seem to bounce around quite a bit and that the futures premium is just another tool to gauge sentiment.

1/9/07 - New Futures Volume Data from CFTC.gov. The Commodity Futures Trading Commission (CFTC) in Washington released a new version of the Commitment of Traders (COT) report yesterday - the CIT. For those of you unfamiliar with the COT report, it reports the level of futures and futures option activity by group: non-commercial, and commercial. The new report adds a new class of traders and reclassifies data from the other two traditional groups. The new group is the index trader and this group represents commodity funds traders, who predominately trade agricultural products for mutual and hedge funds.

The purpose of this new CIT (commitment of index traders) report is to provide a better accounting of who's trading what. By adding this new category of traders, the true commercial and non-commercial activity can be ascertained. Prior to this new CIT report, the index traders activities were co-mingled with these other traders, which obfuscated their activities. This added level of disclosure and transparency is welcomed and over time will tell help all investors understand market behavior better.

For those of you that have been tracking COT activity already, be prepared for a big shock. The data that you were accustomed to is now dramatically different. The values of long/short ratios are now radically different since much of the activity in the non-comm and comm categories has been reclassified. This changes the interpretation of the market participant's activitities and reveals some startling differences from what was normal last month. You'll need to recalibrate yourself as become accusomted to the new data. So if you haven't seen this new data yet then visit cfct.gov and become familiar with all of the various versions of data. It is certainly an education you won't want to miss.

For those you unfamiliar with the CFTC's reports. They basically provide five versions. There's the long and short version for Futures only and then there's the long and short version for the combined activities of both Futures and Futures Options. Lastly, there's the new supplemental CIT report which is an expanded form of the combined Futures and Futures Options short version. So these two should used if you want to compare the old series to the new series. BTW, if you wanted to know the option levels, you would need to compute the difference between the combined version and the futures only version. If you wanted to know the difference in activity between front months and back months, then use the long versions. The short version produces only one set of statistics for all contract months.

Let's compare the old report to the new report. The first two graphs compare the old and new raw data. These series are plotted as is without applying any mathematical filtering or alterations. As you can see, the commerical traders have for the past year remained substantially short but there were two periods in which it appeared that they were increasing their long positions (> 150,000 old). However, with the new disaggregated data, these two spikes disappear plus the drop in long positions makes their short position even more dramatic. The relative difference between long and short jumps substantially. Plus notice that the commercial traders didn't increase their long positions during the dramatic rise in wheat prices. This is an alarming fact as traditional wisdom says to follow the industry professionals. These new data seems to indicate that a flaw in this theory may exist, or is this an artificial abnormality or manufactured artifact ?

As for the non-commerical traders, the differences between the old and new data demonstrate again that a substantial portion of the long position was reduced and was related to index trader activities. However, there wasn't much of a difference in the short data series. In this case, the difference in the short data series is trivial.

Next, let's apply one simple mathematical filter. Let's simply difference the data and compute the rolling 9 period sum. Below are two graphs that illustrate the results. Note that these results are based on the same data as those shown above and yet these graphs look quite different. One graph displays the old data and the other displays the new data.

Since it's hard to distinguish any differences between the old and new graphs above, here are two more graphs that compare each trading class separately. So instead of plotting all of the old data on one graph and all of the new data on another, in these graphs, the old and new are displayed simultaneously for commercial and non-commercial traders. With these graphs, you can see that the differences are much smaller than those of the raw data. 1. The crossovers or intersections occur nearly at the same time. 2. The long plots are generally overlapping. 3. The short plots have minor differences but have the same character and directional movement.

In summary the large differences in the absolute trading levels is dramatic, but these large differences vanish in the weekly changes.

Now the next four graphs illustrate a new method to use with these data and manipulate the data to focus on relative changes in the new data. While the graphs above depicted the actual raw trading activity of both commercial and non-commercial traders, the graphs below do not display the actual data. The graphs below focus on showing you deviations from normal activity which is artibrarily defined as the 9 week average. For example, if the commercial traders over the last 9 weeks were short 255,000 contracts then this is normal. What the graph will show is the deviation from the current 9 week average. The reason for this is to identify when the normal level of activity changes. So let's begin with showing the relative changes of the commerical traders, index traders, and the total volume for the Chicago Wheat futures market.

Below is a graph that combines the long and short data to create a ratio. The long/short ratio for the commerical and non-commercial traders, index traders, and the total volume is computed. This is simply the long volume divided by the short volume for each class of traders. The reason for creating these ratios is to compress the data somewhat. Rather than displaying numerous separate graphs for the long and short side for each class of trader, this long/short ratio reduces the number of plots needed to describe the activities within each market. Plus it serves the purpose of demonstrating when these traders change their positions over time.

The problem is that the commerical traders don't dramatically change their positions and the overall total activity for each market doesn't change much either. As the graph illustrates, the plot of these ratios are basically useless or the purpose of market timing, but these ratios do show us the the trading stance of each trading class. As you can see, the commercial traders are heavily short and the overall total market bias is slightly long. But the index traders are overwhelmingly long (use the left scale) and the non-commercial traders depict discernible fluctuations in their positions from short to long. So how is it that tracking the commerical traders trading activity became long standing axiom of futures trading?

Well, the answer lies in understanding the relative changes of these traders. As was shown in the Chicago Wheat market, commercial traders are highly short this market in 2006. This hasn't changed for over a year. But something more subtle than that changes each week and that is related to the rebalancing of their positions. You can see this if you manipulate the data to show it to you.

Below, is a graph that illustrates how the activity for the commerical and non-commerical traders, index traders, and the total volume changes relative to the 9 week average. Just like above, this graph uses the new supplemental CIT data from the CFTC to create three long/short ratios. The long/short ratio for the commerical traders, index traders, and the total volume is computed. This is simply the long volume divided by the short volume for each class of traders. These ratios are then subjected to a 9 week simple moving average (SMA), which are then coverted into statistical Z-scores. This set of computations produces a z-oscillator for each of the series. Below are the results.

As you can see, small differences become magnified and now what first appeared as meaningless and uncorrelated information (previous graph) has now been transformed into something meaningful and worthy of study. The difference here is that the absolute values for the long/short ratios are not displayed. But instead, the deviations from the 9 week SMA are displayed.

The way to interpret this is to first use the left left scale. Zero here represents normal, the 9 week SMA. The other fact that you know is that the ratio is always long/short. So if the ratio increases this means that there are more longs. Conversely if the ratio decreases in value, this means that there are more short positions. So when a relative change occurs, you see the plot move either above or below zero. If the plot rises above zero then you know that the long/short ratio is rising and that it is above normal. Conversely, if the plot is falling then you know that the long/short ratio is dropping and it is below normal. You see, the absolute level of whether the traders are long or short is less important as is their delta. There change in position. So this graph shows you these trader's deltas, or the changes that they are making to their positions.

For example, let's examine Aug 2006 closely. The graph shows us that Wheat prices dropped to a new low from the previous low in June, but instead of becoming more short, commercial traders became relatively more long. And incidentally, so did the overall market. But notice that that the index and the non-commerical traders did not substantially change their positions. Their level of long/short remained below average, which was slightly more "short" than ususal. Basically, they saw lower prices and were positioning themselves for lower prices while the commercial traders had a different reaction to lower prices. Conversely, they didn't add to their already heavily short position, but rather, they lightened up on their short positions. This lightening up of short positions was so dramatic that the overall market's position was strongly above normal, which means that the long/short ratio was higher than it had been for the past 9 weeks and by a wide margin. In summary, lower prices didn't invite a new wave of selling but rather this event was viewed as a buying opportunity and subsequently the wheat prices rallied.

Now let's look at yet another benefit of this new data. Let's compare the activity of the commercial trader to the index trader. The next two graphs illustrate the different behaviors of the two largest market participants in the Chicago Wheat market: the commercial and index traders. These next two graphs are thanks to the new disaggregated data released by the CFTC. Never before were the actions of these two classes of traders divulged before, and these graphs serve to highlight the differences between these two classes of traders.

The two graphs below again focus on the relative changes rather than displaying the absolute changes, which were shown earlier above. Plus in this case, these two graphs don't combine the long and short data into a ratio, which was also shown above in an earlier section. But rather these graphs, show each series separately. In these graphs, the absolute long and short data are converted into a 9 week z-oscillators so that you can see the deltas for each side of the market for both commercial and index traders. And if you want to see the absolute long and short values for the commercial traders, just return to the first graph (the old versus new comparison for commercial traders). As you compare the two graphs, besides noticing how different they look, pay attention to how untradable the raw data is compared to how tradable the relative changes are. This graph below highlights precisely when the long and short activity increases and decreases. Plus it tells you in relative terms how abnormal the situation is. Lastly, if you look carefully, you'll notice that with regards to commercial traders, if they increase activity on one side of the market, they also do the same for the other. The difference is that one side gets just a little more than the other. This slight difference creates a unique oscillator that shows us exactly when to go long and short. If the short side is greater than the long then sell. And if the the long side is greater than the short, then buy. In other words, watch for any crossings or intersections between the two positions. If longs > shorts; buy. If longs < shorts; sell.

So thanks to the new disaggregated data, we can now understand why the old axiom states that we should follow the industry professionals.

Now let's apply the same computations to the index traders and see what their actions divulge. The first item of interest is in comparing this graph to the index traders long/short ratio above. Please look closely. The long/short ratio computed from the raw data shows that throughout the year, these index traders were heavily long. At no time did the ratio fall below 14. This means that these traders held at least 14 times more longs than they did shorts. But the graph below shows negative values for the long series. This is at first confusing, but let's review what this means. Remember that zero represents the 9 week average value, so if the index traders average value for longs is 100,000 contracts than any dip from this average represents a decrease from normal. So a negative value doesn't mean that they are short. It means that the long position is less than normal. And as a matter of fact this graph doesn't tell us what normal is. So we don't have any way of knowing whether the index traders 9 week average is 100,000 or 50 contracts. The point is that this oscillator isn't designed to provide this information, but rather it is designed to show us the devation from normal. So getting back to the long plot of the index traders, we can see that they were not aggressive buyers in August 2006. As a matter of fact, they maintained their normal level of longs and decreased their long positions as prices rose in Sept. and approached the previous high of May. As for their shorting activity, they increased their shorting activity to above normal levels throughout Aug and Sept while prices were rising. But remember, these traders were nearly 40 times more long than short at this time. So all this means is that they were selling into strength and lightening up their position as prices rose. But they at all times remained long.

Despite this analysis, the main point to walk way with here is that the data for this class of traders presented in this fashion doesn't give provide us with any useful market timing strategies. The good news from this analysis is that these traders' activities have been removed from the commercial and non-commercial data so that the interpretation of their activities are more accurate that they were using the old reporting standards.

As you have seen, once you apply the proper filters to the raw data, you can now see why the old axiom is to follow the industry professionals.

1/4/07 - Wall Street's Fear Factor - Are you nervous regarding investing at these high prices?   You aren't alone. As the graph below depicts, anxiety is high despite all of the hype and optimism.

Now the question is how nervous are you? Or better yet, is there a way to gauge the current level of nervousness in the market so that you can compare your level of nervousness with those of other investors? The question is can you determine if you're a "nervous nelly" (overly cautious) or are your apprehensions "in-sync" with everyone else's? These questions don't seem to have an answer, so we set out to look for one.

Below is a graph that attempts to quantity Wall Street's Fear Factor. In essence, this graph tracks a specific class of options for the SP500 index options. The class of options tracked are only the in-the-money options. These are then sorted to produce a put to call ratio, which is different from the overall put to call ratio. The significance of this class of options will become apparent below as the graph poignantly depicts when the level of fear increases.

Below notice when the in-the-money (IM) put/call (P/C) ratio climbs above 4. These events are linked to the state of nervousness in the market. So they highlight periods when investors are scared or nervous. So if you're scared, here's proof that you're not alone and that you're not a "nervous nelly".

Second, these periods when the value is greater than 4 are generally associated with intermediate market lows. So on the bright side prices rally from these lows and temporarily bounce upwards.

Third, what if this is just like all the other instances that occurred in 2005-06, and that this is just another intermediate market low. It implies that prices will be chaotic for the next 3 months. If you study the chart carefully, you'll see that when the IM P/C ratio rises above 4 for the first time, prices are generally lower not higher three months later. So while a bounce is expected, it should be short lived as lower prices are sure to follow.

Fourth, the current level of nervousness is quite high, so expect wild swings in market prices. This implies that volatility will increase and strategies that capitalize on expanding volatility should be employed.

So maybe your instincts are right, you should be scared at this juncture. Just remember that patience is a virtue. It's your money. There's nothing wrong with holding on to it and protecting it from uncertainty. Plus according to recent past behavior, you've got several months before the next leg of the journey will begin.