SP500: How well does yesterday predict tomorrow?

Did you ever wonder about how different methods of computing the average daily price would influence your success rate in predicting the subsequent day's largest price movement from the opening price? Did you ever wonder about how often yesterday's prices predict tomorrow's? Well, below is a table of results that show you how yesterday has little to do with predicting tomorrow's price movement.

The table presented uses SP500 index daily price data from July 2002. The total number of days in this study was 358 and the number of successful predictions were tabulated using various simple and logical ideas. You'll quickly notice that the best success rate was only fractionally above 50%, so flipping a coin would have been as good as using any of these ideas. This will surprise many of you because it is assumed that there is some correlation to yesterday's prices.

The average daily price of the SP500 cash index was computed using various commonly used methods. The most commonly used high, low, close average is just one of many methods used to calculate the average price used in the creation of technical indicators. Here in this essay, other methods of computing the daily average are computed such as the open, high, low, close average, and the high and low average which is referred to as the midpoint.

In addtion to these methods a daily average was computed using 400 one minute snapshots of the SP500 cash index named the Mean (which is the proper term for average in statistics). Along with the daily Mean (400 minute average), other statistics were computed such as the median, mode, and skewness for each day. The median represents the middle price, while the mode price represents the most commonly found price, and the skewness represents how different the day's distribution of prices is from the normal distribution. So included in this comparison are some basic statistical quantities that should be more representative of the day's price action. However, as you'll see, these scientific methods of summarizing the day's action didn't increase your odds of success.

In the table below you'll see many logical comparisons, but none of them reached a 51% success rate in predicting tomorrow's prices. Now had this study used the closing price as the reference price, maybe there would be better odds. But in this study, the opening price was used.

The manner in which a successful match was recorded was a simple one. The price movement from the open to the high and the price movement from the open to the low were computed. Now as a trader, you're interested in making the most amount of money each day and there are numerous strategies to achieve success, but this study only focused on one dimension. That dimension was correctly identifying if the market went higher or lower from the open. So this study compared the number of times different methods correctly predicted the direction of the largest range or prices from the opening price. The reason for this is was simple. It would be easy just to examine close to close changes and find the highest correlation to that series but what it wouldn't tell you is how many times you would have to sustain large intraday swings against you. By attempting to find the correct direction in the first place, you would be exposing yourself to less risk and less volatility. This would allow you to set your stops tighter; and presumably you would lose less money.

What is important to understand is that these comparisons don't imply that these methods correctly indentified whether the market closed higher or lower, or if you would have made or lost money. It just tells us if there is any future predictability in using yesterday's prices, and as you'll see, yesterday's prices don't predict tomorrow's despite despite our inclination to believe that it does.

The table below computes the following:

1. The difference between the one minute Median and the one minute Mean. Here the difference between the average price and the middle price was used to predict tomorrow's greatest price move from the open.
2. The difference between the one minute Mode and the one minute Median. The difference between the average price and the most common price of the day was used to predict tomorrow's greatest price move from the open.
3. Today's skewness was used to predict tomorrow's greatest price move from the open. If positive then up. If negative then down.
4. The change in today's Mean price from yesterday's Mean was used to predict tomorrow's greatest price move from the open.
5. The change in today's Median price from yesterday's Median was used to predict tomorrow's greatest price move from the open.
6. The change in today's Mode price from yesterday's Mode was used to predict tomorrow's greatest price move from the open.
7. The change in today's Skewness price from yesterday's Skewness was used to predict tomorrow's greatest price move from the open.
8. The change in today's Close price from yesterday's Close was used to predict tomorrow's greatest price move from the open.
9. The change in today's OHLC average from yesterday's OHLC average was used to predict tomorrow's greatest price move from the open.
10. The change in today's HLC average from yesterday's HLC average was used to predict tomorrow's greatest price move from the open.
11. The change in today's Midpoint from yesterday's Midpoint was used to predict tomorrow's greatest price move from the open.
12. The difference between today's Close from today's Mean was used to predict tomorrow's greatest price move from the open.
13. The difference between today's Close from today's Median was used to predict tomorrow's greatest price move from the open.
14. The difference between today's Close from today's OHLC average was used to predict tomorrow's greatest price move from the open.
15. The difference between today's Close from today's HLC average was used to predict tomorrow's greatest price move from the open.
16. The difference between today's Close from today's Midpoint was used to predict tomorrow's greatest price move from the open.
17. The change in today's High from yesterday's High was used to predict tomorrow's greatest price move from the open.
18. The change in today's Low from yesterday's Low was used to predict tomorrow's greatest price move from the open.
19. The change in today's High from yesterday's High and the change in today's Low from yesterday's Low was used to predict tomorrow's greatest price move from the open. Here's how the four possible outcomes were used:
    1. A higher high and higher low was made - up
    2. A higher high and a lower low was made but the amount higher was greater than the amount lower - up
    3. A lower low and a lower high was made - down
    4. A lower low and a higher high was made but the amount lower was greater than the amount higher - down
20. The difference between today's Close from today's Open was used to predict tomorrow's greatest price move from the open.
21. The change in today's Open from yesterday's Open was used to predict tomorrow's greatest price move from the open.

Summary

It's amazing how random the SP500 market is considering that out of 21 different logical methods, not one one of these methods predicted tomorrow's price more than 50% of the time. It's incredible to believe that no matter how you look at it, yesterday isn't useful in forecasting tomorrow. The fact that each of these methods fails is a testament to the diverse nature in which investors make financial decisions. The fact that there are so many ways to invest; so many different trading systems in existence; and so many different decisions being made each day that it makes it even more remarkable that your odds are only 50:50 in using yesterday's price to predict tomorrow's direction. So flip a coin the next time you base your decision on yesterday's prices.

It appears that investors are making different decisions using different methods,  but if you can't predict tomorrow based on yesterday how can a trend develop. These seemingly random fluctuations are being tamed successfully by some traders but many investors get shredded by these unpredictable price movements. Perhaps these facts will enable you to reconsider selecting any trading strategy based on these methods illustrated above. This implies that as an active trader you'll need to use a strategy that is more complex than using yesterday's prices.

created 12/15/03, The Small Investor's Software Co. ©2003, All rights Reserved.