Hidden Markov regime filter
Machine learningAdvanced
A hidden Markov model treats the market as switching among a few unobserved states, often a calm state and a turbulent one, and estimates which state you are in from returns. The trading use is usually a filter, not a standalone signal: take trend trades in the calm state, and stand aside or cut size when the model says volatility has shifted. James Hamilton's 1989 work is the standard economic reference for this kind of switching.
Not investment advice. Nothing here is a recommendation to buy or sell anything. Backtests and historical studies are not guarantees, and a rule that worked in one sample can fail in the next. Trading can lose money. Links are for education, not endorsements. Do your own research.
- Rules
- Fit a two- or three-state Gaussian HMM on a long return history, using only data available at the time. Map states by their variance after the fit, because the labels come out unordered. Allow new long entries only in the low-volatility state, and flatten or tighten risk in the high-volatility state. Refit on a schedule. Decide the number of states before looking at the strategy's profit.
- Where AI fits
- This is already a machine-learning method. The practical help is in the plumbing: a walk-forward fit, a check that states mean the same thing after each refit, and a hard rule that the filter cannot see future returns. An LLM can write the hmmlearn or equivalent code. Choosing three states because the equity curve looked better is still overfitting.
- Markets
- Index futures or a broad ETF as the regime series, then applied to a book of liquid markets. The regime of one stock is a weaker signal.
- Timeframes
- Daily returns. Intraday HMMs need a lot of care around overnight jumps and microstructure noise.
- Risks
- The states are unlabeled, the number of states is a choice, and the filtered probability can jump after a refit. A regime call is often late. A pretty in-sample coloring of past crashes is not evidence the model would have acted in time.