A Review of Price Expectations Prediction Methods
Making increasingly more accurate future stock market price predictions has long been a master objective of portfolio managers, fundamental investors, exchanges, and traders alike. While many long-term investors and short-term traders have somewhat different goals for making price predictions and tend to prefer other analytical methods, both have strategies and models that depend heavily on forecasting returns and making accurate price expectations predictions. Having good price information is essential, but it is less critical than some of the applications derived from price expectation information. No price expectations ever are perfect because markets are very complex adaptive, and evolutionary, chaotic systems that are by nature impossible to predict and have some perpetually novel, not repeatable forces at play.
Price expectations and the variance-covariance matrix plays a critical role in modern portfolio optimization, which minimizes price covariance risk. The variance of the portfolio is not a simple linear combination of the variance of each stock or asset. Still, it accounts for the covariance of assets which can offset each other's risks and losses to some degree. This is why stock diversification is an essential mainstay of modern-day finance. Price expectations are a critical input to this risk-reward optimization process and why reviewing modern methods of price expectations forecasting is part of the foundations for financial industry and portfolio managers.
Literature Review
Fundamental Pricing (Christie & Isidore, 2018):
Stocks for publicly traded companies in equity can be very complex for investors and traders alike (Christie & Isidore, 2018). In their comparative review, Fundamental Analysis Versus Technical Analysis, Christie & Isidore examine the sets of tools used by investors and traders called fundamental analysis (FA) and technical analysis (TA). These two tools are used to determine valuations and pricing expectations of equities in global equities markets. The review explores the costs and benefits of both FA and TA.
FA is the application of "economic analysis, industry analysis, and company analysis" in determining the value of a stock and its appropriateness to add to a portfolio of investments. FA economy analysis examines systemic economic information for determining the systemic, industrial, and company-level exposure to economic conditions and how it will drive capital availability and other conditions for the publicly traded company. FA industry analysis looks at how the conditions for the industry of a given company are affected by economic drivers, government policies, technologies, and market forces. FA company analysis looks at the financial statements, e.g., a Company's income statements (IS), balance sheets (BS), cashflow statement (CS), financial ratios (FR), management, governance, insider activities, competitive advantage, labor forces, market share, and over-all company prospects. Fundamental analysis tends to focus on longer time horizons.
Alternatively, TA, on the other hand, involves the employment of several technical indicators or indicators related to transformations of generally univariate price fluctuations over time transformed into indicators variables in many forms of moving averages (MA) like simple moving averages, weighted moving averages (WMA), exponentially (weighted) moving averages (EMA), Moving Average Convergence/Divergence (MACD), spreads between highs and lows, volume, standard deviations, and so on. The point is that most of these are related to either price data or volume data that has been transformed.
Fundamental analysis tends to have a focus on longer time horizons and is favored by investors, while short-term traders favor ethical analysis because it tends to favor supply and demand movements that reveal very short-term behaviorally derived price arbitrage. Both tools are valid, but the question is for what periods? TA tends to be more accurate over intraday periods to weeks and during bull or rising market phases, while FA is better for a one year or longer with a slight but mixed advantage for TA in the 6-month time frame. This means that different approaches and different model specifications might favor different time horizons.
Economic Benefits of Technical Analysis (Wang et al., 2018):
The value of technical analysis has long been subject to a high level of distrust and scrutiny by academics in finance (Wang et al., 2018). In many ways, this scrutiny directed at TA is derived from a lack of a clear economic and financial theory supporting TA and its apparent contradiction with the main assumption of the Efficient Market Hypothesis (EMH) in all of its forms.
There are three forms of EHM set forth by Fama in his 1970 seminal paper. 1) The strong form of EMH states that the market prices reflect all information, which means no investor has a piece of monopolistic information relevant to price setting, and therefore, there is no information-based arbitrage opportunity in the market which would include the fact that stock prices cannot be predicted through historical price movement (Fama, 1970). This amounts to the old joke that if you were strolling and found $100 lying on the ground, do not pick it up because there is a trick and another would have grabbed it already if it was "free" (Lo, 2019). 2) The semi-strong form of EMH states that stock prices reflect available public information only and that there is private information hidden from common investors, but there might be forms of insider trading that give insiders an (illegal) information advantage, but still, this includes the fact that stock prices cannot be predicted through historical price movements. 3) The weak form states merely that stock prices cannot be predicted through historical price movement. EMH criticism of TA has been significant in academic circles to the point that TA is not seriously taught in most academic finance programs.
Wang and company argue that there is actual empirical evidence for economic utility being gained through TA methods using MA and Bollinger bands (BB) double-or-out (DO) and optimal portfolio (OP) strategies (Wang et al., 2018). Their research shows that TA does appear to generate real economic value empirically. This lends further credence to a blend of TA and FA for forecasting price expectations. In reality, there are over two hundred potential TA indicators with a range of strategies that sometimes blend serval of those indicators. Diagnostic Expectations and Stock Returns (Bordalo et al., 2017):
The National Bureau of Economic Research (NBER) tries to show how long-term earnings growth forecasts (LTG) have strong predictive power, yet analysts who follow a stock tend to be seen as having an overly optimistic bias leading to empirical data showing betting against the extremely positive analyst is on average a good idea (Bordalo et al., 2017). Analysis tends to overvalue stocks that are growing and undervalue stocks in decline (La Porta, 1996). A central feature of their theory is that investors are forward-looking in the extent to which they are said to react to news regarding events that affect publicly held companies. Their paper creates a model by analyzing the dynamics of expectations and how those exception dynamics are formed.
The paper proposes a psychologically founded theory that accounts for the behavior of fundamentals, expectations, and returns. This is followed by a new learning model in which beliefs are forward-looking just as with rational expectations, but distortions stem from biases in the interpretation of the news. Analysts excessively update and overweight in the direction of the positive or negative news. People react to information in the right direction but too strongly. At the same time, the model can be analyzed using a variation of Kalman Filter techniques used in models of rational learning.
This ties in with some of the core concepts of TA indicators about MA and BB, which look at the momentum indicators of overbought and oversold market price movements and provide a strong foundation for some of the gaps in EMH, which fails to explain the economic value being generated by TA. This means that not only is TA have a strong grounding in modern economic theory, but also that they may provide value and a form of arbitrage generated from human psychological biases.
A Systematic Review of Fundamental and Technical Analysis (Nti et al., 2019):
Now that we have a lot of ground perspective on the potential strategies for a different time horizon for stock price expectations and their justifications, it was important to take a survey of the many FA and TA methods that have been used over the years to make predictions (Nti et al., 2019). 77% percent of papers from 2007 - 2018 were focused on using TA (66% without any FA, 11% combined with FA). 33% of papers used FA (32% used only FA, while 11% used FA combined with TA). The Nti review sought to categorize and catalog many price prediction methods and techniques. 89.34% of papers used single sources of price data, while 8.2% used two sources of data, and 2.46% used three sources of data. The most common methods use SVM or ANNs.
Many models use both structured and unstructured data, like textual data from financial statements and news sources. Model Evaluations methods vary. Some use the correlation coefficient (R), some use Root Mean Squared Error (RMSE), some use Mean Absolute Percentage Error (MAPE), and some use Mean Absolute Error (MAE). For price volatility prediction, Root Mean Squared Prediction Error (RMSPE), Mean Squared Prediction Error (MSPE) and Mean Absolute Prediction Error (MAPE) are used. For Categorical variables, Accuracy, Precision for stock price rise, precision for stock price fall, F-score, Normalized Mean Squared Error (NMSE), and Prediction of Change in Direction (POCID) are used. ANNs far outperformed DTs and SVM models in most of the papers. While DT performance was the least effective, it was the most explainable. Further findings suggest that some machine learning methods work better for different parts of the globe. Sees gaps in many areas and suggests that there might be opportunities for using ensemble methods in the future, and suggest that there will be follow-up papers on the subject.
Novel Multi‑Source Information‑Fusion Predictive Framework (Nti et al., 2021):
Two years after the review of TA and FA methods, the same team released research on "a novel multi‑source information‑fusion predictive framework based on deep neural networks for accuracy enhancement in stock market prediction in the Journal of Big Data (Nti et al., 2021). These models used a fusion of macro-economic variables, historical price data, google trends data, macro-economic data, web financial news, tweet sentiment, and forums discussions. They used a blended Convolutional Neural Network (CNN) and stacked Long-Term Short-Term Memory Neural Network (LSTM) called IKN-ConvLSTM to create the feature creation methods for a hybrid Deep Neural Network (DNN) model. This method feeds the results of a CNN into an LSTM model and makes stock price directional predictions. The model was tested with and against SVM, DT, and three-layer Multi-layer Perceptrons (MLP). The average testing accuracy for MPL was 91.31%. SVM with a radial basis function was 74.31% accurate, and DT had 85.31% accuracy. In contrast, the feature augmentation technique of IKN-ConvLSTM was 98.307% accurate.
These methods of information and data fusion combined with feature engineering were highly successful. The authors suggest that more data fusion and additional sources might improve this method, in addition to using Generative Adversarial Networks (GANs), Autoencoders to enhance the current framework. Only made directional predictions still, directional predictions are highly valuable in creating features that attempt to make stock price predictions. The results of directional models can be incorporated into price expectation models and can be part of the overall feature engineering process. The real value of this paper is seeing the effectiveness that automated feature engineering methods can provide.
References
Ang, A. (2014). Asset Management: A Systematic Approach to Factor Investing. Oxford University Press.
Bordalo, P., Gennaioli, N., Porta, R. L., & Shleifer, A. (2017). Diagnostic Expectations and Stock Returns. NBER Working Paper Series. https://doi.org/10.3386/w23863 Chen, Y.-J., Chen, Y.-M., Tsao, S.-T., & Hsieh, S.-F. (2016). A Novel Technical Analysis-based Method for Stock Market Forecasting. Soft Computing, 22(4), 1295–1312. https://doi.org/10.1007/s00500-016-2417-2
Chen, L., Pelger, M., & Zhu, J. (2019). Deep learning in asset pricing. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3350138
Christie, P., & Isidore, R. (2018). Fundamental Analysis Versus Technical Analysis - A Comparative Review. International Journal of Recent Science Research 9(1), 23009- 23013. http://dx.doi.org/10.24327/ijrsr.2018.0901.1380
Fama, E. F. (1970). Efficient Capital Markets: A Review of Theory and Empirical work. The Journal of Finance, 25(2), 383. https://doi.org/10.2307/2325486
García, F., Guijarro, F., Oliver, J., & Tamošiūnienė, R. (2018). Hybrid Fuzzy Neural Network to Predict Price Direction in the German DAX-30 Index. Technological and Economic Development of Economy, 24(6), 2161–2178. https://doi.org/10.3846/tede.2018.6394
Kumar, G., Singh, U. P., & Jain, S. (2021). Swarm Intelligence Based Hybrid Neural Network Approach for Stock Price Forecasting. Computational Economics, 60(3), 991–1039. https://doi.org/10.1007/s10614-021-10176-9
La Porta, Rafael. 1996. Expectations and the Cross-Section of Stock Returns. Journal of Finance 51(5), 1715-1742. https://doi.org/10.3386/w23863
A Review of Price Expectations Prediction Methods 10
Lo, A. W. (2019). Adaptive Markets: Financial Evolution at the Speed of Thought. Princeton University Press. https://doi.org/10.1515/9780691196800.
Mohapatra, U. M., Majhi, B., & Satapathy, S. C. (2017). Financial Time Series Prediction Using Distributed Machine Learning Techniques. Neural Computing and Applications, 31(8), 3369–3384. https://doi.org/10.1007/s00521-017-3283-2
Nti, I. K., Adekoya, A. F., & Weyori, B. A. (2019). A Systematic Review of Fundamental and Technical Analysis of Stock Market Predictions. Artificial Intelligence Review, 53(4), 3007–3057. https://doi.org/10.1007/s10462-019-09754-z
Nti, I. K., Adekoya, A. F., & Weyori, B. A. (2021). A Novel Multi-source Information-Fusion Predictive Framework Based on Deep Neural Networks for Accuracy Enhancement in Stock Market Prediction. Journal of Big Data, 8(1). https://doi.org/10.1186/s40537-020- 00400-y
Wang, J.-N., Liu, H.-C., Du, J., & Hsu, Y.-T. (2018). Economic Benefits of Technical Analysis in Portfolio Management: Evidence from Global Stock Markets. International Journal of Finance and Economics, 24(2), 890–902. https://doi.org/10.1002/ijfe.1697
Yang, J., Wang, Y., & Li, X. (2022). Prediction of Stock Price Direction Using the LASSO LSTM Model Combines Technical Indicators and Financial Sentiment Analysis. PeerJ Computer Science, 8. https://doi.org/10.7717/peerj-cs.1148
