Modeling and Forecasting of the financial market have been an attractive topic to scholars and researchers from various academic fields. The financial market is an abstract concept where financial commodities such as stocks, bonds, and precious metals transactions happen between buyers and sellers. In the present scenario of the financial market world, especially in the stock market, forecasting the trend or the price of stocks using machine learning techniques and artificial neural networks are the most attractive issue to be investigated. As Giles explained, financial forecasting is an instance of signal processing problem which is difficult because of high noise, small sample size, non-stationary, and non-linearity. The noisy characteristics mean the incomplete information gap between past stock trading price and volume with a future price. The stock market is sensitive with the political and macroeconomic environment. However, these two kinds of information are too complex and unstable to gather. The above information that cannot be included in features are considered as noise. The sample size of financial data is determined by real-world transaction records. On one hand, a larger sample size refers a longer period of transaction records; on the other hand, large sample size increases the uncertainty of financial environment during the 2 sample period. In this project, we use stock data instead of daily data in order to reduce the probability of uncertain noise, and relatively increase the sample size within a certain period of time. By non-stationarity, one means that the distribution of stock data is various during time changing. Non-linearity implies that feature correlation of different individual stocks is various. Efficient Market Hypothesis was developed by Burton G. Malkiel in 1991.
Table of Contents
- 1 INTRODUCTION
- 1.1 General Introduction
- 1.2 Problem Statement
- 1.3 Technologies
- 1.3.1 Python
- 1.3.2 Numpy
- 1.3.3 Scikit Learn
- 1.3.4 TensorFlow
- 1.3.5 Keras
- 1.3.6 Compiler Option
- 2 LITERATURE SURVEY
- 3 DATA & TOOLS
- 3.1 Data
- 3.1.1 Choosing the data-set
- 3.1.2 Gathering the data-set
- 4 PREVIOUS ANALYSIS
- 4.1 Technical Analysis Methods
- 4.2 Fundamental Analysis Techniques
- 4.3 Traditional Time Series Prediction
- 4.4 Machine Learning Methods
- 4.5 Deep Learning
- 4.5.1 Artificial Neural Network
- 4.5.2 Convolution Neural Network(CNN)
- 5 THE PROPOSED MODEL
- 5.1 Recurrent Neural Networks
- 5.2 Long Short Term Memory(LSTM)
- 5.3 Advantages of LSTM
- 6 SYSTEM DESIGN
- 6.1 System Architecture
- 6.2 LSTM Architectural Diagram
- 7 SYSTEM REQUIREMENTS
- 8 IMPLEMENTATION
- 8.1 Data Preprocessing
- 8.1.1 Libraries Import
- 8.1.2 Importing the training set
- 8.1.3 Feature Scaling
- 8.1.4 Inputs and Outputs
- 8.1.5 Reshaping
- 8.2 Building the Recurrent Neural Network (LSTM)
- 8.2.1 Libraries Import
- 8.2.2 LSTM Construction
- 8.2.3 Model Fitting
- 8.3 Prediction
- 8.3.1 Importing the Test Data
- 8.3.2 Scaling and Reshaping Test Data
- 8.3.3 Predicting Test Data
- 8.4 Visualization and Results
- 8.4.1 Visualization
- 8.4.2 Results
Objectives and Key Themes
The main objective of this project is to investigate the feasibility and performance of using a Long Short-Term Memory (LSTM) recurrent neural network for stock market price prediction. The project aims to improve upon existing methods, such as Artificial Neural Networks and Convolutional Neural Networks, which have shown average error losses of around 20%.
- Stock Market Prediction using LSTM Networks
- Comparison of LSTM performance with other prediction methods (ANN, CNN)
- Data preprocessing and feature scaling techniques for improved model accuracy
- Analysis of LSTM architecture and hyperparameter optimization
- Evaluation of model efficiency using performance measures.
Chapter Summaries
Chapter 1: INTRODUCTION: This chapter provides a general introduction to financial market forecasting and the challenges involved, particularly the complexities of the stock market. It highlights the limitations of traditional approaches like the Efficient Market Hypothesis and introduces the use of machine learning, specifically Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, as promising alternatives for stock market prediction. The chapter also defines the problem statement—to build a more accurate stock price prediction model using RNNs—and outlines the technologies used in the project (Python, NumPy, Scikit-learn, TensorFlow, Keras).
Chapter 2: LITERATURE SURVEY: This chapter reviews existing literature on stock market prediction, examining various approaches including LSTM-FCNs, gradient descent challenges in RNNs, the use of neural networks to improve RSI calculations, stock trend prediction using simple moving averages and news classification, visualization techniques for recurrent networks, and various LSTM architectural studies. These studies provide a basis for understanding the strengths and limitations of different methods and inform the choice of the LSTM model in this project.
Chapter 3: DATA & TOOLS: This chapter details the data acquisition and preprocessing steps. The authors explain their choice of Google stock data, emphasizing the importance of using a predefined dataset to avoid bias. They describe using Quandl for data gathering and the subsequent creation of a dataset representing daily percentage changes in stock prices. This chapter lays the groundwork for the subsequent modeling and analysis.
Chapter 4: PREVIOUS ANALYSIS: This chapter presents an overview of different stock market prediction methods, categorized into prediction methods, traditional time series analysis, technical analysis, fundamental analysis, machine learning methods, and deep learning. It delves into each category's approaches and limitations, providing background and context for the LSTM model presented later. Specific examples and visualizations of ANN and CNN methods are included.
Chapter 5: OUR PROPOSED MODEL: This chapter introduces the proposed model—an LSTM network. It explains the principles of RNNs and LSTMs, highlighting the advantages of LSTM in handling long-term dependencies and overcoming the vanishing gradient problem encountered in traditional RNNs. The chapter then thoroughly explains the architecture of LSTM units and the functions of their gates (input, output, forget), concluding with a discussion of the benefits of the LSTM architecture chosen for this project.
Chapter 6: SYSTEM DESIGN: This chapter details the system architecture of the proposed LSTM model for stock market prediction. It describes the workflow, which includes data collection, training data import, feature scaling, neural network creation, model training, test data import, result visualization, and efficiency calculation (using RMSE). The chapter illustrates the system architecture and the LSTM network diagram, giving a clear picture of the model's structure and operational flow.
Chapter 7: SYSTEM REQUIREMENTS: This chapter outlines the hardware and software specifications needed to run the LSTM model. It lists the required processor, RAM, CPU speed, GPU, operating system, storage, and compiler (Anaconda Python 3.5 with Spyder Compiler).
Chapter 8: IMPLEMENTATION: This chapter provides a detailed account of the model's implementation. It explains the data preprocessing steps, including library imports (Pandas, NumPy, Matplotlib, Scikit-learn), training set import, feature scaling using MinMaxScaler, splitting data into independent and dependent sets, and reshaping the data for use with Keras. The chapter then elaborates on building the LSTM network using Keras with TensorFlow backend, model fitting, test data processing, prediction, and result visualization using both Matplotlib and Plotly libraries. Finally, the chapter presents the results and evaluates the model's performance using the Root Mean Square Error (RMSE).
Keywords
Stock market prediction, Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Time series analysis, Machine learning, Deep learning, Python, TensorFlow, Keras, Data preprocessing, Feature scaling, RMSE, Model efficiency, Financial forecasting.
Frequently Asked Questions: Stock Market Prediction using LSTM Networks
What is the main objective of this project?
The primary goal is to explore the effectiveness of Long Short-Term Memory (LSTM) recurrent neural networks for predicting stock market prices, aiming to surpass the accuracy of existing methods like Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs).
What are the key themes explored in this project?
Key themes include stock market prediction using LSTM networks, comparing LSTM performance against ANN and CNN, data preprocessing techniques to enhance model accuracy, analysis of LSTM architecture and hyperparameter optimization, and evaluating model efficiency using performance metrics.
What technologies and tools were used in this project?
The project utilizes Python programming language along with libraries such as NumPy, Scikit-learn, TensorFlow, and Keras. The Anaconda Python 3.5 distribution with Spyder compiler was used.
What datasets were used, and how were they obtained?
The project used Google stock data obtained from Quandl. The dataset represents daily percentage changes in stock prices.
What data preprocessing steps were involved?
Data preprocessing included importing libraries (Pandas, NumPy, Matplotlib, Scikit-learn), importing the training set, feature scaling using MinMaxScaler, splitting data into independent and dependent sets, and reshaping the data for compatibility with Keras.
How was the LSTM network built and trained?
The LSTM network was constructed using Keras with a TensorFlow backend. The model was trained using the preprocessed data, and its performance was evaluated.
What other prediction methods were considered and compared?
The project compared the performance of LSTM against traditional time series analysis, technical analysis, fundamental analysis, ANNs, and CNNs. The limitations of each approach were discussed.
What are the advantages of using LSTM networks for stock market prediction?
LSTMs are advantageous because they handle long-term dependencies effectively and overcome the vanishing gradient problem common in traditional recurrent neural networks.
How was the model's performance evaluated?
The model's efficiency was evaluated using the Root Mean Square Error (RMSE).
What are the key chapters and their contents?
The report is structured into chapters covering introduction, literature review, data and tools, previous analysis of other methods, the proposed LSTM model, system design, system requirements, and implementation details including results and visualization.
What are the key findings or results of the project?
The project's results, including RMSE values and visualizations, demonstrate the performance of the LSTM model in predicting stock prices. Specific details are provided in Chapter 8.
Where can I find more details about the LSTM architecture used?
Chapter 5 and Chapter 6 provide detailed information on the LSTM architecture and the system design, respectively. This includes diagrams and explanations of the LSTM network's structure and workflow.
- Quote paper
- Joish Bosco (Author), Fateh Khan (Author), 2018, Stock Market Prediction and Efficiency Analysis using Recurrent Neural Network, Munich, GRIN Verlag, https://www.grin.com/document/419380
