NewsInn is an Artificial Intelligence Algorithm that parses the major news publications for news articles, groups them based on the main story, and presents them in order of importance.
To do a sentiment analysis on each article, the algorithm used a derivative of SentiWord, but it quickly became clear that the library was not created to be used in a news Environment. So a new library was developed, that would better suit the case it was being used in. It manages to provide a good estimate of the impact that the text of an article has.
Since the stock market is heavily affected by world problems, and therefore, world news, it is highly likely that a general sentiment analysis of the news would produce a result directly related to the general sentiment of the public on the world affairs. This would theoretically, in time, affect the stock market. The time-to market of world news we estimated to be of a few days, since only huge golbal events have the power to change the stock price in-day.
Therefore we started testing at a one-day offset, then went on for two, three, and even four days offset between the news data and the stock data. We only kept the data that provided the highest accuracy in predicting the stock market indexes flow.
ABOUT NEWSINN
Abbildung in dieser Leseprobe nicht enthalten
Description
NewsInn is an Artificial Intelligence Algorithm that parses the major news publications for news articles, groups them based on the main story, and presents them in order of importance. [1] It was design to become a news aggregator at first, but since important stories are covered be most publications, an Artificial Intelligence algorithm was implemented in order to group such articles.
Opinion Extraction
To do a sentiment analysis on each article, the algorithm used a derivative of SentiWord [2], but it quickly became clear that the library was not created to be used in a news Environment. So a new library was developed, that would better suit the case it was being used in. It manages to provide a good estimate of the impact that the text of an article has (see image 1).
Abbildung in dieser Leseprobe nicht enthalten
Image 1. An example of an article’s Sentiment Analysis Result. Partial screenshot of http://newsinn.org/display/mostnegativeweek/ on 2015/10/18
ABOUT STOCK INDEXES
Definition
A stock index or stock market index is a measurement of the value of a section of the stock market. It is computed from the prices of selected stocks (typically a weighted average). It is a tool used by investors and financial managers to describe the market, and to compare the return on specific investments. [3]
Theory
Since the stock market is heavily affected by world problems, and therefore, world news, it is highly likely that a general sentiment analysis of the news would produce a result directly related to the general sentiment of the public on the world affairs. This would theoretically, in time, affect the stock market. The time-to market of world news we estimated to be of a few days, since only huge global events have the power to change the stock price in-day.
Therefore we started testing at a one-day offset, then went on for two, three, and even four days offset between the news data and the stock data. We only kept the data that provided the highest accuracy in predicting the stock market indexes flow.
Data Collection
Taking into account the fact that the final version of the NewsInn A.I. Algorithm started gathering information, on the 14th of September, the NewsInn algorithm checked more than 24,000 articles against the Opinion Extraction library. We used this data to gather an average opinion score of the all the news written that day.
The opinion, however, was extracted from the conglomerated articles. We found no resemblance between the raw article opinion and the indexes. The explanation relies probably in the idea that people only register one event, even if, in some cases, it is detailed in more than 20 articles.
The Data extracted can be found below. Please note that week-end data was removed, as it is not highly accurate (contains less than half the number of the articles that a week day has), and the fact that Stock Markets are closed during week-ends.
Abbildung in dieser Leseprobe nicht enthalten
Table 1. Result of the Average Opinion of Conglomerated Articles (excluding week-ends)
We used the historical prices of Dow Jones Index , the S&P 500 Index and the NASDAQ Index for analysis inside of the graphs, while offsetting them by two days in the case of Dow Jones and NASDAQ, and one day for S&P 500, to accommodate the prediction time of the algorithm. [4]
We arrived at the following data, normalized for ease of use:
Abbildung in dieser Leseprobe nicht enthalten
Table 2. Table used to plot the graphs and extract data
Were data could not be found, we duplicated the last known data. That is, the last row of the table. But this row was not taken into account when doing the calculations. Please see below in blue (on the right).
RESULTS
Dow Jones Index
The prediction, two days in advance , for the Dow Jones Index had the highest degree of accuracy, with an average prediction rate of 69.44%. For 4 of the days, for unknown reasons, the prediction was offset by a day, so it could only predict the moving of the index in a one day interval. This was probably due to the refugee crisis.
For the offsetted case we divided the number of days correctly predicted by a factor of two.
Abbildung in dieser Leseprobe nicht enthalten
Image 2. Results of Dow Jones Prediction Two Days Ahead
Abbildung in dieser Leseprobe nicht enthalten
NASDAQ Index
The smallest amount of success in predicting the Stock Market Indexes was for NASDAQ. this index as well had an almost 100% rate of prediction for the last week. Since no data was available for the 23rd of October, the last day of the graph was not taken into consideration.
Abbildung in dieser Leseprobe nicht enthalten
Image 3. Results of NASDAQ Prediction Two Days Ahead
Abbildung in dieser Leseprobe nicht enthalten
S&P 500 Index
The S&P 500 index was predicted with an accuracy rate of 66.66%. That is 2 out of 3 predictions made by the algorithm were correct. As with the previous indexes, the last two weeks showed a very high accuracy, missing only two out of 16 days. The issue with this was that it could only predict the Stock Market fluctuation only a day ahead.
Abbildung in dieser Leseprobe nicht enthalten
Image 4. Results of Dow Jones Prediction Two Days Ahead
Abbildung in dieser Leseprobe nicht enthalten
CONCLUSIONS
Abbildung in dieser Leseprobe nicht enthalten
As predicted, Stock Markets are heavily influenced by the news, and therefore, a sentiment analysis on them can indeed predict the fluctuation with a medium degree of accuracy. For our experiment, the medium prediction was the following
Abbildung in dieser Leseprobe nicht enthalten
All indexes had an almost 100% rate of prediction for the last week. Since the algorithm is not yet in its final form, we can expect these numbers to go higher. It also helps the fact that the more data goes into the system, the more the NewsInn Algorithm learns, and the better associations it can do.
FURTHER RESEARCH
Abbildung in dieser Leseprobe nicht enthalten
In order to thoroughly test the Artificial Intelligence Algorithm, and its prediction on the stock market, we will need to wait some more time, so that it can collect more news data. Even if it was tested against 24,000 articles, we do not feel this has enough significance. We will keep monitoring and hope that, in six months’ time, NewsInn will have collected enough information to properly test it.
SOURCES
Abbildung in dieser Leseprobe nicht enthalten
[...]
[1] NewsInn Website http://newsinn.org/
[2] SentiWordNet http://sentiwordnet.isti.cnr.it/
[3] Wikipedia - Stock market index page https://en.wikipedia.org/wiki/Stock_market_index
[4] Yahoo Finance Historical Quotes https://uk.finance.yahoo.com/q/hp?s=%5EDJI
-
-
-
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X.