In this thesis, I address this heuristic crisis with the development of a fully-automated machine learning framework capable of optimizing arbitrary econometric state space ARIMA methods in a completely data-driven manner. With this framework, I compare the predictions of a model portfolio consisting of all 8 possible combinations of a standard ARIMA, a seasonal SARIMA, an ARIMAX model with socio-economic variables, and an ARIMAX model with conflict indicators of neighboring countries as exogenous predictors.
In addition, each model is examined on a monthly and quarterly periodicity. By comparing the out-of-sample prediction errors, I find that this approach can beat the no-change heuristic in the country-level one-year ahead prediction of the log change of conflict fatalities in all metrics used, including the TADDA score.
While the urgency for early detection of crises is increasing, truly reliable conflict prediction systems are still not in place, despite the emergence of better data sources and the use of state-of-the art machine learning algorithms in recent years. Researchers still face the rarity of conflict onset events, which makes it difficult for machine learning-based systems to detect crucial escalation or de-escalation signals. As a result, prediction models can be outperformed by naive heuristics, such as the no-change model, which leads to a lack of confidence and thus limited practical usability.
Contents
Abstract
Lists of Abbreviations and Symbols
1 Introduction
2 Method
2.1 State Space Modelling Approach
2.1.1 Linear Gaussian State Space Model
2.1.2 ARIMA State Space Model
2.1.3 SARIMA State Space Model
2.1.4 (S)ARIMAX - Regression with (S)ARIMA errors
2.1.5 Kalman Filter
2.2 No-Change Baseline Model
2.3 Evaluation Metrics
2.3.1 TADDA
2.3.2 Mean Absolute Error (MAE)
2.3.3 Root Mean Square Error (RMSE)
3 Data
3.1 Armed Conflict Location and Event Data Project (ACLED)
3.1.1 Analysis of Conflict Incidence and Country Categorization
3.2 International Monetary Fund - World Economic Outlook Database (IMF-WEO) .
3.3 World Bank - World Development Indicators (WB-WDI)
3.4 Variable Overview and Missing Data
4 Implementation in Python
4.1 No-Change Forecaster Class
4.2 State Space ARIMA Forecaster Class
4.3 Grid Search Class
4.4 Automated Model Building Process
5 Results
5.1 Global Model Performances
5.2 Country-Level Model Performances
6 Conclusion
Bibliography
A Appendix
A.1 Figures
A.2 Tables
Abstract
While the urgency for early detection of crises is increasing, truly reliable conflict prediction systems are still not in place, despite the emergence of better data sources and the use of state-of-the- art machine learning algorithms in recent years. Researchers still face the rarity of conflict onset events, which makes it difficult for machine learning-based systems to detect crucial escalation or de-escalation signals. As a result, prediction models can be outperformed by naive heuristics, such as the no-change model, which leads to a lack of confidence and thus limited practical usability. In this thesis, I address this heuristic crisis with the development of a fully-automated machine learning framework capable of optimizing arbitrary econometric state space ARIMA methods in a completely data-driven manner. With this framework, I compare the predictions of a model portfolio consisting of all 8 possible combinations of a standard ARIMA, a seasonal SARIMA, an ARIMAX model with socio-economic variables, and an ARIMAX model with conflict indicators of neighboring countries as exogenous predictors. In addition, each model is examined on a monthly and quarterly periodicity. By comparing the out-of-sample prediction errors, I find that this approach can beat the no-change heuristic in the country-level one-year ahead prediction of the log change of conflict fatalities in all metrics used, including the TADDA score.
List of Figures
1 Global out-of-sample MAE, MSE and TADDA error scores of the PREVIEW models as of July 2022
2 Map and histogram of average monthly conflict fatalities per 1 million inhabitants of the 227 ACLED countries
3 The geographical distribution of countries and their conflict incidences by the three country categories
4 The distributions of missing monthly observations for the socio-economic indicators from the training set over the subset of the 158 analyzed ACLED countries
5 Hierarchy of the forecaster classes and interfaces
6 Visualization of the 5-fold expanding-window cross-validation
7 In-sample fit and out-of-sample forecast of the log absolute number of conflict fatalities with the ARIMAX_SE_MONTHLY model for Afghanistan.
8 The interplay of all classes and functions for the automated generation of forecasts for all countries.
9 Average out-of-sample TADDA, MAE and RMSE prediction errors aggregated over all countries and prediction periods.
10 Out-of-sample prediction error of the monthly models over the 12 prediction periods.
11 Comparison of the average forecast performance of the monthly models and the quarterly models
12 Comparison of out-of-sample forecast errors of the seasonal and non-seasonal models.
13 Average out-of-sample prediction errors by model complexity
14 Country-level out-of-sample TADDA prediction errors of the best model ARIMAX_ j SE_MONTHLY
15 Country-level difference in the out-of-sample TADDA prediction errors of the ARIMAX_ j SE_MONTHLY and the NO_CHANGE_MONTHLY model
16 The 5 best and the 5 worst predicted countries of the ARIMAX_SE_MONTHLY compared to the baseline predictions.
17 Average out-of-sample prediction errors by continent of the no-change baseline, ARIMAX_SE_MONTHLY and all monthly state space ARIMA models
18 Average out-of-sample prediction errors of the no-change baseline, ARIMAX_SE_ j MONTHLY and all monthly state space ARIMA models aggregated by subcontinental regions.
19 Average out-of-sample prediction errors of the no-change baseline, ARIMAX_SE_ j MONTHLY and all monthly state space ARIMA models aggregated by country category.
20 Prediction error of the quarterly models over the 12 prediction periods.
21 Average regional out-of-sample prediction errors of the best model ARIMAX_SE_ j MONTHLY
List of Tables
1 Overview of all models whose conflict fatality forecasts are evaluated in this work ordered by complexity
2 Disjoint classification of the 227 ACLED countries according to their average number of monthly conflict fatalities.
3 The selected 20 of the originally 44 macroeconomic indicators of the IMF World Economic Outlook published in April 2022.
4 Average share of missing exogenous observations in the training data sets for each of the 8 models
5 The share of the countries with successful model fits in the automated fitting process by model.
6 Disjoint classification of the 227 ACLED countries according to their average number of monthly conflict fatalities.
7 Count and ISO-3 codes of countries grouped by their ACLED start date.
8 Descriptive statistics for the 8 ACLED conflict indicators
9 Overview of the conflict indicators generated from the ACLED data set
10 The selected 47 of the originally 56 socio-economic WB indicators used together with the IMF indicators as exogenous predictors
Lists of Abbreviations and Symbols
Abbreviations
Abbildung in dieser Leseprobe nicht enthalten
Symbols
Abbildung in dieser Leseprobe nicht enthalten
1 Introduction
New crises emerge around the world every year, posing an ever-growing challenge to government and international organizations. Recognizing the high social and economic damage caused by crises every year, the urgency of anticipating crises before they occur has gained more and more importance. UN Secretary-General Antonio Guterres even addressed crisis prevention as “not merely a priority, but the priority” in a speech to the Security Council in 2017 (Guterres 2017). This urgency has led to efforts in the development of crisis early warning systems, intended to initiate a paradigm shift from responding to conflict to anticipatory actions for crisis prevention.
An important component of crisis early warning is the prediction of the emergence and development of violent conflicts. Violent conflict is of particular interest because its consequences extend far beyond the direct effects of armed confrontation. For instance, armed conflict is responsible for an all-time high of 89.3 million displaced persons in 2021 according to UNHCR's annual Global Trends Report 2021, leading to significant migration flows (UNHCR 2022). Due to its severe socioeconomic impacts, violent conflict can also be understood as “development in reverse” (Collier et al. 2003). In addition, violent conflict can have global implications and can threaten food security, even in countries not directly affected as, for example, most recently after the start of the Ukraine war in February 2022.
Instead of a random scattering of resources, in theory, a conflict prediction system allows a much more targeted allocation of resources and actions. In reality, however, despite recent advances in research, the prediction of conflicts is still at an early stage. For a long time, conflict modelling focused on the identification of statistically significant variables, instead of out-of-sample prediction or relied on very simple forecasting models, such as low-resolution logit regression models that aimed at classifying countries into conflict or no-conflict. With the emergence of better data sources and technologies, state-of-the-art machine learning methods and out-of-sample validation have become the standard in modern conflict prediction. (Hegre et al. 2013, 2022, Ward et al. 2010)
Recently, there has been a shift from the previous classification approach to predicting escalation and de-escalation based on the log change in the number of conflict fatalities. This fatality log change was also the target variable of the 2021 ViEWS prediction competition organized by Uppsala University in Sweden. In the ViEWS prediction competition, 9 different models for country-month conflict prediction were submitted for the African continent. In the subsequent evaluation of the competition by Vesco et al. (2022), their prediction results were compared with each other, as well as with a naive no-change heuristic that always predicts a log change of 0. Surprisingly, none of the 9 submitted country-month models could beat the no-change heuristic, in the TADDA metric developed for measuring conflict predictions. With all the state-of-the-art models from conflict research being outperformed by a very simple heuristic, the credibility and thus practicality of current conflict prediction systems in a policy context has to be questioned. (Vesco et al. 2022)
Also, within the German Federal Government, efforts to develop practical crisis early warning systems have also been ongoing for several years. In the PREVIEW Division of the German Federal Foreign Office, which collaborated on this work, a quarterly conflict prediction system is already in use and is undergoing continuous development in close exchange with the research community. As of July 2022, machine learning algorithms such as Random Forest, XGBoost, but also time series models such as Auto ARIMA and Prophet are in use. The machine learning models developed at PREVIEW come very close to the performance of the no-change heuristic, which is used as a baseline, in the TADDA score, as shown in Figure 1. The time series models Auto ARIMA and Prophet, in contrast, perform very poorly with TADDA scores far above that of the no-change heuristic.
Abbildung in dieser Leseprobe nicht enthalten
Figure 1: Global out-of-sample MAE, MSE and TADDA error scores of the PREVIEW models for the forecast of the log change in conflict fatalities as of July 2022.
This heuristic crisis provides the motivation for this thesis, in which I have investigated whether and how an approach of econometric state space ARIMA methods can be used as a proposed alternative model to generate successful conflict predictions at country level. To this end, I develop an automated machine learning framework that optimizes the hyperparameters of state space ARIMA methods in a data-driven manner using a pipeline approach and then evaluates them out- of-sample using the metrics TADDA, MAE, and RMSE. With this framework, I test 8 different state space ARIMA variants. These include standard ARIMA, seasonal ARIMA models, ARIMAX models with socio-economic variables and neighbor conflict indicators as exogenous predictors. Since PREVIEW, unlike ViEWS, uses quarterly, rather than monthly, models, I test each of the 8 model variants at both temporal aggregation levels.
It turns out that the automated conflict prediction system developed in this thesis is able to beat the no-change baseline in all metrics, including the TADDA metric. Thus, this approach outperforms both the PREVIEW models and all models submitted to the ViEWS competition in the TADDA score, demonstrating that the no-change heuristic can indeed be beaten in the TADDA score, albeit just marginally.
In particular, these results prove that even less data-hungry time series models like the state space ARIMA methods can at least keep up with state-of-the-art machine learning models with proper hyperparameter tuning. Furthermore, by testing a total of 16 state space ARIMA variants, I identify the model components that are crucial for success. As a first result, it can be shown that the monthly models significantly outperform the quarterly models. While the implementation of seasonality seems to disturb the forecasts, the inclusion of socio-economic exogenous variables or conflict indicators of neighboring countries leads to the best forecasting results. Moreover, in the subsequent country comparison, I find that in addition to geographical patterns, the prediction errors also exhibit a certain dependence on the past country conflict incidence.
This thesis is organized as follows. In the next chapter, I describe the motivation for the models examined, which is derived from conflict theory, and discuss in detail how they are modeled and computed in state space form. Chapter 3 introduces the three datasets used, ACLED, the IMF World Economic Outlook, and the WB World Development Indicators, discussing their properties and their use in the models. Chapter 4 then deals with the development of the automatic machine learning framework for the implementation of the state space ARIMA approach in Python. Chapter 5 then presents the out-of-sample prediction errors of the models, which are then evaluated first at the global level and then at the country level. Finally, in chapter 6, the model results are critically reviewed and compared to the current state-of-the-art, after which the chapter concludes with an outlook for further research in the field of conflict prediction.
2 Method
In this thesis, I develop an automated framework for the global prediction of violent conflict that optimizes 16 different configurations of state space ARIMA methods in a data-driven approach for each country. Each of these ARIMA state space methods is then used to generate individual country-level 1-year ahead conflict forecasts. Therefore, I first specify a conflict indicator to act as the endogenous target variable y of the models which will be subject to the forecasts. For this purpose, I choose the (log) absolute number of conflict fatalities, which is a good indicator for the occurrence of violent conflict for several reasons. First, the mere existence of conflict fatalities is a clear signal that a conflict has escalated into violence. Second, the number of conflict fatalities provides a good quantifiable indication of the severity of the escalation of such conflict. Overall, the choice of this outcome variable ensures that magnitude only of escalated violent conflict is modeled. Furthermore, with sources such as the ACLED dataset, described in Chapter 3.1, a comprehensive data basis on the number of conflict fatalities is available.
For a successful modeling of conflict fatalities for forecasting, I rely on well-known conflict patterns and findings from conflict research. It is, for instance, well known in conflict theory that the probability for the emergence of a conflict is not equally distributed across all countries Collier et al. (2003). Mueller & Rauh (2022) show that, in fact, there is a strong dependence of future conflicts on conflict history. In general, there is very little empirical risk for the outbreak of a violent conflict and thus also for an increased number of conflict-related deaths. However, once a conflict erupts in a country, this country often remains stuck in a repeated cycle of violence, and an increased probability for the onset of new conflicts persists for several years after the start of the conflict. Collier & Sambanis (2002) refer to this phenomenon as the conflict trap.
Mueller & Rauh (2022) find that for countries with recurrent violence, conflict history plays a crucial role in predicting future conflicts. For these countries, the use of an autoregressive model that predicts future conflict fatalities using the number of conflict fatalities from previous periods seems to be a plausible straightforward approach. Hence, I choose to use the popular ARIMA time series model as the foundation for the statistical modelling in this work. The detailed structure of the ARIMA model with all its components is described in section 2.1.2 of this chapter.
However, the influence of conflict history on future conflict might exhibit more complex patterns than the simple dependence from the previous periods covered by the ARIMA model. Guardado & Pennings (2020), for example, found a substantial decrease in conflict intensity in the months of the harvest season in Iraq, Pakistan and Afghanistan. To cover such potentially also to other countries applicable seasonal patterns in the conflict history, I extend the basic ARIMA model with annual multiplicative effects to the seasonal ARIMA model SARIMA. In addition to the harvest effect, this generically modeled annual seasonality also allows for picking up other possible seasonal effects, such as annually fluctuating climate conditions for example. The mathematical implementation of the addition of multiplicative annual seasonality to the ARIMA model is discussed in Section 2.1.3.
While according to findings by Mueller & Rauh (2022) conflicts in conflict-rich countries can be predicted very well with their conflict history, which I cover with the (S)ARIMA models, Mueller & Rauh (2022) also show that for the prediction of the hard cases, namely the prediction of so far largely conflict-free countries, additional predictors are needed. Unlike Mueller & Rauh (2022), who solve this problem using text indicators from news articles, I pursue an approach with two other types of indicator variables.
In one attempt, I additionaly incorporate socio-economic country indicators XSE into the (S)ARIMA models. In addition to conflict history, Collier et al. (2003) and Collier & Sambanis (2002) also consider the economic conditions of a country as a driver for higher or lower conflict risk. They point out that slower economic growth substantially increases a country's conflict risk. As with the conflict trap, this is a vicious circle mechanism, because in the same way that a weaker economy leads to an increased risk of conflict, an economic slowdown is a possible consequence of conflict. I attempt to exploit this relationship for predicting conflict fatalities in the models I refer to as (S)ARIMAX_SE.
In a second attempt, I extend the (S)ARIMA models by integrating conflict indicators of all surrounding neighboring countries X N, resulting in the models labeled (S)ARIMAX_N. In doing so, I make use of the phenomenon of the spillover effect pointed out by Murdoch & Sandler (2002), Collier & Sambanis (2002). The spillover property of violent conflicts describes that the negative effects of a conflict can spill over to neighboring countries and destabilize them as well. For a country's conflict forecast, this means that the existence of a conflict in a neighboring country also increases its own conflict risk. The inclusion of conflict indicators from neighboring countries can therefore potentially help in forecasting the hard predictable cases, where there are no signs of erupting conflict in the country's conflict history.
Finally, in the (S)ARIMAX_SE+N models, I combine both types of exogenous variables just presented into a set X SE + N of exogenous predictors to see how useful these additional variables are in combination for the prediction of conflict fatalities. In all three cases of ARIMA models with additional predictor variables, I assume that all predictor variables are exogenous to conflict fatalities. This, however, is neither for the socio-economic indicators X SE + N nor for the neighbor conflict indicators X N very likely to be true. As stated above, Collier & Sambanis (2002) point out that conflicts not only have a negative impact on the socio-economic state of a country, but at the same time the negative socio-economic impacts of conflict are a driver for further conflicts. Likewise, the perspective of the spill-over effect can be reversed. The increased risk of conflict in a country caused by conflicts in neighboring countries can also be reflected back to all neighboring countries, where it in turn can contribute to further destabilization. Thus, these effects have a two-way working mechanism. How the indicators, which are assumed to be exogenous, can be elegantly combined with the (S)ARIMA approach to form a regression model with (S)ARIMA errors, is described in detail in Section 2.1.4 of this chapter.
Since it is a priori unknown, which of the model components are actually successful in predicting conflict fatalities and which ones rather disturbing the model, I will test all combinations of the above described model specifications. An overview of these ARIMA variants is provided in Table 1. These models can all be formulated and computed using the state space modeling approach. I discuss the motivation for using this approach, which differs from classical ARIMA modeling, in Section 2.1. I explain the components of the Linear Guassian state space model underlying all state space ARIMA variants in Section 2.1.1. To compute this general state space model, an efficient algorithm, the Kalman filter, can be formulated. With the one-time definition of the Kalman filter for the linear gaussian state space model in Section 2.1.5, the computation algorithm for all derived ARIMA state space models is specified simultaneously.
To be able to compare the predictive power of the models not only to relative to each other, I additionally generate naive baseline predictions for each country. For this purpose, I choose the nochange model used in the ViEWS Predictions Competition, which extrapolates the log absolute number of fatalities for each country unchanged into the future, see Hegre et al. (2022). The absolute forecast quality of the State Space ARIMA methods can thus be measured by whether and by how much they can beat these baseline predictions. The no-change baseline model is presented in detail in Section 2.2.
Moreover, I examine the influence of the periodicity of the data used in the modelling. In the German Federal Foreign Office, PREVIEW currently uses quarterly models for quantitative crisis early warning. In the ViEWS prediction competition, on the other hand, a monthly periodicity of input data and forecasts was used, see Vesco et al. (2022). In my work, I therefore test the forecast performance of all mentioned models, which are summarized in Table 1, once with monthly data (suffix: _MONTHLY) and once with quarterly data (suffix: _QUARTERLY). In total, this yields 16 different state space ARIMA model configurations and two no-change baseline models, whose model performances I will analyze and compare in chapter 5.
Abbildung in dieser Leseprobe nicht enthalten
Table 1: Overview of all models whose conflict fatality forecasts are evaluated in this work ordered by complexity. The NO_CHANGE model provides the baseline for the evaluation of ARIMA state space methods.
2.1 State Space Modelling Approach
In contrast to the classical Box-Jenkins modelling of ARIMA processes, I rely on the state space approach, which allows the formulation of equivalent state space ARIMA equations that can be converted back to the classical ARIMA equations. The State Space approach is kept very general and allows the modeling of any kind of system, which is why it has a wide range of applications in the natural sciences as well as in engineering. For time series forecasting, state space modeling allows a reinterpretation of the underlying problem as the evolution of a system over time: Instead of directly modeling the time series of an observed variable yt, yt is linked to the unknown internal states at of the system and then the temporal state evolution of the system is modeled. The training of a state space model corresponds to learning the unknown state components based on the observation time series yt. During forecasting, an estimate for a future state can be generated based on all previous states, which in turn can be used to derive the corresponding future value of the observed variable.
This approach offers the advantage that all ARIMA variants tested in this work can be constructed based on the general linear Gaussian state space model, which is introduced in the next Section 2.1.1. This is a convenient advantage in comparison to the Box-Jenkins approach, where, for example, the integration of exogenous variables is much more difficult to handle. For the computation of the linear Gaussian state space model, there exists a general formulation of the very efficient recursive Kalman filter algorithm, whose properties and mathematical properties are discussed in more detail in section 2.1.5. This property makes it possible to set up the Kalman filter equations only once in a general form for the linear Gaussian state space model and then apply them to all derived state space ARIMA models in the same manner. For the computation of the ARIMA state space models by the Kalman filter, only the state space matrices imposed by the general model have to be specified, which is the subject of the Sections 2.1.2 to 2.1.4 (Commandeur & Koopman 2007).
2.1.1 Linear Gaussian State Space Model
The linear Gaussian state space model (LGSSM) is the general framework for linear models with a Gaussian error term and is therefore capable of handling all ARIMA variants used in this work.
It is described by the following two equations, the observation equation and the state equation.
Abbildung in dieser Leseprobe nicht enthalten
The observation equation (2.1) establishes a relation between the observations y t and the state vector a t via the matrix Z t and the error term E t. y t thereby is a p x 1 vector containing the observations of the p endogenous variables at a time t. In my univariate fatality prediction problem, p = 1 holds and the y t correspond to the logarithmic conflict fatalities of a country at time t. The n states x 1 state vector a t is unobservable and determines the current state of the system at a point of time t. The matrix Z t is called design matrix. Depending on the model specification, it establishes a relation between the observation vector y t and the state vector a t and thus has to be of dimension p x n states. The p x 1 vector E t is a Gaussian error term characterized by an expected value of 0 and the diagonal p x p covariance matrix H t.
The state equation (2.2) describes the transition of the system from one state a t to the next state a t +1. The new state consists of an autoregressive part, determined by the n states x n states transition matrix T t, and a part that depends on the r x 1 error term n t and is controlled by the n states x r selection matrix R t. This property will be used in the following state space definition of the ARIMA models, which also consist of an autoregressive and an error dependent part. The error term n t follows an r -dimensional normal distribution with expected value 0 and the diagonal r x r covariance matrix Q t. The two equations hold for all n time points t = 1 , ..., n for which observations y t are available.
The initial state a 1 (2.3) is assumed to follow a normal distribution whose expected value a 1 and covariance matrix P 1 are assumed to be known.
Theoretically, the design, the transition and the selection matrices Z t, T t, R t as well as the two covariance matrices H t, Q t could contain the model parameters to be estimated. In the following sections, however, it becomes clear that in (S)ARIMA modeling T t will contain all (seasonal) autoregressive parameters and R t will contain all (seasonal) moving average parameters. The covariance matrix Q t contains the variance a [2] of the error term p t as parameter. The remaining matrices Z t and H t will remain unparameterized. It is also worth mentioning that all model components in the Equations (2.1) and (2.2), as implied by the t in the index, are theoretically time-variable in the general model. For the ARIMA formulations in the Sections 2.1.2 to 2.1.4, however, this optional generalization opportunity will be omitted for the matrices Z t, T t, R t, H t and Q t.
The linear Gaussian state space model makes the following assumptions for the two error terms:
Abbildung in dieser Leseprobe nicht enthalten
2.1.2 ARIMA State Space Model
The ARIMA state space model is the foundation of the state space ARIMA methods discussed in this thesis. The mathematical formulation of the ARIMA state space model presented in this section will serve as a basis for the following Section 2.1.3 on the seasonal ARIMA and Section 2.1.4 on the regression with (S)ARIMA errors.
The acronym ARIMA stands for the A uto R egressive I ntegrated M oving A verage model. According to the name, the ARIMA model is composed of 3 parts:
1. The AR part (2.4) is constructed similar to a classical regression with the difference that instead of an exogenous predictor variable x, the observations of the endogenous variable y from the p previous periods, in this case the log fatalities from the previous periods, are used as regressors.
Abbildung in dieser Leseprobe nicht enthalten
2. The integration part (2.5) calculates the d-th difference of yt with its preceding values and thereby aims to eliminate any trends from the time series. This enables also non-stationary conflict time series to be modeled with an ARMA process. The differenced time series is assumed to be stationary and is denoted by yt Hyndman & Athanasopoulos (2014), Durbin & Koopman (2001). For forecasting of conflict fatalities, I restrict myself to using only the simple differencing d < 1, which is sufficient for most cases, in order to avoid unintended side effects, which, for example, led to a steeply increasing linear trend for the forecast of Ukraine, which could not be justified by the data.
Abbildung in dieser Leseprobe nicht enthalten
3. The MA part (2.6) models the dependence of the endogenous variable on its previous observations, unlike the AR part, indirectly by regressing on the error terms of the q previous periods nt — j. The residual values rjt = yt — faP=1 fayt — i are realized upon the estimation of the AR parameters fa and are therefore initially unobservable.
Abbildung in dieser Leseprobe nicht enthalten
Summarized in Equation (2.7), these three parts yield the ARIMA(p,d,q) model in its classical form:
Abbildung in dieser Leseprobe nicht enthalten
In the following, I introduce the lag operator L, which can be used to represent the ARIMA model in a shortened notation. The lag operator L, when applied to a time dependent variable such as yt, shifts the variable back by one period in time(2.8). For the k times application of the lag operator (2.9), the variable is shifted back by k periods respectively.
Abbildung in dieser Leseprobe nicht enthalten
By that, the AR and MA part can be expressed by the lag polynomials
Abbildung in dieser Leseprobe nicht enthalten
and the d-th order differencing part can be written with the differencing operator A = (1 — L)
Abbildung in dieser Leseprobe nicht enthalten
yielding an notation of the ARIMA(p,d,q) model following Fulton (2015) that is equivalent to (2.7):
Abbildung in dieser Leseprobe nicht enthalten
To convert the ARIMA(p, d, q) model into state space form, the model constants m and n states are defined first. The order of the ARIMA(p, d, q) state space model is given by m and depends on the number of integrated AR(p) and MA(q) lags:
Abbildung in dieser Leseprobe nicht enthalten
Adding the differencing order d gives the constant n states, whichspecifieslengthofthestatevector a t:
Abbildung in dieser Leseprobe nicht enthalten
Together, the constants m and n states also determine the size of the state space model matrices Z t, T t, R t. In the linear Gaussian state space model (2.1),(2.2) all matrices are time-varying in their general form. However, for all ARIMA variants in this work, the state space matrices will be constant over all time points t.
Abbildung in dieser Leseprobe nicht enthalten
The design matrix Z in the univariate case of ARIMA(p, d, q) state space model takes the form of a 1 x n states row vector of zeros whose first d + 1 elements are filled with ones.
Abbildung in dieser Leseprobe nicht enthalten
For all (S)ARIMA modeling, the error term E t is not required. Consequently, its covariance matrix H is set to zero H:= 0, which, together with the expected value of 0, causes the error term E t to disappear from the model.
The transiton matrix T is constructed from an autoregressive component M ar and a differencing component M d. The autoregressive component (2.17) handles the AR part of the model and hence also contains the AR coefficients fa, i = 1, ...,p. However, for the matrix dimensions to fit together later, there must always be m AR coefficients in the matrix. The p - m artificially added coefficients are set to zero fa =0, Vi > p, so that they do not interfere with the model. The remaining empty space in T is filled with the m x n zero matrices 0 m,n. Since T transitions the a t state to the a t +1 state with the aid of Rn t, T must be of dimension n states x n states.
Abbildung in dieser Leseprobe nicht enthalten
The selection matrix R is multiplied by the r dimensional error term n t and thus controls the MA part of the ARIMA state space models. R here shrinks with r = 1 to a d + m column vector of zeros whose (d + 1)-th element is set to one. The last m elements are the moving average coefficients S 1 , ..., S m. As with the AR parameters, in the case of m > q + 1 the excess MA parameters are set to zero S j = 0 , V j > q + 1.
Abbildung in dieser Leseprobe nicht enthalten
Due to the only one-dimensional normal distribution of n t, the r x r covariance matrix Q with r =1 shrinks to a scalar Q := [a^].
For the ARIMA(p, d, q) state space model, the state vector at takes the form given in (2.21), which implicitly follows from the observation and state equation using the matrices defined above. The first d - 1 elements contain the increasingly differenced values of the prior period y t -1, which corresponds to the correspondingly differenced current value y t of the state vector prior period a t — 1. The d — th entry is the d times differenced value yf. The following m entries represent the ARIMA relationship.
Abbildung in dieser Leseprobe nicht enthalten
In the case of an ARIMA(2, 2, 2) model the model order would be m = max{2, 2 + 1} = 3 and a t would contain n states = 3 + 2 = 5 elements. The observation and state equation with all of the above defined components would consequently take the following explicit form:
Abbildung in dieser Leseprobe nicht enthalten
2.1.3 SARIMA State Space Model
The SARIMA(p, d, q) x (P, D, Q, s) state space model complements the ARIMA(p,d,q) by multiplicative seasonal effects. It is difficult to make a general statement about whether and how seasonality really occurs for each country without an elaborate analysis of all 227 conflict time series. In this work, I therefore use a straight forward approach and assume an annual seasonal periodicity. For the monthly models, the seasonal periodicity is hence set to s = 12, and for the quarterly models to s = 4. As a result, the log number of conflict fatalities in the autoregression is modeled not only by the values of the p previous periods yt—i, i = 1, ..., p, but also by the values of the P previous year periods yt—is, i = 1, ..., P. The same principle applies to the seasonal moving average term with the Q prior seasonal values of the error nt—js, j = 1, .., Q. In seasonal differencing, the previous years' values yt—ks, k = 1, ..., D are used accordingly.
To set up the SARIMA equation, I first define the seasonal lag operators Phi(L) and 0(L), as well as the seasonal differencing operator As, analogously to the non-seasonal operators from the previous section.
Abbildung in dieser Leseprobe nicht enthalten
This now allows the SARIMA(p, d, q) x (P, D, Q, s) model equation to be written in a shorthand notation:
Abbildung in dieser Leseprobe nicht enthalten
The general form of the seasonal ARIMA in state space form is obtained by integrating seasonal components into the ARIMA state space matrices defined in Section 2.1.2. For this purpose, the ARIMA model order (2.14) is first enhanced with the order of the seasonal parameters m s which leads to the SARIMA model order m '.
Abbildung in dieser Leseprobe nicht enthalten
Besides the model order, the dimension of the state vector a t, which is the number of states, changes accordingly:
Abbildung in dieser Leseprobe nicht enthalten
Within the design matrix Z in the seasonal case the differencing part from the ARIMA state space model (2.16) is extended by Ds entries with ones. Accordingly, all but the first d + Ds +1 ones are now zero entries and the matrix has grown to a dimension of 1 x (d + Ds + m '), which again corresponds to 1 x n states.
Abbildung in dieser Leseprobe nicht enthalten
The transition matrix transition matrix T, which contains an autoregressive and a differencing component in the ARIMA case, now additionally has to account for the seasonal autoregression and the seasonal differencing in the SARIMA case. For this purpose, the seasonal differencing component (2.30) is introduced, whose diagonal structure, is somewhat similar to that of the transposed matrix M AR (see (2.17)).
Abbildung in dieser Leseprobe nicht enthalten
The autoregressive component M AR from the ARIMA state space definition (2.17) is extended by the seasonal AR parameters (f>1,..., (f> ms to its seasonal counterpart M S aR. Again, all excess parameters with i > P are set to zero.
Abbildung in dieser Leseprobe nicht enthalten
Together with the non-seasonal differencing component M d (2.18) the seasonal autoregressive component (2.31), D diagonal repetitions of the seasonal differencing component M D (2.30) and the matrices M10, M01 and 0 n,m form the complete SARIMA transition matrix T. The dimensionality of T in the seasonal case is again n states x n states, where n states this time represents the number of states in the seasonal model (2.28).
Abbildung in dieser Leseprobe nicht enthalten
The selection matrix R is, because of the univariate error term nt (r = 1), again a column vector with nstates entries. The differencing part of the selection matrix increases with seasonal differencing by Ds zero entries. And in addition to the MA parameters of the ARIMA model 6 j,j = 1,m, R now contains additional ms entries in which the max{P, Q} seasonal MA parameters are each followed by s - 1 zeros. As with the artificial seasonal AR parameters in T, excess seasonal MA parameters O j with j > Q can occur and will also be set to zero in this case.
Abbildung in dieser Leseprobe nicht enthalten
The SARIMA model also has only a one-dimensional mean-zero error term nt, whose covariance matrix Q therefore again collapses to the scalar Q := [^J.
In the case of a SARIMA(1,1,1) x (1,1,1,4) model the model order is m' = max{1,1 + 1} + max {1 x 4 , 1 x 4} = 2 + 4 = 6, the number of states is n states = 6 + 1 + 1 x 4 = 10 and the above defined matrices take the following explicit form:
Abbildung in dieser Leseprobe nicht enthalten
2.1.4 (S)ARIMAX - Regression with (S)ARIMA errors
The (S)ARIMAX model class covers the ARIMA extensions with exogenous variables. The three models (S)ARIMAX_SE, (S)ARIMAX_N, (S)ARIMAX_SE+N differ in the type of exogenous variables used, which however makes no difference for the statistical modeling. These three ARIMA variants are implemented in the form of a regression whose error term E t follows an (S)ARIMA process. The state space model for the regression with (S)ARIMA errors can be formulated very elegantly by combining a state space regression model and a (S)ARIMA state space model.
First, the not yet defined state space regression model has to be introduced. The model equation of a multiple regression with intercept ß o and K exogenous variables x t,k has the general form given in Equation (2.34), which is valid for all observations t = 1 , ..., n. In contrast to a standard regression, the error term in the (S)ARIMAX model does not follow a Gaussian distribution, but a (S)ARIMA process.
Abbildung in dieser Leseprobe nicht enthalten
The regression model can be transformed to state space form with the below formulation of the state space matrices following (Hyndman n.d., Durbin & Koopman 2001), where 1 represents the unit matrix and 0 K +1 , 1 the corresponding zero matrix.
Abbildung in dieser Leseprobe nicht enthalten
Altogether, this gives the observation and the state equation of the state space regression model
Abbildung in dieser Leseprobe nicht enthalten
For the modeling of the (S)ARIMA error term, the (S)ARIMA state space matrices formulated in the previous sections 2.1.2 and 2.1.3 can be used, which I will label in the following with the indices (s)arima for better distinction. However, it is important to note that now E t instead of y t follows the (S)ARIMA process. The state space matrices for the combined model, the regression with (S)ARIMA errors, are now obtained by simply stacking the regression state space matrices onto the (S)ARIMA state space matrices:
Abbildung in dieser Leseprobe nicht enthalten
By plugging these matrices into the observation and state equation of the linear Gaussian state space model (see Equation 2.2 and 2.1), the complete regression model with (S)ARIMA errors can be written down in its general form:
Abbildung in dieser Leseprobe nicht enthalten
2.1.5 Kalman Filter
The Kalman filter is a recursive algorithm that allows the efficient computation of optimal forecasts for the state space models presented in the previous sections. Once formulated for the state space matrices of the LGSSM, the Kalman filter equations can be easily applied to all state space models derived from the LGSSM.
The goal of the Kalman filter is to obtain the best possible estimate for the distribution of a future state of a state space system a t +1 based on a known set of t observations Y t = { y 1 , • • • , y t } at the current time t and the distribution of the current state a t.
In the LGSSM the current state a t is assumed to follow a conditional normal distribution whose expected value E[ a t | Y t -1] = a t and covariance matrix Cov(a t | Y t -1) = P t are considered to be given for the recursion step from t to t + 1. Given a t and P t, the parameters for the distribution of the next state can be obtained recursively by inserting the state equation (2.2) of the LGSSM
Abbildung in dieser Leseprobe nicht enthalten
This result can now be further simplified and reformulated, as described in Durbin & Koopman (2001), which results in the equations of the Kalman filter (2.37):
Abbildung in dieser Leseprobe nicht enthalten
For models with time-invariant state space matrices, like the ones I use in this work, it can be shown that the covariance estimates Pt eventually converge to a steady state covariance P when the Kalman recursion steps are repeatedly applied. From the estimate of the future state vector an+1 resulting from Kalman filtering, the forecast of future observations yn+1 can be derived using the observation equation (2.1) of the LGSSM.
Abbildung in dieser Leseprobe nicht enthalten
With the normal distribution assumptions made for the LGSSM in Section 2.1.1, the Kalman filter with yn+1 provides the best estimate in terms of the mean square error, introduced in the following Section 2.3. In other words, this means that the mean square prediction error matrix is minimized by the Kalman estimator yn+1. Even for non-normally distributed observations y t, it can be shown for the ARIMA state space models of this paper with time invariant matrices that under certain conditions the Kalman algorithm still provides the best linear estimation. This, together with its other performance benefits, makes the Kalman filter a powerful tool for the automated prediction of conflict time series. (Kleeman 1996, Durbin & Koopman 2001)
Abbildung in dieser Leseprobe nicht enthalten
[...]
- Citar trabajo
- Adrian Leon Scholl (Autor), 2022, Development of an Automated Conflict Prediction System. State Space ARIMA Methods, Múnich, GRIN Verlag, https://www.grin.com/document/1325354
-
-
-
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X. -
-
¡Carge sus propios textos! Gane dinero y un iPhone X.