# 5. Idea 3: removal of remaining statistical discrepancy with

## Comments

## Transcription

5. Idea 3: removal of remaining statistical discrepancy with

Reducing statistical discrepancy between direct and indirect GDP Ronald van der Stegen, Jan van den Brakel, Pim Ouwehand and Reinier Bikker Statistics Netherlands Abstract Due to the need for recent macro-economic information for policy decisions, the attention has shifted from year to year (y-to-y) growth to quarter to quarter (q-to-q) growth of gross domestic product(GDP). However, the calculation method of the qto-q GDP growth has a much larger influence on the outcome than on the y-to-y result. For instance, comparing direct (calculated out of the GDP itself) and indirect (calculated out of the components) seasonal adjustment, the difference in growth of the GDP is sometimes in the same order of magnitude as the q-to-q growth itself. This statistical discrepancy therefore causes difficulty in, amongst others, the explanation of the growth in terms of the contribution of the final expenditure components to the growth. To close this gap, we propose a method based on, a pretreatment using a multivariate state-space model before applying X12 in order to minimize the discrepancy. Then X12 is followed by a multivariate Denton technique. With this technique, the statistical discrepancy resulting from the seasonal adjustment is attributed to the GDP and its components in a sophisticated way. To judge the quality of the results hereof we calculate the eleven quality measures of X12-arima (M1 till M11) and compare the adjustments of the GDP between the releases. Key words: Seasonal adjustment, X12-Arima, Denton technique, state space unobserved components 1. Introduction Statistics Netherlands publishes the seasonally adjusted volume of the gross domestic product (GDP) and its main expenditure components in millions of euros. The published growth of the GDP is calculated directly (i.e. seasonal adjustment of the GDP time series). Therefore, it is an easy task to calculate the quarter-to-quarter (q-to-q) growth of the GDP directly and indirectly (sum of seasonally adjusted components of the GDP). This has attracted the attention of the media and resulted in newspaper headlines questioning the reliability of the published GDP-growth. For example, the first publication of GDP growth after the 2013recession (2013-Q3) was 0.1 % calculated directly or –0.4 % calculated indirectly. This is much larger than the statistical discrepancy from the index formula. The behaviour of the directly calculated GDP is smooth, while the indirectly calculated GDP contains a strong seasonal pattern. Therefore Statistics Netherlands considers the directly calculated GDP as leading, however small changes are acceptable. However this discrepancy is difficult to explain to the users. Therefore we want to reduce the statistical discrepancy between the directly and indirectly calculated GDP. Figure 1: Direct and indirect growth of the GDP in de Netherlands and the discrepancy. Unfortunately, the third quarter of 2013 was not an incident, as can be seen in Figure 1. Since 2008 the economy is characterized by a frequent change from negative to positive growth combined with changing seasonal patterns. Therefore, it is difficult to determine seasonally adjusted series, often resulting in large discrepancies between the directly and indirectly determined GDP growth. With the increased interest in seasonally adjusted numbers, the problems associated with seasonal adjustment also attract more attention. At Statistics Netherlands, we think that the quality of the seasonally adjusted results is determined by three factors: 1. The quality of the seasonal adjustment on a univariate basis. X12-Arima and most other seasonal adjustment software have predefined, widely accepted, quality measures. However they were designed with certain assumptions about the series under consideration. When those assumptions are not valid or partly valid, the quality measure loses its value. For example consumption of households in the Netherlands, see Figure 2, has a changing seasonal pattern (determined with X12Arima). Therefore the quality parameters of X12 defining the stability of the seasonal pattern (M8 till M11) indicate bad quality, in case a flexible short seasonal filter is used. However, this filter is the right choice because an longer more averaging seasonal filter wouldn’t capture the change in seasonal pattern. Figure 2: seasonal pattern of consumption of households using X12Arima. 2. In the multivariate case, the statistical discrepancy between the aggregate and its components after seasonal adjustment. 3. Revisions of published results which are not based on new information on that particular period. In this paper, we present a method to improve the second quality parameter, the statistical discrepancy under the consideration that the quality of the seasonal adjustment of the individual series may not deteriorate too much. A few years ago Buijtenhek[2] made a first attempt to reduce the statistical discrepancy in the expenditure approximation of the GDP. This work is a continuation. In 2012 Quenneville and Fortier[5] published a rebasing method for reducing the statistical discrepancy. The discrepancy of our method with the ones mentioned is that the seasonal adjustment procedure is altered. Because of time constraints, the standardised use of X12 by Statistics Netherlands, the use Tramo-Seats instead of X12Arima was not considered. We tested the following ideas: The first idea was to improve the settings of X12-Arima, by not only looking at quality measures presented by X12-Arima, but also looking at the statistical discrepancy. This means that a slightly worse quality of the seasonal adjustment is acceptable when the statistical discrepancy is reduced. The second idea is to modify the seasonal adjustment procedure of X12-Arima. In the pretreatment of X12-Arima, instead of using an Arima model, we apply a multivariate structural time series model to remove outliers, working day effects and other regression effects. The extrapolation of the series is still done by X12. Because the proposed pretreatment is multivariate, the sum of the removal of an effect in the components of the GDP is equal to the removal in the GDP. After this pretreatment X12 (or actually X11) is applied. In case the results of the second idea are still not satisfying, a reconciliation procedure is applied based on the rebasing of the seasonally adjusted series of the GDP and its components, which further reduces the statistical discrepancies, keeping in mind that the quality of the seasonal adjustment must not alter too much. These ideas are explained in more detail below. The work presented is work in progress. 2. GDP The GDP in constant prices is broken down in seven components, namely: consumption of households (ch), consumption of government (cg), gross fixed capital formation (cf), changes in stocks (st), export (ex), import (im) and the initial statistical discrepancy (sd). The latter arises from the calculation of the chained volume series. The initial statistical discrepancy also contains a seasonal component that can be removed with seasonal adjustment. In case of current prices, the initial statistical discrepancy is zero. Before seasonal adjustment, all the component series add up the GDP and contain a seasonal component. In mathematical formulation: Eq. 1 : X(t)=R(t)*T(t)*I(t)* S(t) (multiplicative formulation), Eq 2 : X(t)=R(t)+T(t)+I(t)+ S(t) (additive formulation), Where X(t) denotes a time series, R(t) the time dependent regression effects, T(t) the time dependent trend, I(t) a fluctuating irregular component and S(t) the time dependent seasonal component. Before seasonal adjustment, the following equation holds: Eq 3 : XGDP(t) = XCH (t) + XCG(t) + XCF (t) + XST(t) + XEX (t) – XIM(t) + XSD(t). Equation 3 should still hold after the seasonal components in the eight series are removed. However, in practice we have two statistical discrepancies, one resulting from the index formula, XindexSD(t) and one from the seasonal adjustment Xs.a.SD(t). The Xs.a.SD(t) is not calculated by seasonal adjustment of the XindexSD(t) but as a remainder between then the seasonally adjusted GDP and its seasonally adjusted components. Often Xs.a.SD(t) has a much larger seasonal component than XindexSD(t) which is also not correlated with the original one. In the results we present, the seasonal component in all components is removed in a better way. This results in Xs.a.SD(t) without the S(t) component. The fluctuations in the statistical discrepancy are not zero, however much smaller than the current Xs.a.SD(t) and has even smaller fluctuations than XindexSD(t). In general, when we speak about the statistical discrepancy, we mean: Eq 4 : %sd(t) = (Xs.a.SD (t) - Xs.a.SD (t-1))/ Xs.a.GDP(t-1), the change of the statistical discrepancy compared to the seasonally adjusted GDP. 3. Idea 1: Improve the settings of X12-Arima Several ideas for the improvement of the seasonal adjustment were tested leading to the following conclusions: For each component, a detailed study was carried out to improve the current settings. The general conclusion was that, a higher quality of the seasonal adjustment of the individual series (in terms of the M-measures) didn’t automatically imply a smaller statistical discrepancy. Several other solutions were examined, all focusing on improving the pretreatment of the series. o Choosing similar setups for all series, for instance treating all in additive fashion and using the same regression effects such as outliers, and the same Arima-model was used for all series. Some of these options resulted in a bit smaller discrepancies after the pretreatment than the optimal univariate setups, but still the discrepancies were large. The main conclusion was that treating all component series in a similar fashion reduces the statistical discrepancy. The problem of the large statistical discrepancy is at its peak in 2010 (see Figure 1). In the years before and after it is, however still large. Therefore we assume that this is caused the changes in the seasonal patterns in that year. A break in the time series in that year would be logical however the period 2010-today is too short for seasonal adjustment. A break in the series in 2008 was also modelled, but this also gave no satisfying results. Therefore our conclusion is that with X12-Arima, a satisfactory reduction of the additional statistical discrepancy caused by correction for working day and seasonal effects, is not possible. 4. Idea 2: Multivariate pretreatment before X12-Arima In order to decrease the statistical discrepancy, we tried to reduce it in every stage of the seasonal adjustment procedure. The first stage is the pretreatment. Differences in regression effects and outliers between the individual series directly result in an additional statistical discrepancy. Yet, the differences arising in the pretreatment stage also lead to different choices in a later phase, resulting in an even larger statistical discrepancy. Harmonizing the pretreatment of the individual series therefore should result in more similar filters in X12 and less statistical discrepancy. The pretreatment of X12-Arima is replaced by multivariate seasonal structural time series model by using Ssf-Pack (Koopman e.a. [3]). Every series is modelled in the additive seasonal formulation, with all terms time dependent. We chose this software package because it is able to apply consistency constraints over the series. For instance, the regression or seasonal effects in the aggregate should sum up from the component series for every quarter in the series. The new pretreatment removes outliers, levelshifts and time dependent workingday effects from the series. The applied constraint is that the regression effects in the components sum up to those in the GDP. It is also possible to seasonally adjust the series with this method. However the determined seasonal pattern is too volatile to be published. For that we applied X12 with similar setups for all series. 5. Idea 3: removal of remaining statistical discrepancy with Denton The multivariate Denton technique is used to reduce the statistical discrepancy caused by the seasonal adjustment to its logical minimum, the seasonally adjusted statistical discrepancy caused by the index formula. The implemented version is described in Bikker[1]. The difference in magnitude of the series causes a problem. This is solved as follows. For every component of the GDP, a linear approximation is calculated with the least squares method. This is subtracted from the series and the sum of the linear approximations is subtracted from the GDP. The result is a set of eight series, very similar in magnitude. The rebasing with equal weights gives satisfying results. After the rebasing, we calculate the Quality measures of X12-Arima as given in Ladiray[5]. 6. Results We compare the three methods according to our three quality criteria. The first one is the current method used in the production process, the second is the improved method based on idea 2 (the multivariate pretreatment) and the third one is based on idea 2 followed by 3. The series start in the first quarter of 1988 and end in 2013. In total seven releases, one ends with the first quarter of 2013 and two end with the second quarter of 2013, two end with the third and two releases of the fourth. 6.1. X12-quality measures In table 1, the quality measures are calculated for the Dutch GDP. We see a small deterioration of the quality compared to the present situation, but the results are still acceptable. Note that the Denton technique alters the GDP series in idea 3, but this has minimal effect on the quality measures. Table 1: Quality measures of X12Arima for the Dutch GDP M1 M2 M3 M4 M5 M6 M7 M8 M9 M10 M11 Q Current 0.03 0.01 0.00 0.09 0.20 0.17 0.06 0.26 0.07 0.15 0.07 0.08 Idea 2 0.02 0.01 0.00 0.65 0.20 0.28 0.07 0.30 0.08 0.26 0.20 0.16 Idea2+3 0.02 0.01 0.00 0.65 0.20 0.28 0.07 0.29 0.08 0.29 0.21 0.16 6.2. Statistical discrepancy The second quality measure is the additional statistical discrepancy introduced by the correction for woking day and seasonal effects. Again we compare the same three methods for the 7 available releases of the GDP in 2013. Three measures are defined, the last, the maximum in the last 4 years and the average of four years of the absolute value of the statistical discrepancy. The results are presented in table 2. Idea 2 is better than the current method and Idea 3 removes all statistical discrepancies caused by the seasonal adjustment. The remaining statistical discrepancy is the seasonally adjusted statistical discrepancy of the indexformula. Table 2: Statistical discrepancy in % of the GDP in the expenditure formulation of the GDP (m.v.= multivariate petreatment). Release Current method Last quarter Idea 2: m.v. pretreatment Maximum Last 4 years Last quarter Maximum Idea 2 + 3: Denton Last 4 years Last quarter Maximum Last 4 years 1 0.10% 1.59% 0.58% 0.33% 0.57% 0.33% 0.09% 0.11% 0.05% 2 0.58% 0.93% 0.44% 0.32% 0.67% 0.29% 0.04% 0.11% 0.05% 3 0.50% 1.01% 0.36% 0.07% 0.59% 0.23% 0.01% 0.11% 0.05% 4 0.56% 0.99% 0.37% 0.16% 0.61% 0.28% 0.02% 0.11% 0.05% 5 0.43% 0.95% 0.51% 0.28% 0.61% 0.31% 0.04% 0.11% 0.05% 6 0.33% 0.99% 0.47% 0.20% 0.59% 0.32% 0.06% 0.11% 0.05% 7 0.84% 0.84% 0.42% 0.37% 0.57% 0.24% 0.02% 0.11% 0.05% 6.3. Revisions of the GDP In table 3, we present the revision of the GDP growth by comparing a new release of the GDP with the previous one. Three measures are defined, the last, the maximum in the last 4 years and the average of four years of the absolute value of the statistical discrepancy. The results are given in %-points. The current method has the smallest adjustments, followed by the multivariate pretreatment in idea 2. We had hoped that the Denton method would adjust every release with same adjustments, resulting in no extra adjustments, compared to idea 2. However this not the case. Table 3: Revisions of the GDP in new releases in %-points (m.v.=multivariate pretreatment). Release Current method Last quarter Idea 2 m.v. pretreatment Maximum Last 4 years Last quarter Maximum Idea 2+3: Denton Last 4 years Last quarter Maximum Last 4 years 1-2 0.0 0.1 0.02 0.0 0.1 0.01 0.1 0.1 0.03 2-3 0.1 0.1 0.06 0.1 0.1 0.06 0.1 0.1 0.03 3-4 0.1 0.1 0.01 0.2 0.2 0.02 0.1 0.1 0.02 4-5 0.1 0.1 0.04 0.1 0.1 0.05 0.0 0.1 0.03 5-6 0.2 0.2 0.06 0.2 0.2 0.03 0.2 0.2 0.03 6-7 0.1 0.2 0.07 0.0 0.2 0.04 0.1 0.3 0.09 7. Conclusions This paper discusses ongoing work to reduce the statistical discrepancies between a direct seasonal adjustment of GDP and the indirect adjustment based on the expenditure components. Our conclusions so far are: The quality of seasonal adjusted series is not only determined by the quality of the univariate seasonal adjustment, but also by the statistical discrepancy between a direct and an indirect approach and the revisions of previously published figures. The latter two criteria are more easily observed by the users than the first. The proposed methods worsen the first a bit (which is acceptable), improve the second and worsen the third. The initial statistical discrepancy between the chained volume GDP and its expenditure components before seasonal adjustment already contains a seasonal component. Ideal seasonal adjustment should result in a small discrepancy between the direct and indirect volume GDP, namely the seasonally adjusted initial statistical discrepancy. Differences in the pretreatment of the seasonal adjustment procedure are causing a significant part of the discrepancy between the direct and indirect approach. We developed a time series model which harmonizes the removal of outliers and regression effects and the extrapolation of the series (idea 2). Our model could replace the pretreatment of X12-Arima. The advantages of this model are smaller additional statistical discrepancies and easier X12 setups. The remainder of the discrepancy between the direct and the indirect approach can be removed with a rebasing method. Acknowledgements The authors would like to thank al colleagues of the methods and national accounts departments for their comments and especially Zi-Zhao Chang for programming M1 till M11. References [1] Bikker, R e. a.(2012), Macro-integration-multivariate Denton, Den Haag/Heerlen, Statistics Netherlands, http://www.cbs.nl/NR/rdonlyres/6CEAAB46-AE7D-4F6D-92CACF2DD49157A9/0/2012Macrointegrationmulitivariatedentonart.pdf [2] Buijtenhek, S. (2006), A model based approach for benchmarking seasonally adjusted time series, http://epp.eurostat.ec.europa.eu/portal/page/portal/euroindicators_conferences/documents_sea sons/BUIJTENHEK%20AB.pdf [3] Koopman, S.J., N. Shephard and J.A. Doornik (2008), SsfPack 3.0: Statistical algorithms for models in state space form, London Timberlake Consultants Press. [4] Ladiray, D. and B. Quenneville(2001), Seasonal adjustment with the X-11 Method, New York Springer-Verlag. [5] Quenneville, B. and S. Fortier (2012), Restoring accounting constraints in time series – Methods and Software for a Statistical Agency. In Bell. W.R. et al (ed.). Economic time series – Modelling and seasonality, Boca Raton, CRC Press.