I. INTRODUCTION

A large body of literature shows that in-sample stock return predictability can be established using a number of financial fundamentals or macroeconomic indicators. There is, however, ample evidence that overturns the proposition. Stambaugh (1986) and Mankiw and Shapiro (1986) are the first to suggest that stock return predictability is due to small sample biases in the coefficient of prediction regression. Nelson and Kim (1993) assert that the predictability of stock returns are ascribed to small sample biases and that the empirical distribution of the test statistic by randomizing residuals is useful to address the issue. Nelson and Kim show that the more autocorrelated is the predictor variable and the smaller is the sample, the more the predictor appears to have predictive power even if there is no predictability. This is specifically relevant to our study because two predictor variables, dividend yield and price-earnings ratio, are highly persistent as is generally the case for many predictor variables in the extant literature. Bossaerts and Hillion (1999) address model overfitting biases by applying optimal model selection criteria in the statistics literature to find external validity. They claim that the pseudo-predictability is not induced by model overfitting if the forecasting model is chosen by various information criteria. However, they find a disparity between in-sample and out-of-sample evidence of predictability. That is, they find a dismal out-of-sample predictability from forecasting models based on information-theoretic criteria.

There are other studies that employ bootstrap correction for data-mining biases to analyze technical trading rules in the context of investment performance. Sullivan, Timmermann and White (1999), for instance, use White’s Reality Check bootstrap procedure to measure data-mining biases and the effects on the technical trading rules. Their empirical finding is that the best performance of the technical trading rules is revealed only in the in-sample analysis. Goyal and Welch (2003), decomposing dividend yield components, show that dividend ratios forecast itself but not stock returns. This lends support to the view that dividend ratios do not have predictability of stock returns and that the persistency of the dividend ratios cause unpredictability of stock returns.

Another line of research reports that, although there exists an empirical evidence of significant predictability in stock returns, the statistical evidence primarily comes from in-sample test of forecasting models. Some studies report that we do not have enough evidence of out-of-sample stock return predictability using a similar set of predictors. There has been a growing concern that in-sample predictive regression procedure is vulnerable to model overfitting pretest biases examined in Lo and MacKinlay (1990). This is especially the case in the study of asset pricing since a variety of predictability tests inspired by prior investigations are applied to the same data set. The accumulated empirical results unavoidably affect the current approach to testing the null hypothesis of unpredictability of stock returns. Prior information related to statistics to test the null hypothesis can significantly distort statistical inference. To overcome the issue, our study attempts to cover different markets, eight Asia-Pacific stock markets.

Inoue and Kilian (2004), on the other hand, seek to explain the conflicting results by investigating the low power of the out-of-sample tests and the increased size of the in-sample tests of predictability. Furthermore, Inoue and Kilian argue that it should be advisable to derive data-mining robust out-of-sample tests as well as data-mining robust in-sample tests following the spirit of White (2000). The influence of structural changes on the power of both in-sample and out-of-sample predictability tests is analyzed in Clark and McCracken (2001, 2004). Rapach and Wohar (2006), employing data-mining bootstrap procedure and powerful out-of-sample test statistics, show that stock market returns in the US can be predictable both in-sample and out-of-sample with a variety of financial predictors. Synthesizing the existing literature on contradictory results of in-sample and out-of-sample predictability tests, Rapach and Wohar (2006) report interesting results that stock returns have a predictable component out-ofsample as well as in-sample. In assessing out-of-sample forecasts, they correct for datamining biases by employing the bootstrap procedure proposed by Inoue and Kilian (2004). The study also employs a variety of statistics to test out-of-sample predictability. They find not much of a disparity between in-sample and out-of-sample predictability. This result is encouraging considering the fact due to Inoue and Kilian (2004) that the power of the out-of-sample predictability tests is higher than the power of the in-sample predictability tests even without data-mining and structural change in the forecasting model. Recently, Gupta and Modise (2010, 2012) apply the data-mining bootstrap procedure to South African stock market and concludes that the out-ofsample stock return predictability can be attributed to data-mining.

This paper aims to overcome the elusive evidence of out-of-sample predictability while we have abundant evidence of in-sample predictability. Without any presumption that data-mining is more of an issue for in-sample predictability test, we investigate in-sample and out-of-sample forecasting model. To achieve the objective, we employ a data-mining bootstrap procedure due to Inoue and Kilian (2004) and Rapach and Wohar (2006) along with a variety of test statistics to test out-of-sample predictability due to McCracken (2007) and Clark and McCracken (2001, 2004). We use a dataset of eight Asia-Pacific stock markets in the empirical analysis.

The organization of the article is as follows. Section 2 presents in-sample and out-of-sample predictability comparison methods and data-mining bootstrap procedure. The benchmark econometric methodology employed in the section is from Rapach and Wohar (2006). In section 3, we describe the data from the eight Asia-Pacific stock markets and report the in-sample and out-of-sample predictability test results. Section 4 concludes the discussion.

II. METHODOLOGY

1. Forecast Comparison

We assume a prediction regression of the following form:

where r_t is the real return on the stock from holding period t-1 to t, x_t is the financial fundamentals or macroeconomic indicators assumed to predict stock returns, ε_t is the disturbance term and ℎ is the h-step-ahead forecasting horizon. Under the null hypothesis of no predictability (β = 0), future stock returns have no predictable components when x_t is a predictor variable.

We split the available data into estimation and forecasting subsamples, where the estimation subsample spans the first τ observations. To test out-of-sample predictability, we estimate the model (1) via the OLS over the estimation subsample and construct the h-step-ahead forecasts The recursive procedure by pushing the forecast subsample one period forward is repeated from τ to T − ℎ. The h-stepahead forecast error from the unrestricted and restricted models and the respective mean squared error (MSE) are computed.

The four statistics, the Theil’s U statistic, the Diebold and Mariano (1995) statistic, the McCracken (2007) MSE-F statistic and the Clark and McCracken (2001) ENC-NEW statistic, are constructed to perform predictability comparison. The Theil’s U statistic is the ratio of the root mean squared prediction error (RMSE) for the forecasting model in equation (1) to the RMSE for the model under the null hypothesis of no predictability. The Diebold and Mariano (1995) statistic for equal predictability is also based on the MSE criterion. The null hypothesis of the MSE-F statistic is calculated under the null hypothesis of equal predictability measured by the model forecast MSE against the one-sided alternative hypothesis. The ENC-NEW statistic due to Clark and McCracken (2001) compares the forecast accuracy of two competing models, the constant expected returns model and the equation (1). The encompassing test of Clark and McCracken (2001) test for nested forecast models, where the naive constant expected returns model encompasses the competing model (1) if the equation (1) contains no further information in forecasting returns beyond the predictability of the constant expected returns model. Therefore, rejecting the null of forecast encompassing means that forecasts from equation (1) outperform significantly better than those from the model under the null hypothesis of no predictability. Rapach and Wohar (2006) extensively compare recursive out-of-sample forecasts from equation (1) to those from the constant expected returns model using the log real returns on the S&P 500 and the financial fundamental variables. We use the bootstrap procedure to address the issues related to a non-standard and non-pivotal asymptotic distributions of the MSE-F and the ENC-NEW statistics.

2. Data-Mining Bootstrap

We use a data-mining bootstrap procedure as in Inoue and Kilian (2004) and Rapach and Wohar (2006) to evaluate the degree of predictability of stock returns with a variety of financial fundamentals as predictors. Their proposed procedure is employing a variety of candidate predictors in setting up critical values for both in-sample and out-of-sample predictability tests. Since the asymptotic distributions of the in-sample and out-of-sample test statistics are data-dependent, it is difficult to base our inferences on the limiting distributions. The data-mining bootstrap algorithm for testing the predictability due to Rapach and Wohar (2006) can be specified as follows. Estimate the predictive regression with the actual data set.

where r_t is the continuously compounded real stock return. x_t is the financial fundamentals or macroeconomic indicators assumed to predict future real stock returns. For each h of forecasting horizon, construct the test statistic. We assume the following data generating process (DGP) under the null hypothesis of no predictability,

where the disturbance vector, ε_t = (ε_1,t, ε_1,2,t, ⋯, ε_J,2,t)', is independently and identically distributed with covariance matrix Σ. This process can be estimated by OLS with a maximum lag order of four and (p₁, p₂, ⋯, p_J)' are selected using the Akaike Information Criteria (AIC). From the OLS estimate of the DGP in equation (2) and the randomly drawn T+100 estimated residuals in tandem, we can construct a T+100 pseudo-sample of observations for r_t and x_t. The disturbance terms used in setting up the pseudo-sample will keep the contemporaneous serial dynamics of the original sample and are i.i.d.. By discarding the first 100 burn-in observations, we have a pseudo-sample of size T. We repeat the aforementioned procedure 1,000 times in order to generate the empirical distribution of the test statistics for each predictor j for the pseudo-sample. For each repetition, we calculate the maximal in-sample and out-of-sample test statistics. The empirical significance level of the maximal in-sample and out-of-sample test statistics for each j can be calculated as the proportion of the bootstrap statistics that are greater than the in-sample and out-of-sample test statistics computed from the original data set in the first step of the procedure. We can evaluate the significance of the predictor variable of the forecasting model under the null hypothesis of no predictability.

III. ESTIMATION1

The data used in the study consists of the annual observations for the S&P 500 index (1871-2016, 146 observations), Australia ASX All-Ordinaries index (1882-2016 for the index and the dividend yield, 135 observations, 1969-2016 for the price-earnings ratio, 48 observations), Canada S&P/TSX 300 index (1934-2016 for the index and the dividend yield, 83 observations, 1956-2016 for the price-earnings ratio, 61 observations), Hong Kong Hang Seng index (1972-2016, 45 observations), Japan Nikkei 225 index (1914-2016 for the index and the dividend yield, 103 observations, 1956-2016 for the price-earnings ratio, 61 observations), KOSPI (1963-2016 for the index and the dividend yield, 54 observations, 1974-2016 for the price-earnings ratio, 43 observations), Malaysia KLSE index (1973-2016, 44 observations), and Singapore SES All-Share index (1972-2016, 45 observations). Each market spans different time period due to data availability. The financial fundamental variables, the dividend yield and the priceearnings ratio, are used as the predictors of stock returns.

The descriptive statistics are reported in Table 1. The minimum annual return on the stock market index is -60.55 percent in Hong Kong, and the maximum annual return on the stock market index is 119.48 percent in Korea. The volatility in the stock market is the highest with the standard deviation of 41.67 percent in Hong Kong, and the lowest with the standard deviation of 15.94 percent in Australia. While the distributions of the returns on six stock market index, Australia, Hong Kong, Japan, Korea, Malaysia, and Singapore, have long tail to the right, the distributions of the US and Canada stock market index returns have long tail to the left. The return distributions have fatter tails than the normal distribution in the six stock markets except the US and Canada. The Jarque-Bera test for normality is rejected at the 5 percent significance level for Australia and Japan. For Hong Kong, Korea, and Singapore, the Jarque-Bera normality is rejected at the 10 percent significance level. For the US, Canada, and Malaysia, the stock market index returns are normally distributed.

Table 2 reports the estimation results for the in-sample predictive regression model with dividend yield as the predictor variable for horizons of up to 10 years. The numbers in the first row for each forecasting horizon are the slope coefficients of the prediction regression and the numbers in parentheses are the p-values calculated by the data-mining bootstrap procedure. Based on the p-values generated from the bootstrap procedure, the dividend yield is statistically significant under the 10 percent significance level in predicting the annual returns over all forecasting horizons for Hong Kong and Korea. The boldfaced numbers in parentheses in the table indicate the exact significance level and the corresponding slope coefficients are significant under the 10 percent significance level. For Malaysia and Singapore, the dividend yield can predict the real returns in the short and long forecasting horizons but not in the intermediate horizons. For the US, Australia, Canada, and Japan, the dividend yield has no ability in predicting the returns. Table 3 reports the estimation results for the in-sample predictive regression model with price-earnings ratio as the predictor variable for horizons of up to 10 years. Based on the p-values for the test statistics, however, we can conclude that the price-earnings ratio has no predictive power at the 10 percent significance level.

From the analyses of the in-sample predictability tests, the two financial variables show notable difference in predicting real stock index returns. While dividend yield is useful in predicting real returns at most of forecasting horizons in some countries, we find no evidence of return predictability for the price-earnings ratio in our sample of data.

Table 4 and 5 report the out-of-sample predictability test results with dividend yield and price-earnings ratio as the financial fundamental variables. The numbers for each forecasting horizon are the Theil’s U, MSE-T, MSE-F, ENC-T, ENC-NEW statistics, and the corresponding p-values in parentheses calculated by the bootstrap procedure. When the Theil’s U statistic is less than 1, the RMSE from the forecasting model with the financial fundamentals are smaller than the RMSE from the model under the null of no predictability. When the MSE-T, MSE-F, ENC-T, and the ENC-NEW are statistically significant, the unrestricted model is superior in predictability to the restricted model. We can conclude that the financial fundamentals have out-of-sample predictive power in forecasting future returns.

Considering the possibility of structural instability of the forecasting model proposed by Stock and Watson (2003), it should be advisable to balance the number of data in the estimation window for the prediction model. The rationale behind this is to take the small sample bias and the structural break during the estimation period into account. We estimate an in-sample prediction model using a half of the full sample observations to form first out-of-sample forecast. To form second out-of-sample forecast, we include one more observation in the window and estimate an in-sample prediction model using a half of the full sample observations. The repetition of the procedure is made until we exhaust available data depending on the out-of-sample forecast horizon.

In our analysis with the international stock markets, as shown in Table 4, stock returns are predictable in Hong Kong and Malaysia for all forecasting horizons when we use dividend yield as the out-of-sample forecasting variable. Stock returns in Hong Kong and Malaysia are predictable in-sample as well as out-of-sample for all forecasting horizons. As shown in Table 2 and 4, stock return predictability can also be found in Singapore with a few exceptions in some forecasting horizons. In Singapore, stock returns are predictable with dividend yield as a forecasting variable in the short and long horizons but not in the medium-term horizons. A notable point in the analyses is that there is little congruity between in-sample and out-of-sample predictability in the Korean stock market return. As shown in Table 2, the in-sample predictability can be established when we use dividend yield as the forecasting variable for all horizons. This is not the case for the out-of-sample predictability. The dividend yield cannot predict stock returns in the Korean stock market at all.

Table 3 shows that none of stock market returns can be predicted by price-earnings ratio for all forecasting horizons. According to Table 5, however, we find some evidence of stock return predictability in the US and Australia for long horizons over nine years. This is the case where we find significant evidence of out-of-sample predictability without having no significant in-sample predictability. The only market that we cannot find evidence of return predictability is Canada. For Hong Kong and Japan, stock return predictability is conspicuous for all horizons. The Korean stock market shows return predictability for over six-year horizon, but evidence of predictability is not very strong. For Malaysia, stock return predictability shows up intermittently without any specific pattern. For Singapore, stock returns are predictable in out-of-sample only for one-year and two-year horizon.

1)We use the GAUSS programs by Rapach and Wohar (2006) in producing the results reported here. We thank Dr. Rapach for providing us with the programs.

IV. CONCLUSION

The major finding of this study is that both in-sample and out-of-sample predictability are established using dividend yield and price-earnings ratio as financial fundamentals. The stock returns from eight Asia-Pacific market sample where we find the existence of predictability are predictable without notable difference between in-sample and out-of-sample forecasting models. In this paper, we use a data-mining bootstrap procedure proposed by Inoue and Kilian (2004) and Rapach and Wohar (2006) to fill the gap between in-sample and out-of-sample predictability. In order to show that stock returns are predictable, we employ the eight international stock market index data and the two financial fundamental variables. Depending on the data availability, the annual observations for the US S&P 500 index, Australia ASX All-Ordinaries index, Canada S&P/TSX 300 index, Hong Kong Hang Seng index, Japan Nikkei 225 index, Korea KOSPI, Malaysia KLSE index, and Singapore SES All-Share index are used. The empirical findings are summarized as follows.

We present the estimation results for the in-sample predictive regression model with dividend yield as the predictor variable for horizons of up to 10 years. Based on the p-values generated from the data-mining bootstrap procedure, the dividend yield is statistically significant under the 10 percent significance level in predicting the annual returns over all forecasting horizons for Hong Kong and Korea. For Malaysia and Singapore, the dividend yield can predict the log real returns in the short and long forecasting horizons but not in the intermediate horizons. For the US, Australia, Canada and Japan, the dividend yield has no ability in predicting the returns. We also report the estimation results for the in-sample predictive regression model with price-earnings ratio as the predictor variable for horizons of up to 10 years. Based on the p-values for the test statistics, however, we can conclude that the price-earnings ratio has no predictive power at the 10 percent significance level for any of the country investigated.

In our analysis with the international stock markets, stock returns are predictable in Hong Kong for all forecasting horizons when we use dividend yield as the out-of-sample forecasting variable. Stock returns in Hong Kong are predictable in-sample as well as out-of-sample for all forecasting horizons. Stock return predictability can also be found in Malaysia and Singapore with a few exceptions in some forecasting horizons. In the two countries, stock returns are predictable with dividend yield as a forecasting variable in the short and long horizons but not in medium-term horizons. There is significant disparity between in-sample and out-of-sample predictability in Korean stock market return. For the out-of-sample predictability test, the dividend yield cannot predict stock returns in the Korean stock market.

When we use price-earnings ratio as the forecasting variable, none of stock market returns can be predicted for all forecasting horizons. However, we find some evidence of stock return predictability in the US and Australia for long horizons over nine years. This is the case where we find significant evidence of out-of-sample predictability without having no significant in-sample predictability. The only market that we cannot find evidence of return predictability is Canada. For Hong Kong and Japan, stock return predictability is conspicuous for all horizons. The Korean stock market shows return predictability for over six-year horizon, but evidence of predictability is not very strong. For Malaysia, stock return predictability shows up intermittently without any specific pattern. For Singapore, stock returns are predictable only for one-year and two-year horizon.

This paper intends to overcome the disparate evidence between in-sample and out-of-sample predictability. The empirical findings from this study suggest that stock returns are predictable not only in-sample but out-of-sample in Hong Kong, Malaysia, Singapore, and Korea with a few exceptions for some forecasting horizons. However, we find some significant disparity between in-sample and out-of-sample predictability in Korean stock market. For Hong Kong, Malaysia, and Singapore, stock returns have predictable components in-sample and out-of-sample. For the US, Australia, Canada and Japan, we do not find any evidence of return predictability in-sample and out-ofsample with a few exceptions. It is encouraging from this study that we find not much of a difference between in-sample and out-of-sample predictability.