ARTICLES

 

Enhancing the Accuracy of Fiscal Projections in South Africa

 

 

Estian Calitz^I; Krige Siebrits^I; Ian Stuart^II

^IDepartment of Economics, University of Stellenbosch
^IINational Treasury and research associate, Department of Economics, University of Stellenbosch

 

 

---

ABSTRACT

The accuracy of the National Treasury's projections of GDP and key fiscal aggregates is comparable to that of the projections of private sector economists, other reputable organisations and the fiscal authorities of other countries. The errors in the projections of the National Treasury have nonetheless been substantial in some years, and have increased from 2000/01 to 2010/11. This paper argues that the credibility of fiscal policy would have been severely tested if the largest annual errors in respect of the various aggregates had coincided. Against this backdrop, the paper makes the case for structured bi-annual discussions of government's forecasts during public parliamentary hearings as a mechanism for improving the accuracy and credibility of official projections. It also discusses the potential benefits for South Africa of two alternative mechanisms, namely fiscal councils and committees of independent experts.

Key words: fiscal policy, fiscal forecasts, fiscal credibility; fiscal transparency, fiscal council

JEL: E60, H3, 61, 62, 68

---

 

 

1 Introduction

The accuracy and impartiality of the forecasts used to set the annual national budget have important implications for the credibility of a country's fiscal policy. This paper argues that the establishment of a regular forum for the discussion of such forecasts should enhance the credibility of fiscal policymaking in South Africa by contributing to improved projections and more mature, non-partisan public discussion of the fiscal outlook. Hence, it proposes the introduction of biannual discussions, during public parliamentary hearings, of the forecasts used to set the national budgets.

The paper is organised as follows. Section 2 discusses the link between the accuracy of official forecasts and the credibility of fiscal policymaking, and reviews other countries' experiences with institutional mechanisms for improving such forecasting. Against this background, Section 3 assesses the accuracy of official forecasts in South Africa from 2000/01 to 2010/11, discusses the implications of the analysis for the credibility of fiscal policymaking, and outlines our reform proposal. Section 4 concludes the paper.

 

2 Theory and international experience

2.1 Fiscal forecasts and the credibility of fiscal policy

Empirical studies show substantial inaccuracy in many policymakers' forecasts of fiscal and macroeconomic aggregates (Danninger, Cangiano & Kyobe, 2005; Frankel, 2011b; Jonung & Larch, 2006). Such inaccuracy has at least two pernicious effects. First, it is one of the manifestations of "budgetary opacity" (Hagemann, 2011:79), which has been linked to fiscal problems such as persistent deficits, excessive debt burdens and output-destabilising procyclicality (Alt & Lassen, 2006:1420-1430; Andersen & Nielsen, 2010:24-35; Jonung & Larch, 2006:505- 513). Second, by undermining government credibility, it reduces the effectiveness of fiscal policy. Modern economic theory views macroeconomic policymaking as a game-theoretic problem involving rational economic agents who base their decisions on the past, current and expected future states of their environments, including anticipated policies (Blackburn & Christensen, 1989:2). Hence, the efficacy of policy is linked to its credibility, that is, "... the degree to which people and markets believe that a policy announcement will actually be implemented and followed through" (Blanchard, 2009:632). It follows that the credibility of fiscal policy is undermined by inaccurate forecasts that contribute to large discrepancies between budget plans and fiscal outcomes.

Discrepancies of this nature could result from errors by policymakers (caused by shocks to the economy or inaccurate assessment of the economic cycle) or from deliberate misrepresentation. For example, over-optimistic GDP growth and revenue forecasts could be used to temporarily mask the effect on the budget balance of the introduction of new spending programmes.

The findings of empirical analyses are consistent with the proposition that government agencies often publish over-optimistic forecasts, especially in rules-based fiscal policymaking regimes and in the run-up to elections. Frankel (2011b) finds significant positive biases in the economic growth and budget-balance forecasts of government agencies in 33 countries in subperiods between 1985 and 2009. Furthermore, these biases had been stronger in European countries subject to the numerical rules of the Stability and Growth Pact than in countries whose fiscal authorities were not burdened by such restrictions (cf. Frankel, 2011b:545-551). Beetsma, Giuliodori & Wierts (2009) and Jonung and Larch (2006) also find evidence of systematic positive biases in the official forecasts of rules-bound European Union countries. In a study covering the years from 1995 to 2004, Brück and Stephan (2006) show that the governments of Eurozone countries had been especially likely to submit over-optimistic forecasts to the European Commission in election years. This link was confirmed in a more recent study by Merola and Pérez (2013), who analysed forecasts of 15 Eurozone governments in the period 1999 to 2007. The findings of Danninger, Cangiano and Kyobe (2005) on official forecasts in 34 developing countries are consistent with the claim that policymakers sometimes adjust their revenue forecasts upwards to accommodate spending increases without weakening budgeted fiscal balances.

In conjunction with other factors, these results have stimulated interest in institutional mechanisms to reduce or eradicate manipulation and improve the accuracy of fiscal forecasts. The most prominent remedial idea has been to involve independent fiscal agencies (also known as "fiscal councils") in official forecasting. Section 2.2 discusses this idea in more detail and summarises the available evidence on its effectiveness.

2.2 Institutional reforms: Fiscal councils and committees of independent experts

Fiscal councils are non-partisan agencies with monitoring and advisory tasks related to fiscal policy. A sizeable number of countries already involve such agencies in fiscal forecasting: according to Debrun, Gérard and Harris (2012:15), the remits of more than three-quarters of the 29 fiscal councils in existence in January 2013 included forecasting-related tasks. In fact, fiscal councils are now the most common sources of independent forecasts for fiscal policymaking purposes. While involvement of fiscal councils in budget-related forecasting is not new - the Centraal Planbureau, which was established in 1945, has been providing fiscal and macroeconomic forecasts for the Dutch government since the 1950s (Bos & Teulings, 2010:9-10) - interest in this practice has grown dramatically during the past two decades. An important reason for this development is the proliferation of rules-based fiscal policymaking frameworks:¹independent forecasting by fiscal councils is now widely seen as a method of enhancing the credibility of such regimes by countering policymakers' tendency to manipulate macroeconomic or budgetary projections (Frankel, 2011a; Jonung & Larch, 2006). A complementary task of many fiscal councils is to monitor compliance with numerical rules, which makes policymakers more accountable by enabling voters to assess their management of public finances more accurately (Debrun & Kumar, 2007). Other tasks of fiscal councils include advising policymakers on fiscal policy options; costing of budgetary initiatives; and analysis of fiscal trends, their short-term macroeconomic context and long-term sustainability implications (Calmfors & Wren-Lewis, 2011: 667-671; Debrun et al., 2013:13-17).²

Governments use the forecasts prepared by fiscal councils in a number of ways (cf. Debrun et al., 2013:16). The Belgian fiscal authorities are bound by legislation to use the macroeconomic forecasts prepared by the Federal Planning Bureau on behalf of the National Accounts Institute. In the Netherlands, by contrast, successive governments have opted to use forecasts prepared by the Centraal Planbureau (CPB) despite the absence of a legal requirement to do so. The government of the United Kingdom has also committed itself to using the fiscal and economic forecasts of the Office for Budget Responsibility as the official forecasts in its budget reports. It has retained the right to disagree with the numbers produced by the Office, but has promised to explain its reasons for doing so to parliament. Other fiscal councils undertake periodic reviews of official forecasts and forecasting methods (e.g. the Swedish Fiscal Policy Council) or prepare forecasts as benchmarks for assessing official projections (e.g. the Congressional Budget Office in the United States and the Danish Economic Council).

Several international studies have found that the involvement of fiscal councils has contributed to more accurate macroeconomic and fiscal forecasts. According to Debrun et al. (2012), the forecasts of EU countries have improved with the introduction of fiscal councils and such bodies have delivered important fiscal policy messages successfully and in a timely fashion. Frankel and Schreger (2013) find that the existence of independent fiscal agencies that produce budget forecasts at the national level reduces the likelihood that Eurozone countries might announce over-optimistic forecasts while in violation of the rule that prohibits budget deficits in excess of 3 per cent of GDP. Having analysed the impact of 29 fiscal councils in European and other countries, Debrun et al. (2013:31-34) conclude that the involvement of fiscal councils with high levels of operational or legal independence from the fiscal authorities and strong media profiles enhances precision and reduces biases in forecasts of primary balances, cyclically adjusted primary balances and real GDP growth. Debrun et al. (2013:26-29) have also found that the statistical relationship between the soundness of a country's fiscal outcomes and the existence of a fiscal council is strengthened significantly when the remit of the council includes forecasting tasks.

Adequate financial and human resources and protection against interference by politicians are important determinants of the effectiveness of fiscal councils with regard to forecasting and other tasks. Councils that generate their own fiscal and macroeconomic forecasts typically employ at least 20 professionals, but a smaller staff would suffice for assessing the forecasts produced by fiscal policymakers (Debrun et al., 2013:35). The independence of fiscal councils, which largely determines their impact on the credibility of policymaking, is enhanced by strong legal frameworks that cover issues such as appointment criteria and procedures, the terms of office of senior staff members, dismissal procedures and access to information and funding (cf. Calmfors & Wren-Lewis, 2011:665-666; Debrun et al., 2013:45-47).

It should be noted, however, that legal frameworks provide only partial protection when criticism by "fiscal watchdogs" moves government to initiate retaliatory action. The history of the Hungarian Fiscal Council, which was established in 2008 as an amply resourced body, starkly shows the vulnerability of such agencies. When the Council criticised the medium-term fiscal plans in the 2011 budget bill and other aspects of economic policy, the Hungarian government amended the fiscal responsibility law that governed its work, changed its composition and cut its funding and personnel drastically (Curristine, Harris & Seiwald, 2013:21). The agency survived, but the subsequent inadequacy of its resources and pliancy of its approach moved Kopits (2011:13) - a renowned economist who served as its first chairperson but resigned in protest against the 2011 reforms - to describe these events as the "de facto termination of Hungary's Fiscal Council". In similar vein, the publication of controversial reports about the state of the economy and public finances, as well as the cost of the war in Afghanistan, resulted in a marked reduction in the budget of the Canadian Parliamentary Budget Office in 2010, while the Swedish Minister of Finance repeatedly belittled the work of the Fiscal Policy Council in response to criticisms of fiscal policies in 2009 and 2010 (Calmfors & Wren-Lewis, 2011:676, 683). Ultimately, the viability and de facto independence of fiscal councils depend on the value attached to policymaking transparency by governments and voters.

As indicated previously, fiscal councils often have important transparency-enhancing roles in countries with numerical fiscal rules. Fiscal councils could of course perform similar roles in the absence of numerical rules. Well-known examples of fiscal councils that were created before the adoption of numerical rules and functioned for long periods in discretionary policymaking environments are the German Council of Economic Experts and the Congressional Budget Office (CBO) in the United States (Debrun et al., 2013:22). The tasks of the CBO include costing new legislative proposals, re-estimation of the budgets of incumbent administrations based on its own macroeconomic forecasts and other assumptions, and advising Congress on various fiscal matters (Debrun, Hauner & Kumar, 2009:68).

Chile instituted a different model in 2001, when the government adopted a Fiscal Responsibility Law and a rule that prescribed a structural budget surplus of 1 per cent of GDP (cf. Hagemann, 2011:90-91). Instead of establishing a formal fiscal council, the government appointed panels of experts to provide key inputs for the fiscal authorities' macroeconomic and fiscal forecasts in the form of projections of copper prices³ and the growth rates of capital, labour and total factor productivity. The two bodies - the twelve-member Advisory Committee on Trend GDP and the sixteen-member Advisory Committee for the Reference Copper Price - are chosen by the Chilean minister of finance from local macroeconomists and copper market experts (Schmidt-Hebbel, 2012:10-11). They attend one technical meeting every year, usually in July or August, and submit individual projections some weeks later. After discarding the two extreme values, the fiscal authorities use the averages of these projections to calculate the output gap, the structural levels of copper revenues and total tax revenues, and the level of expenditure required to reach the surplus target (cf. Schmidt-Hebbel, 2012:10). As pointed out by Hagemann (2011:91), the Ministry of Finance retains a critical role in the process, having remained responsible for the method of determining the output gap. The Chilean parliament monitors compliance with the rule and the minister of finance has to provide an explanation in a parliamentary hearing when the target is missed. The members of the two committees do not communicate directly with the press or with parliament.

Assessments of this aspect of Chile's policymaking framework have varied: Frankel (2011a:38-39) argues that it has helped the country to avoid over-optimistic forecasts and to achieve countercyclical fiscal outcomes, while Hagemann (2011:91) claims that regular overestimation of the output gap and copper prices has contributed to excessive spending levels and pro-cyclical outcomes. Chile changed the structural balance rule and established a fiscal council in April 2013 (Debrun et al., 2013:13), but the two advisory committees have retained their forecasting responsibilities.

The impact of the involvement of fiscal councils in forecasting is of considerable interest to South Africa following the establishment of the Parliamentary Budget Office (PBO), which has been in operation since 2014. In fact, parliamentary budget offices in several other countries undertake forecasting-related activities and some experts regard these as core functions of such agencies (cf. Johnson & Stapenhurst, 2008). We now turn to the South African situation.

 

3 South Africa

3.1 The accuracy of the National Treasury's forecasts

The following analysis of the accuracy of the National Treasury's forecasts in the fiscal years from 2001/02 to 2010/11 focuses on three questions:

• How accurate are the forecasts of key macroeconomic aggregates?

• How accurate are the forecasts of fiscal aggregates?

• Has the accuracy of the forecasts improved or deteriorated over time?

3.1.1 Forecasts of macroeconomic aggregates

Van der Wath (2013) compares the accuracy of the macroeconomic forecasts of the Bureau of Economic Research (BER) at the University of Stellenbosch with those of the National Treasury, the International Monetary Fund (IMF), the consensus forecast of the private-sector and academic economists participating in the Economist of the Year competition sponsored by Sake24, and two mechanical benchmarks (a naïve forecast that leaves the last recorded value unchanged over the forecast period and a trend-based forecast). The comparison is between the forecast values in March and October and the first official figures (known as "first vintage data") published for five variables: the GDP growth rate, the inflation rate, the prime interest rate, the rand-dollar exchange rate, and the gold price. "First-vintage" official data are subject to further, and sometimes significant, revision, but remain useful as a basis for comparing the accuracy of forecasts. The comparison is based on two measures of accuracy, namely the mean absolute error (MAE) and the root mean square error (RMSE). These measures, which denote the absolute error in respect of the * variable in period tby L_ti, are defined as follows:

For each variable, the number of observations depends on the number of future years for which forecasts were made and on the year in which forecasting commenced. The numbers of observations for the GDP growth rate were 40 for four-year forecasts (undertaken by the BER and the IMF), 82 for three-year forecasts (undertaken by the BER and National Treasury) and 42 for two-year forecasts (undertaken by the BER and participants in the Economist of the Year competition). The corresponding numbers of observations for the inflation rate were 38, 69 and 39, respectively.

On average, the BER was 0.5 percentage points off target in forecasting the GDP growth rate of the nearest year, while the average errors of the IMF, National Treasury and participants in the Economist of the Year competition were all close to 0.6 percentage points. When all the forecasting periods are taken into consideration, the National Treasury's economic growth forecasts in terms of the MAE outperformed those of the BER by about 2 per cent. On balance, therefore, the National Treasury's growth forecasts were slightly closer (by 2 per cent) to the eventual outcome than the BER's. Van der Wath (2013:7) summarises his findings as follows:

All the forecasters' models come remarkably close to each other in terms of accuracy, and ... in the first two years all of them perform significantly better than a purely trended or naïve forecast. For periods longer than that they lose their competitive advantage (in terms of pure accuracy).

The MAE of the National Treasury's inflation forecasts is close to that of the BER, but both institutions missed the spikes in inflation in 2002 and 2008. In fact, all the forecasting models exhibit an inability to anticipate directional changes (Van der Wath, 2013:12). Although Van der Wath conducts pairwise comparisons of the projections of the BER and other forecasters (each with its own sample size), the findings suggest that neither the BER nor the IMF nor participants in the Economist of the Year competition outperformed the National Treasury as far as the accuracy of macroeconomic forecasts is concerned.

3.1.2 Forecasts of fiscal aggregates

We now turn to the accuracy of the National Treasury's forecasts, in current prices of GDP (Y), national budget revenue (T, which includes tax and non-tax revenue), national budget spending (G) and the national budget balance (B). The analysis identifies three sets of figures.

• The subscript '-1' (e.g. T-i) refers to forecasts for a particular budget year made before the start of that budget year, that is, the data in the Budget Review when the minister of finance presents the budget to parliament in February. We refer to these as budget estimates.

• The subscript '0' (e.g. T₀) refers to the revised forecasts for a particular budget year that are compiled during the course of that budget year and appear in the Budget Review for the next budget year. We refer to these as revised estimates.

• The subscript 'f (e.g. T_f) refers to final official figures. We refer to these as final figures.

Following Beetsma et al. (2011), we distinguish three types of forecasting errors. Implementation error (IE) is the difference between the revised estimate and the budget estimate. It indicates the extent to which the original budget was changed during the fiscal year to incorporate new information about the economy, expenditure pressures and other factors. Forecast error (FE) is the difference between the final figure and the budget estimate. The importance of this indicator, which is a more demanding measure of forecasting accuracy than Van der Wath's "first vintage comparison", stems from the reality that final figures are the most common yardsticks in retrospective assessments of fiscal policy outcomes. Revision error is the final figure minus the revised estimate; hence, it indicates the discrepancy that arose after the budget figures had been adjusted to account for updated economic and other information. The following analysis focuses on forecast error.

Figure 1 (a) shows that the final outturn significantly exceeded budgeted revenue in every year from 2001/02 to 2007/08. Efforts by the South African Revenue Service (SARS) to improve tax compliance contributed markedly to this trend (South African Revenue Service, 2012). Actual expenditure also exceeded budgeted outlays in each of these years,⁴ but the revenue overruns consistently exceeded the spending overruns (cf. Figure 1(b)). Hence, the budget balance turned out more favourably than the budgeted figure for seven consecutive years (cf. Figure 1(c)). Large revenue under-collections caused by the recession from the fourth quarter of 2008 to the second quarter of 2009 reversed this trend in 2008/09 and 2009/10.

Errors in the forecasted budget-balance/GDP ratios mainly reflect incorrect revenue projections. The average contributions to the forecast errors in the budget-balance/GDP ratios were as follows: revenue errors - 72.1 per cent, expenditure errors - 22.1 per cent, and GDP errors - 5.8 per cent (cf. Table 1). In several years, the effects of the errors on the final budget-balance/GDP ratio partly offset each other; on average, government spending turned out to be lower than projected, whereas revenue and GDP were higher. (Under-spending is calculated here as the difference between the final figures and the adjusted budget figures published in October every year. Expenditure outcomes would exceed projections if the original budget figures published in February were taken as baseline numbers.)

Given the obvious correlation between GDP and government revenue, it is important to establish the extent to which GDP forecast errors (Y_f - Y_-1) contributed to revenue forecast errors (T_f - T_-1). A simple regression generates the following:

Although there is a relatively high correlation of 79.3 per cent between the two sets of forecast errors, the regression indicates that (Y_f - Y_-1) cannot explain a significant part of the forecast error in T. This suggests that incorrect projection of revenue was an important driver of the extent of the error in forecasting T.

Lines (6) and (7) in Table 1 show the largest annual overestimates and underestimates of the three variables, measured as the differences between the final figures and the original budget estimates. Such errors have been large in some years: for example, revenue was underestimated by roughly R42 billion in 2005/06, expenditure by R24.9 billion in 2008/09, and nominal GDP by R120.3 billion in 2002/03. The forecast errors of nominal GDP exhibited a particularly clear bias: GDP was underestimated in 10 of the past 11 years. Estimated budget balances were often revised downwards as nominal GDP figures exceeded expectations. Upward revisions to GDP in subsequent years often reinforced such adjustments and produced even smaller budget-balance outturns.

By international standards, official South African forecasts of budget balances are relatively accurate. In a review of the accuracy of official forecasts in 33 countries from the first available projections up to 2009, Frankel (2011b:541) finds that South Africa's deficit forecast errors are similar in magnitude to the international average. Whereas South Africa tends to overestimate the size of the deficit, however, the average tendency for all 33 countries is to underestimate the deficit. During the past decade, the MAE for the South African national budget balance was 26 per cent smaller than the average of 1.8 percentage points for 14 European Union countries (the so-called EU15 excluding Luxembourg).⁵ The margin of error for South Africa remains 23 per cent lower than the average for these countries even if 2009 is regarded as an outlier and ignored.

New Zealand, which has a transparent and credible budget process, replaced South Africa at the top of the Open Budget Index ranking compiled by the International Budget Partnership (2013) in 2012.⁶ As such, New Zealand is a useful benchmark for forecasting accuracy. A comparison of the MAEs and the associated standard deviations of one-year-ahead projections of budget balances for the period 2000/01 to 2010/11 suggests that South Africa's official forecasts were more accurate than those of New Zealand. The MAE for New Zealand was 2.3 percentage points (with a standard deviation of 2.6), whereas the average for South Africa was 1.4 percentage points (with a standard deviation of 1).⁷

3.1.3 Trends in the accuracy offorecasts from 2001/02 to 2010/11

Figure 2 shows the annual values of the absolute forecast error of the budget-balance/GDP ratio from 2000/01 to 2010/11. The positively sloped trend line suggests that the extent of the forecast errors increased over the decade. Government revenue and spending exhibit similar trends, even when data for the recession year 2009/10 are ignored. The upward trend may reflect the inability of most models to predict turning points in the business cycle, however, and the small number of data points limits the significance of the analysis. Be that as it may, there is no evidence of improvements in the forecasting accuracy of the National Treasury in the period under review.

 

 

As was pointed out earlier, a change in the nature of the forecast error of the budget-balance/GDP ratio accompanied this increase in its size: while the actual deficits were smaller than the forecasted ones in the seven years from 2001/02 to 2007/08, bigger-than-forecasted deficits occurred in 2008/09 and 2009/10, which may in part reflect the unpredictability (and severity) of the recession. We discuss the credibility implications of these developments in Section 3.2.

3.2 Implications for the credibility of fiscal policy

South Africa is in an enviable position as far as fiscal policy credibility is concerned. Two factors have contributed to this state of affairs. The first is the country's good fiscal record: the fiscus never once experienced crises of non-sustainability during the past century, despite major changes in the international and domestic political and economic environments (cf. Calitz, Du Plessis & Siebrits, 2014:56). Burger, Stuart, Jooste & Cuevas (2012) show that the South African government maintained a sustainable fiscal policy since 1946 by reducing the primary deficit or increasing the primary surplus in response to rising debt. While there is some evidence of pro-cyclical spending during the boom years of the mid-2000s, South Africa has generally avoided destabilising pro-cyclical fiscal policy during the past three decades (Du Plessis, Smit & Sturzenegger, 2007). The second factor is the overall quality of fiscal management, including the high degree of transparency: the International Budget Partnership (2013) ranked South Africa's budget process as the most transparent among 100 countries in 2010 and the second most transparent in 2012. A particularly important element of fiscal management in South Africa is the Public Finance Management Act (PFMA) of 1999 (South Africa, 1999), which aims to address various dimensions of the accountability aspect of fiscal transparency. Section 28 of the Act stipulates that all national and provincial budgets must contain multi-year projections of revenue and expenditure and that the national budget should outline the macroeconomic projections underlying the estimates. The introduction in 1998/99 of the Medium-Term Expenditure Framework (MTEF), which consists of three-year rolling budgets for the national and provincial governments, accompanied by detailed explanations of the fiscal and broader macroeconomic policy stance, gave effect to this section.

Safeguarding the credibility of fiscal policy is an ongoing process. The South African National Treasury has continued to make fiscal reporting more transparent: among other things, a debt management report was published in 2015 and a fiscal risks report will be forthcoming in 2016 (National Treasury, 2015:21). Furthermore, since 2012/13, the government has succeeded in staying within its spending targets, which have taken the form of nominal expenditure ceilings for the main budget (National Treasury, 2014:21). From a credibility point of view, it remains pertinent to ask how reliable the budgets presented by the Minister of Finance are as indicators of the professed fiscal policy stance.

Section 3 showed that the actual budget balances in the period from 2000/01 to 2010/11 often differed markedly from the budgeted figures. Moreover, the absolute average forecast error was 1.35 percentage points of GDP. To put this in perspective: an increase in the budget-deficit/GDP ratio of 1.35 percentage points from one year to the next would be regarded as a significant fiscal stimulus. An average error of this magnitude over an eleven-year period could raise doubts about the ability to apply an active countercyclical fiscal policy and, hence, about the reliability of such policy announcements in the annual budget.

Fortunately for the National Treasury, the biggest revenue, spending and GDP forecast errors have never coincided. Table 2 presents counterfactual calculations for 2010/11. It shows the difference between the actual deficit/GDP ratio (column 2) and the deficit/GDP ratio that would have resulted if each of the following had materialised: the average historical error (column 3), the biggest historical errors that would jointly have yielded the lowest deficit ratio (column 4), and the biggest historical errors that would jointly have yielded the highest deficit ratio (column 5). The exercise gives rise to two conclusions. First, if the pattern follows the average path - which entails the components of the aggregate error largely neutralising each other - the forecast error in the budget-balance/GDP ratio and the risk to the credibility of fiscal policy is modest. If all the biggest errors of the past had occurred in the same year, however, the actual deficit would have been underestimated by 3.8 percentage points (the difference between -1.1% and -4.9%) or overestimated by 3.3 percentage points (the difference between -8.2% and -4.9%). Forecast errors of this magnitude would damage fiscal credibility.

Section 3.1.2 presented evidence of a bias towards underestimation of actual revenue. Consistent underestimation of revenue might be regarded as a useful instrument for managing expenditure expectations, achieving budget surpluses and building buffers to counter future fiscal shocks.

Yet such a strategy would not be risk-free. The snag lies in the probable dynamic response when it is implemented repeatedly. When revenue exceeds the forecast in six out of seven years, as happened in South African between 2001/02 and 2007/08, the spending ministries may well get the impression that the fiscal authority is deliberately underestimating revenue. This would change the dynamics of the budgeting game: champions of higher expenditure would become increasingly certain that the National Treasury was sounding false alarms about budget constraints. Persistent underestimation of revenue can also erode fiscal credibility.

It follows that measures to improve the accuracy and impartiality of official forecasts should safeguard and perhaps also boost the credibility of fiscal policy in South Africa. Such measures could be of two types: initiatives to improve the skills of the forecasters and the quality of the models they use, and the creation of structures to discourage attempts by policymakers to manipulate forecasts for political or other purposes. Although the first type is by no means less important, this paper focuses on measures of the second type.

3.3 Proposal for a pre-budget parliamentary hearing in South Africa

The experiences of other countries suggest that fiscal forecasting can be improved by the involvement of independent experts (in the presence or absence of numerical rules) and that several models exist for doing this. The South African context should be kept in mind when considering the applicability of these experiences.

At present, South Africa has flexible ("soft") fiscal rules in the form of expenditure ceilings for the main budget, but lacks a fiscal council. However, the Money Bills Amendment Procedure and Related Matters Act 9 of 2009 (South Africa, 2009), the aim of which is to strengthen parliamentary oversight of budget-related matters, established the Parliamentary Budget Office to "... provide independent, objective and professional advice and analysis to Parliament on matters related to the budget and other money bills" (cf. section 15 of the Act). The Office has been in operation since 2014. Forecasting-related tasks are compatible with the remit of the Parliamentary Budget Office outlined in section 15 of the Money Bills Amendment Procedure and Related Matters Act, but are not mentioned explicitly. As was indicated earlier, however, much expertise and other resources are required to produce high-quality forecasts. Given that it will take time for the Parliamentary Budget Office to build its capacity and to establish itself in the policy arena, we believe that an appropriate starting-point for using independent forecasts in fiscal policymaking in South Africa is a model that involves the Office but draws on existing forecasting expertise to strengthen the oversight function of the legislature (this model is reminiscent of the earlier Chilean one in its emphasis on using existing expertise as an alternative to creating new formal structures).

Given South Africa's good fiscal record and transparent budget process, the major benefits of the involvement of independent experts in forecasting are likely to be the enhancement and protection of the credibility of fiscal policy. The purpose of such involvement should be to generate benchmarks for the assessment and opportunities for discussion of official forecasts: the case for delegating the responsibility for producing forecasts for fiscal policymaking seems weak, especially in the light of the evidence in Section 2 that the forecasting accuracy of the National Treasury compares well with that of private sector economists and other reputable organisations.

In view of these considerations, we believe that the introduction of parliamentary public hearings on the macroeconomic forecasts that inform the budget could incentivise more transparent and credible policymaking over time. This reform would build upon South Africa's successful transparency-based discretionary fiscal regime, as described by Siebrits and Calitz (2004:781). By giving a wide range of organisations an opportunity to make inputs into the budget-making process, it could promote more inclusive policymaking and accountability.⁸Finally, it would not require significant financial setup costs, nor would it create a new institution that might attract the public sector's most talented economists away from government. Given South Africa's small skilled workforce, this is a significant consideration.

The hearings could take place bi-annually: in January (one month before the Budget Speech) and in September (one month before the Medium-Term Budget Policy Statement). Government, represented by the National Treasury, could open the proceedings by presenting a baseline scenario of the major determinants of the fiscal framework. In line with the medium-term expenditure framework, government would be required to present a four-year forecast (in-year plus three years ahead) of real and nominal GDP, CPI and GDP inflation, and tax revenue. Gross taxes would be presented in aggregate, broken up into the major components: personal income tax, corporate income tax, valued-added tax and customs duties. The tax forecasts should reflect a no-policy change scenario, with tax brackets adjusted upwards for CPI inflation.

We propose that any recognised institution be allowed to send its forecasts and assumptions to parliament, with the deadline set one week prior to the hearing. This could include compulsory submissions to the public debate by certain established interest groups even if they submit an empty document. This is to avoid "free riding" on criticism after the event without having grappled with the issues in advance. Parliament would then invite a broad range of representatives to present their forecasts in parliament, as is currently the case with the post-budget hearings. One hearing could include, for example, the South African Chamber of Commerce and Industry (SACCI), the Congress of South Africa Trade Unions (Cosatu), the Bureau for Economic Research (BER), individual academics, and private economic consultancies. Government would then respond briefly to the other speakers.

The following day, parliament would be required to publish the minutes of the proceedings, including the areas of disagreement and consensus. All forecasts received by parliament could also be published in table form, by variable and by institution. The relative accuracy of the forecasts could be calculated by the Parliamentary Budget Office and presented at the next round of hearings. Over time, the accuracy of the forecasts of different institutions could be estimated and ranked. This simple feedback mechanism should incentivise better forecasts.

The first few sets of hearings could be used to test the usefulness of the reform. One quantitative measure of the success of the reform should be less biased fiscal projections over the medium term. Even without a significant improvement in the accuracy of the forecasts, however, the hearings should promote more mature, non-partisan public discussion of the fiscal outlook. Over time, the hearings could be formalised and enshrined in the Public Finance Management Act (1999), and perhaps integrated more closely with the work of the Parliamentary Budget Office. Furthermore, should government wish to introduce a structural budget-balance rule, the list of required variables could be expanded to include potential GDP.

An obvious question is whether parliament should set the macro fiscal framework after having had the public debate, or whether the results should become mere "advice" to the National Treasury. One could also think of a process whereby the debate presents a range of opinions or even a definite macro framework for the budget, with the Minister of Finance having to explain any significant deviation from it when presenting the budget. Rather than setting these rules at the outset, we propose a "learning-by-doing" approach to developing a platform for constructive discussion of assumptions, policy goals, forecasts and an understanding of the transmission mechanism of fiscal policies.

 

4 Conclusion

South Africa has a history of fiscal sustainability and transparency that was achieved without "hard" numerical rules or a fiscal council. The government has continued to strengthen its fiscal framework in various ways, among others by adopting expenditure ceilings for the main budget and by expanding the availability of information on aspects of fiscal policy. To enhance further the credibility of government 's macro-fiscal projections, and promote public debate on the fiscal outlook, we propose the introduction of bi-annual public parliamentary hearings on the forecasts. This reform builds on the successes of the past, and reflects South Africa's political dynamics. It puts some distance between the generators and users of forecasts, which would enhance the credibility of fiscal policy. Depending on the usefulness of this reform, government could formalise the process in future by writing it into law and by aligning it more closely with the work of the Parliamentary Budget Office. Hence, a fiscal council may or may not eventually be the result of such a "learning-by-doing" process.

 

References

ALT, J.E. & LASSEN, D.D. 2006. Fiscal transparency, political parties, and debt in OECD countries. European Economic Review, 50:1403-1439.         [ Links ]

ANDERSEN, A.L. & NIELSEN, L.H.W. 2010. Fiscal transparency and procyclical fiscal policy. EPRU Working Paper Series 2010-01. Copenhagen: University of Copenhagen (Economic Policy Research Unit).         [ Links ]

ARBATLI, E. & ESCOLANO, J. 2012. Fiscal transparency, fiscal performance and credit ratings. IMF Working Paper WP/12/156. Washington, D.C.: The International Monetary Fund.         [ Links ]

BEETSMA, R.M.W.J., BLUHM, B., GIULIODORI, M. & WIERTS. P. 2011. From first-release to ex- postfiscal data: exploring the sources of revision errors in the EU. Discussion Paper TI2011-080/2. Amsterdam: The Tinbergen Institute.         [ Links ]

BEETSMA, R.M.W.J., GIULIODORI, M. & WIERTS. P. 2009. Planning to cheat: EU fiscal policy in real time. Economic Policy, 24(60):753-804.         [ Links ]

BLACKBURN, K. & CHRISTENSEN. M. 1989. Monetary policy and policy credibility: Theories and evidence. Journal of Economic Literature, 27(1):1-45.         [ Links ]

BLANCHARD, O.J. 2009. Macroeconomics (5^th ed.) Upper Saddle River: Pearson.         [ Links ]

BOS, F. & TEULINGS, C. 2010. CPB and Dutch fiscal policy in view of the financial crisis and ageing. Proceedings of the Conference on Independent Fiscal Institutions, 18-19 March, Budapest (Hungary).         [ Links ]

BRÜCK, T. & STEPHAN, A. 2006. Do Eurozone countries cheat with their budget deficit forecasts? Kyklos, 59(1):3-15.         [ Links ]

BURGER, P., STUART, I., JOOSTE, C. & CUEVAS. A. 2012. Fiscal sustainability and the fiscal reaction function for South Africa: Assessment of the past and future policy applications. South African Journal of Economics, 80(2):209-227.         [ Links ]

CALITZ, E., DU PLESSIS, S.A. & SIEBRITS, F.K. 2014. Fiscal sustainability in South Africa: Will history repeat itself? Journal for Studies in Economics and Econometrics, 38(3):55-78.         [ Links ]

CALMFORS, L. & WREN-LEWIS, S. 2011. What should fiscal councils do? Economic Policy, 26(68): 649-695.         [ Links ]

CURRISTINE, T., HARRIS, J. & SEIWALD. J. 2013. Case studies of fiscal councils - functions and impact. IMF Policy Paper. Washington, D.C.: The International Monetary Fund.         [ Links ]

DANNINGER, S., CANGIANO, M. & KYOBE, A. 2005. The political economy of revenue-forecasting: Experience from low-income countries. IMF Working Paper WP/05/2. Washington, D.C.: The International Monetary Fund.         [ Links ]

DEBRUN, X, GÉRARD, M. & HARRIS, J. 2012. Fiscal policies in crisis mode: Has the time for fiscal councils come at last? Paper presented at the fourth annual meeting of OECD parliamentary budget officials and independent fiscal institutions, 23-24 February, Paris (France).         [ Links ]

DEBRUN, X., HAUNER, D. & KUMAR, M.S. 2009. Independent fiscal agencies. Journal of Economic Surveys, 23(1):44-81.         [ Links ]

DEBRUN, X., KINDA, T., CURRISTINE, T., EYRAUD, L., HARRIS, J. & SEIWALD, J. 2013. The functions and impact of fiscal councils. IMF Policy Paper. Washington, D.C.: The International Monetary Fund.         [ Links ]

DEBRUN, X. & KUMAR, M.S. 2007. Fiscal rules, fiscal councils and all that: Commitment devices, signaling tools or smokescreens? Proceedings of the Bank of Italy workshop, 29-31 March, Perugia (Italy): 479-512.         [ Links ]

DU PLESSIS, S.A., SMIT, B.W. & STURZENEGGER, F. 2007. The cyclicality of monetary and fiscal policy in South Africa since 1994. South African Journal of Economics, 75(3):391-411.         [ Links ]

EUROPEAN COMMISSION. 2014. Multilateral economic coordination and surveillance. Available at: http://ec.europa.eu/economy_finance/economic_governance/sgp/convergence/index_en.htm [accessed April 2014].         [ Links ]

FRANKEL, J.A. 2011a. A solution to overoptimistic forecasts and fiscal procyclicality: The structural budget institutions pioneered by Chile. HKS Faculty Research Working Paper Series RWP11-012, Cambridge, MA: Harvard University (John F. Kennedy School of Government).         [ Links ]

FRANKEL, J.A. 2011b. Over-optimism in forecasts by official budget agencies and its implications. Oxford Review of Economic Policy, 27(4):536-562.         [ Links ]

FRANKEL, J.A. & SCHREGER, J. 2013. Over-optimistic official forecasts in the Eurozone and fiscal rules. Review of World Economics, 149(2):247-272.         [ Links ]

HAGEMANN, R. 2011. How can fiscal councils strengthen fiscal performance? OECD Journal: Economic Studies, 2011/1:75-98.         [ Links ]

INTERNATIONAL BUDGET PARTNERSHIP. 2013. Open budget index 2012. Available at: http://internationalbudget.org/what-we-do/open-budget-survey/ [accessed August 2013].         [ Links ]

JOHNSON, J.K. & STAPENHURST, R. 2008. Legislative budget offices: International experience. Published in: Stapenhurst, R., Pelizzo, R., Olson, D. & Von Trapp, L. (eds.). Legislative oversight and government accountability: A world perspective. Washington, D.C.: The World Bank: 141-158.         [ Links ]

JONUNG, L. & LARCH, M. 2006. Improving fiscal policy in the EU: The case for independent forecasts. Economic Policy, 21(47):492-534.         [ Links ]

KOPITS, G.2011. Independent fiscal institutions: developing good practices. OECD Journal on Budgeting, 11(3):1-18.         [ Links ]

MEROLA R. & PÉREZ, J.J. 2013. Fiscal forecast errors: Governments versus independent agencies? European Journal of Political Economy, 32:285-299.         [ Links ]

NEW ZEALAND TREASURY. Various dates. Budget policy statement. Available at: http://www.treasury.govt.nz/budget [accessed July 2014].         [ Links ]

SCHAECHTER, A., KINDA, T., BUDINA, N. & WEBER, A. 2012. Fiscal rules in response to the crisis - toward the 'next-generation' rules. A new dataset. IMF Working Paper No. WP/12/187. Washington, D.C.: The International Monetary Fund.         [ Links ]

SCHMIDT-HEBBEL, K. 2012. Fiscal policy for commodity exporting countries: Chile's experience. Documentos de Trabajo No. 415. Santiago: Pontificia Universidad Católica de Chile.         [ Links ]

SIEBRITS, F.K. & CALITZ, E. 2004. Should South Africa adopt numerical fiscal rules? South African Journal of Economics, 72(4):759-783.         [ Links ]

SOUTH AFRICA. 1999. Public Finance Management Act, Act 1 of 1999 (as amended). Available at: http://www.treasury.gov.za/legislation/PFMA/act.pdf [accessed August 2012].         [ Links ]

SOUTH AFRICA. 2009. Money Bills Amendment Procedure and Related Matters Act 9 of 2009. Available at: http://www.sals.gov.za/docs/pubs/moneybills.pdf [accessed August 2012].         [ Links ]

SOUTH AFRICAN NATIONAL TREASURY. Various dates. Budget review. Pretoria. Available at: http://www.treasury.gov.za/documents/national%20budget/ [accessed November 2015].         [ Links ]

SOUTH AFRICAN RESERVE BANK. Various dates. Quarterly bulletin. Pretoria. Available at: https://www.resbank.co.za/Publications/QuarterlyBulletins/Pages/QuarterlyBulletins-Home.aspx [accessed November 2015].         [ Links ]

SOUTH AFRICAN REVENUE SERVICE (SARS). 2012. Annual report 2011-2012. Pretoria. Available at: http://www.sars.gov.za/AllDocs/SARSEntDoclib/AnnualReports/SARS-AR-16%20-%20Annual%20Report%202011-2012.pdf [accessed November 2015].         [ Links ]

VAN DER WATH, N. 2013. Comparing forecasts. Unpublished research note. Stellenbosch: University of Stellenbosch (Bureau for Economic Research).         [ Links ]

 

 

Accepted: February 2016

 

 

1 The number of countries with rules-based fiscal regimes increased from 5 in 1990 to 76 in March 2012 (Schaechter, Kinda, Budina & Weber, 2012:10).
2 It could be argued that the appellation "fiscal council" should be restricted to agencies whose mandates are limited to these fiscal policy-related tasks (for example, the United Kingdom's Office for Budget Responsibility and Korea's National Assembly Budget Office). This paper, however, follows the common practice of extending the appellation to agencies with broader economic policy remits that include some of these tasks, such as the Centraal Planbureau in the Netherlands and the Federal Planning Bureau in Belgium.
3 This reflects the role of copper as Chile's primary export product.
4 Here we encounter the problem of moving targets. Because budgets were increased at the time of the medium-term budget in October, comparisons look different if the adjusted (October) estimate becomes the basis. For purposes of consistency and reliability of pronouncements on the envisaged impact of fiscal policy at the time of the annual budget, we maintain our focus on the latter.
5 This finding is derived from the authors' calculations, based on South African National Treasury (various dates) budget data and the annual Stability and Convergence Programme reports of European Union countries (cf. European Commission, 2014).
6 The Open Budget Index measures the transparency and accountability of government budgeting in 70 countries. Arbatli and Escolano (2012), among others, use it as a measure of fiscal credibility.
7 Authors' own calculations from budget data of the New Zealand Treasury (various years).
8 It might be asked whether the Monetary and Fiscal Policy Chamber of the National Economic Development and Labour Council (Nedlac) does not constitute such a forum. The Chamber cannot perform this role, however, because Nedlac functions as a consensus-seeking mechanism in which vested interests shape the mandates of the participants. This inhibits transparent presentation of analyses and public debate in the national interest.