This case study explains how Korea developed performance evaluation systems for government expenditure programs through two major reforms: the Self-Assessment of Budgetary Programs (SABP) introduced in 2005 and the Integrated Evaluation System on Government Programs (IESGP) introduced in 2016. Both systems were created by the Ministry of Strategy and Finance (MOSF) to improve accountability, strengthen performance management, and allocate budgets based on program results. Under SABP, ministries evaluated one third of their programs each year using a standard checklist, after which MOSF reviewed and adjusted the scores. SABP revealed several issues, including uneven program coverage, inflated self-evaluations, and weak connections between performance scores and budget decisions. To resolve these problems, MOSF introduced IESGP, which evaluates all programs annually, integrates previously separate evaluation systems, incorporates meta-assessment, and applies clearer incentives and penalties. The overall lesson is that performance evaluation must be paired with adequate capacity, credible incentives, and transparent budget links to drive improvement in government programs.
#budgeting #expenditure #evaluation
This case study aims to show the whole cycle of policy adoption, implementation, evaluation, and feedback of the Korean performance evaluation systems: the Self-Assessment of Budgetary Programs (SABP) in 2005 and the Integrated Evaluation System on Government Programs (IESGP) in 2016. The main objectives of these two policies were to evaluate the performance of all government expenditure programs and assign budget across the programs based on their performance scores. The cases are chosen because they are closely related to the fundamental question in budgetary reform and development: On what basis should governments distribute public resources across diverse projects and programs? The answer is to assign budget across programs based on their performance. Both SABP and IESGP are practical tools of the Ministry of Strategy and Finance (MOSF) of Korea to assess the performance of all expenditure programs that the central ministries launch. Through SABP and IESGP, the central ministries have evaluated themselves on the performance of their own expenditure programs. SABP was an initial endeavor of MOSF in this policy area, and IESGP was a reformative effort of MOSF to improve the problems found with SABP since SABP was implemented.
The central government of Korea introduced fundamental budgetary reforms in the late 1990s and the early 2000s under the name of the Four Major Fiscal Reforms. The reforms aimed to promote the effectiveness of performance-oriented management of expenditure programs and enhance long-term fiscal sustainability of Korea. The reforms comprise the medium-term expenditure framework known as the National Fiscal Management Plan, the Top-Down Budgeting System, the Performance Management System, and the Digital Budget Information System. The performance evaluation systems of government expenditure programs are related to the Performance Management System.
The adoption of these performance evaluation systems started in 1999 when the Ministry of Planning and Management renamed as the Ministry of Strategy and Finance launched the pilot projects with 16 selected central ministries. The ministry introduced a series of subsysne items such as the Performance Goal Management of Budgetary Programs in 2003, the SABP in 2005, and the In-Depth Evaluation of Budgetary Programs in 2006. The main objectives of the SABP were to link the performance of government expenditure programs with the amount of budget assigned to the individual expenditure programs. The MOSF amended SABP and adopted the IESGP in 2016 (Park, 2012).
In order to provide the legal foundation for performance evaluation of government expenditure programs, the central government of Korea enacted the Framework Act on Government Performance Evaluation (Act No. 6347, 2001), the National Finance Act, and the National Act Enforcement Decree. Based on these acts, MOSF evaluates the performance of all expenditure programs launched by the central government of Korea (Liu, 2017).
The central government of Korea introduced SABP as a practical tool to evaluate the performance of expenditure programs. The government benchmarked PART of the United States. Both SABP of Korea and PART of the United States are “self-assessment” programs, in which the ministries that are responsible for the expenditure programs evaluate the performance of their own expenditure programs. SABP was designed to review all major expenditure programs launched by the central government over a three-year cycle, one-third of the programs per year. SABP benchmarks PART but added some innovations. Different from PART, the Korean SABP system encourages active participation of external experts in the evaluations. For this purpose, each ministry organizes a self-assessment committee in which external experts participate. The consultation from the external experts is reflected in the ministerial evaluations. After the ministries evaluate the performance of their expenditure programs, the Ministry of Strategy and Finance double-checks the ministerial self-assessments and uses the performance information for budgeting.
One of the major objectives of these self-assessment systems was to provide performance information so that MOSF might assign budget to all expenditure programs based on their performance. After its introduction in 2005, the central government of Korea found that SABP should be amended in several ways. The central government of Korea updated SABP and introduced IESGP in 2016. The In-Depth Evaluation of Budgetary Programs was also adopted to examine the performance of government expenditure programs comprehensively. External experts were invited to examine the performance of the government expenditure programs with analytical and scientific methods (Park, 2012).
The SABP was designed to evaluate the annual performance of one-third of all expenditure programs launched by the central government. Eventually, the performance of all government expenditure programs was to be assessed over a three-year cycle. MOSF provided a checklist for evaluation for all central ministries. Based on the checklist, all central government ministries conducted standardized assessments of the performance of the expenditure programs for which they were responsible. MOSF double-checked the self-assessment results submitted by the government agencies and determined the final annual performance evaluation results. MOSF also added recommendations on how to improve the performance of the expenditure programs of the central ministries. MOSF used the final performance results in budgeting by reexamining and adjusting the size and priority of expenditure programs according to the performance results. MOSF intended to restructure the existing expenditure programs and even terminate some expenditure programs based on their performance results. In sum, the primary objective of SABP was to make central government agencies accountable for the performance of their programs (Park, 2012).
The procedures of SABP are summarized as follows. First, all central government agencies choose one-third of their expenditure programs to be evaluated by SABP. They consult MOSF for the selections. MOSF provides a guideline for the selection of target programs, reflecting “duplication or overlapping with other programs, improvement of delivery system or resource allocation process, modification of performance objectives or indicators, establishment of monitoring system on ongoing programs, program evaluation and performance analysis, and enhancement of program achievement” (Park, 2012).
Second, after selecting their target expenditure programs, the evaluation divisions of the central government agencies assess the performance of their target programs according to the guidelines provided by MOSF.
Third, the evaluation division of each government agency submits its preliminary self-assessment reports to MOSF for review. MOSF forms a committee of external experts to determine if the agencies’ self-assessment reports are consistent with the guideline that it distributed to all government agencies. The Center for Performance Evaluation and Management of KIPF and NISA participate in the committee with MOSF (Park, 2012).
Fourth, after reviewing the preliminary self-assessment reports, the committee transmits them to the Advisory Board of Evaluation of Budgetary Programs for another review. The board comprises several experts in performance evaluation from universities, research institutes, and private entities (Park, 2012).
Fifth, consulting on the comments and opinions of the Advisory Board of Evaluation of Budgetary Programs, MOSF revises the preliminary self-assessment reports of all the government agencies. After soliciting appeals from the government agencies evaluated by SABP, MOSF finalizes the SABP reports and makes them public (Park, 2012).
MOSF frequently updated the evaluation questions in the guidelines for self-assessment of the central government agencies. They provided 15 questions in 2005, 13 questions in 2007, 11 questions in 2008, and 13 questions in 2011. MOSF divided all government expenditure programs into two sectors of general administration and information technology. The programs in general administration comprised “SOC investment, equipment and facility, direct service provision, equity investment, loan, grants to private sector, and grants to local governments.” There were also two subcategories of “information system” and “supporting system for information” in the information technology sector. As of 2011, for example, the guideline suggested 12 evaluation questions commonly applied to all programs and two questions relevant only to programs in the information technology sector (Park, 2012).
The questions evaluated government expenditure programs in the areas of planning, management, and performance and feedback. In scoring, they applied different weights across the sections: 20 points, 30 points, and 50 points, respectively. The planning section examined two subsections: “adequacy of program plan (three questions)” and “adequacy of performance plan” (two questions). The management section evaluated the extent of “adequacy of program management” (three common questions and two specific questions relevant to information technology). The performance and feedback section checked the level of “accomplishment of performance objectives and feedback of evaluation results” with three questions. The guideline provided not only 13 evaluation questions but also detailed explanations of how to answer the listed questions and assign scores to each evaluation question.
The evaluated divisions of the government agencies added the scores of all evaluation questions to get the total score of each expenditure program, whose maximum was 100. Any expenditure programs with a total score of 90 or higher were categorized as “very good,” ones with a score of 80-90 were “good,” ones with a total score of 70-80 were “fair,” ones with a score of 60-70 were “unsatisfactory,” and programs with a score of 60 and less were categorized as “very unsatisfactory” (Park, 2012).
MOSF announced how to use the performance evaluation information of expenditure programs for budgeting and resource allocation across the expenditure programs. MOSF noted that any expenditure programs in the categories of “unsatisfactory” or “very unsatisfactory” should be penalized by cutting up to 10% of their previous year’s budget, although the cut was not automatic but applied after considering other conditions. In contrast, the expenditure programs whose categories were “good” or “very good” became candidates for a budget increase in the following year (Park, 2012).
Several studies found that SABP revealed multiple unexpected problems and outcomes different from the original intents. First, the number of government expenditure programs assessed by SABP has not been even across years: 555 programs in 2005, 577 programs in 2006, 585 programs in 2007, 384 programs in 2008, 346 programs in 2009, 473 programs in 2010, and 389 programs in 2011. There was a slightly decreasing trend in the number of expenditure programs under the SABP assessment. Additionally, the sectoral distributions of expenditure programs evaluated by SABP have not been balanced. Note that MOSF categorized the government expenditure programs as “SOC investment, equipment and facility, direct service provision, equity investment, loans, grants to private sector, and grants to local governments.” In the period of 2005-2011, in numbers, the expenditure programs in the category of “direct service provision” (in total, 985) received the strongest attention from the SABP assessment, followed by “grants to private sector” (957), “grants to local governments” (603), “equity investment” (284), “loans” (269), and “SOC” (173), while the expenditure programs in the category of “equipment and facility” (38) got the least attention from the SABP assessment (Park and Won, 2012).
Second, it seems that SABP did not enhance significantly the overall performance of government expenditure programs in the period of 2005- 2011, which conflicted with the expectation of MOSF. The average scores of the SABP assessment were 60.1 in 2005, 59.9 in 2006, 66.0 in 2007, 66.6 in 2008, 65.9 in 2009, 62.2 in 2010, and 61.9 in 2011. The average scores across the three subcategories (planning, management, and performance and feedback) have also fluctuated without any clear improvements (Park and Won, 2012).
Third, the scores of the SABP seemed to be inflated. The central government agencies showed a tendency to evaluate the majority of their programs as “fair,” neither too positive nor too negative. The portions of expenditure programs that were evaluated as “fair” and above were 84.3% in 2005, 88.7% in 2006, 94.7% in 2007, 73.2% in 2008, 79.5% in 2009, 75.5% in 2010, and 69.6% in 2011. Without clear reasons, the agencies became rather conservative in self-assessing their expenditure programs after 2008. Park and Won (2012) conjectured that MOSF evaluated that the central agencies were too generous in assessing their expenditure programs in the period of 2005-2007 and ordered them to be stricter in their self-assessment after 2008.
Some studies revealed the actual and potential problems of the SABP system. First, most central government agencies were incapable of producing essential performance information due to their limited technical capability. Regarding this, note that a typical human resource management practice used to place most public employees on diverse posts and rotate them across distinct tasks, which hindered most government officials from deepening their skills and enhancing their expertise in performance evaluation. Second, most central agencies struggled to identify a handful of concrete measures useful in evaluating their outcome. Third, since most government officials in Korea were not accustomed to performance evaluation systems including SABP, they were reluctant to conform to the newly introduced performance evaluation systems that intended to link the performance scores of expenditure programs to budgeting assigned to the programs (Park, 2012).
Some studies tried to evaluate the performance of SABP, although it is too early to make a clear verdict on it. First, the central government agencies favored planning and management rather than performance and feedback in evaluating their own expenditure programs. This tendency is consistent with their typical practices that focused on input-oriented management rather than outcome- and performance-oriented management (Park, 2012).
Second, a positive relationship between the performance scores of the expenditure programs and the budget allocated to the programs was not clear. The main objective of SABP was to use performance information to assign budget across government programs. SABP was expected to provide public employees and government agencies with substantial incentives so that they might be more accountable to the public and perform better. However, scholars evaluated SABP to be unsatisfactory in this area. It is not easy to find a clear association between SABP scores and changes in budget size across years. This implies that a stronger linkage between SABP and resource allocation is required to enhance the conformity of central government agencies to SABP and encourage government officials to seek higher performance (Park, 2009).
Third, the preliminary self-assessment scores by the government agencies were not consistent with the final scores modified by MOSF. Park (2012) found that government agencies tended to overestimate the performance of their expenditure programs. In contrast, MOSF behaved more conservatively in evaluating the performance of government agencies. It was found that the gaps between the preliminary SABP scores and final scores confirmed by MOSF were 25.6 in 2005, 26.8 in 2006, and 24.6 in 2007. More interesting, the majority of the evaluation gaps resulted from the discrepancies of the section of performance rather than the categories of planning and management (Lee, 2010).
Fourth, the National Assembly of Korea did not pay serious attention to the performance evaluation scores collected from SABP when it appropriated budget. The FY 2008 National Assembly assigned larger budget than requested to two expenditure programs whose SABP scores were “unsatisfactory” in 2007. In contrast, the FY 2008 National Assembly allocated lower budget than requested to 11 expenditure programs whose scores were “very good” in 2007 (Park and Park, 2008). The practice of the National Assembly failing to link SABP scores to budget allocation undermined substantially the incentives that might have encouraged the government agencies to conform to the SABP system (Park, 2012).
MOSF introduced IESGP in 2016 in order to address the problems related to the SABP. In particular, IESGP was adopted to integrate the divided evaluations across general budgetary projects, R&D projects, and regional development projects. Consistent with SABP, the primary objectives of IESGP are to enhance the accountability and responsibility of central government agencies relevant to their expenditure programs, to improve performance of central agencies by providing them with tangible incentives, and to link their performance results to their allocated budget. Different from SABP, IESGP underlines the integration of divided evaluations of general budgetary projects, R&D projects, and regional development projects.
There are a couple of IESGP features that are distinct from SABP. First, MOSF intends to evaluate all the expenditure programs of the central government agencies every year by implementing IESGP. It is different from SABP, which evaluated one-third of government expenditure programs annually uo
Second, IESGP combines the existing multiple performance evaluation systems and makes an integrated system. Prior to the introduction of IESGP in 2016, the expenditure programs launched by the central agencies were classified into general budgetary projects, fund projects, R&D projects, and regional development projects. MOSF was responsible for the evaluation of general budgetary projects and fund projects. MSIP assessed the performance of R&D projects. PCRD was in charge of evaluating all regional development projects. Since the evaluation systems were independent across the projects, they lacked consistency in terms of subject, time, and criteria for evaluation. By adopting a universal evaluation system, IESGP intends to overcome the problems that resulted from the dispersed evaluation system (GPEC, 2016a).
Third, MOSF allows more discretion and autonomy to the central ministries in assessing the performance of their expenditure programs through IESGP. Each ministry should form a master committee in charge of the whole process of ministerial self-assessment and subcommittees for sectoral self-assessment within the ministry. All expenditure programs for which each ministry is responsible are divided into general budgetary programs, R&D programs, and regional development programs. The subcommittees within each ministry are in charge of evaluating expenditure programs sector by sector. Thus, a master evaluation committee and sectoral subcommittees in each central ministry are subject to evaluate all expenditure programs for which the ministry is responsible. All central ministries are required to hold information sessions with their evaluation committee boards and the members of their expenditure programs who will become the subjects of evaluation (GPEC, 2016a).
MOSF allows all central ministries to propose a ministerial “Self-Expenditure Adjustment Plan” according to the self-assessment of their expenditure programs. MOSF directs all the central ministries to classify their expenditure programs into three groups based on their performance: “satisfactory” (less than 20% of the total), “fair” (around 65%), and “unsatisfactory” (more than 15%). The classification should be based on relative evaluations in terms of the size of budget assigned to the ministries rather than the number of programs within each ministry. MOSF adopts this system to keep the central ministries from a typical strategy; i.e., they used to classify many small-sized programs in the category of “unsatisfactory.” MOSF sets a target of the ministerial Self- Expenditure Adjustment Plan of 1% of the total budget of all programs under evaluation each year (GPEC, 2016a).
Fourth, compared to SABP, IESGP streamlined the items and criteria (indexes) of the ministerial performance evaluation. SABP evaluated program performance in three areas: planning, management, and performance and feedback. There were four evaluation items of SABP: “adequacy of program plan,” “adequacy of performance plan,” “adequacy of program management,” and “accomplishment of performance objectives and feedback of evaluation results.” SABP used 11 evaluation indexes. IESGP will evaluate central ministerial expenditure programs in the areas of management and results. The evaluation items of IESGP are “adequacy of program management” and “goal attainment and excellence in performance.” IESGP will use four evaluation indexes (GPEC, 2016a).
Fifth, IESGP adopts an additional step in performance evaluation of government expenditure programs, which is called meta-assessment. The meta-assessment determines if the ministerial self-assessment and evaluation results are appropriated. There are two components of the meta-assessment: meta-assessment based on sectoral expenditure programs and meta-assessment at the level of each ministry. For example, the IESGP system classifies all expenditure programs of the Ministry of Education (MOE) into three sectors: general budgetary programs, R&D programs, and regional development programs. MOE will self-assess the performance of its all expenditure programs. MOE’s self-assessment of its general budgetary programs will be subject to the meta-assessment of MOSF. MOE’s self-assessment of its R&D programs should pass the meta-assessment of MSIP. MOE’s self-assessment of its regional development programs will be subject to meta-assessment of the PCRD. The sectoral meta-assessment results will be transmitted to the Council of Meta-Assessment, which overhauls the sectoral meta-assessment ministry by ministry. The Council of Meta- Assessment will be made up of experts and institutions that have expertise in performance evaluation, including the Ministry of Strategy and Finance, MOSF, MSIP, PCRD, KIPF, the National Information Society Agency, the Korea Institute of S&T Evaluation and Planning, and the Korea Institute for Industrial Economics and Trade (GPEC, 2016a).
The meta-assessment should follow the guideline provided by MOSF. There are three evaluation items in the guideline: “adequacy of evaluation process (20%)”, “adequacy of evaluation results (30%)”, and “adequacy of expenditure adjustment plan (50%)”. Two evaluation indexes are assigned to examine the “adequacy of evaluation process”: “whether the organization and operation of a self-assessment board are planned properly (10%),” and “whether a self-assessment board operates properly (10%).” Three evaluation indexes are assigned to examine the “adequacy of evaluation results”: “whether the self-assessment follows performance plans relevant (10%)” and “whether the self-assessment complies with the relative evaluation scheme (20%).” Likewise, two more evaluation indexes are assigned to examine the “adequacy of expenditure adjustment plan”: “whether the expenditure adjustment meets the target assigned by the Ministry of Strategy and Finance (30%)” and “whether the details of adjustment are appropriate (20%)” (Liu, 2017).
Sixth, MOSF announces that it will provide substantial incentives to any central ministry of high performance according to the meta-assessment. MOSF promises that any ministry with high performance deserves a prize, a decrease in the amount of its expenditure adjustment in the following year, and a reduction of the proportion assigned to the evaluation category of “unsatisfactory.” MOSF will penalize any ministries with poor performance. They deserve an increase in the amount of its expenditure adjustment both in the current and following years, a decrease in their general operational costs, and an increase in the proportion assigned to the evaluation category of “unsatisfactory” in the following year. MOSF adds comments and suggestions in the final performance evaluation reports regarding how to enhance program management and performance of the expenditure programs launched by the central government ministries. MOSF will also determine if the central agencies follow the suggestions and comments in the following fiscal year (Liu, 2017).
Liu (2017) summarizes the lessons to be learned and challenges to be overcome from the case of the performance evaluation systems in Korea as follows.
The central government of Korea adopted performance-oriented reforms in order to make its ministries and agencies pay closer attention to their performance, which means to attain their planned goals and objectives. The reforms also aimed to make them responsible for their expenditures. The early stage of the performance-oriented evaluation system was SABP starting in 2005, which let the central ministries assess their expenditure programs themselves. After about ten years, MOSF developed the system into IESGP in 2016. Both SABP and IESGP provide MOSF with practical tools to evaluate the performance of all expenditure programs by the central ministries of Korea.
MOSF and other central ministries subject to performance evaluation found that it was not possible to apply some generalized evaluation criteria to all expenditure programs, particularly to programs whose performance was not easy to quantify. Most central ministries and agencies did not have sufficient knowledge and expertise to evaluate the performance of their expenditure programs quantitatively. In this perspective, MOSF was required to provide them with practical help and financial support including consultation services on how to develop their performance indexes and use them appropriately. MOSF also required to make effective incentive systems and a penalizing mechanism to increase their participation in the performance evaluation system.
MOSF found that many central ministries and agencies carried out many deceptive practices relating to the performance evaluation system. They artificially inflated the self-assessment of their expenditure programs. As for their budget adjustment, they assigned programs for which an actual budget cut would not be available for some reason. MOSF should take heed and uncover these practices by the central ministries and agencies, which impede the effective performance evaluation of government expenditure programs.
One of the most important objectives of a performance evaluation system is to link the performance evaluation scores of government expenditure programs with the budget, which is assigned to the programs in the process of budgeting. However, the Korean cases of SABP and IESGP showed that the mechanism linking performance with budgeting did not work as effectively as anticipated. There existed two phases for this linkage. First, MOSF should reflect the performance evaluation scores of all expenditure programs when it compiles the executive budget. Second, the National Assembly of Korea also should consult the performance evaluation scores of ministerial programs when it appropriates resources across the expenditure programs. Under the previous and current performance evaluation systems, the association between the performance scores of the expenditure programs and the budget assigned to the programs was not clear. When the central ministries are not sure of a clear association between performance scores and budget, they would lose motivation to comply with the performance evaluation systems. Thus, both MOSF and the National Assembly should seek out effective ways to reflect the performance evaluation scores of government expenditure programs in their budget in the course of budget compilation and appropriation, which will result in performance improvement of government expenditure programs and effective resource allocation across government programs.
This case study explains how Korea developed performance evaluation systems for government expenditure programs through two major reforms: the Self-Assessment of Budgetary Programs (SABP) introduced in 2005 and the Integrated Evaluation System on Government Programs (IESGP) introduced in 2016. Both systems were created by the Ministry of Strategy and Finance (MOSF) to improve accountability, strengthen performance management, and allocate budgets based on program results. Under SABP, ministries evaluated one third of their programs each year using a standard checklist, after which MOSF reviewed and adjusted the scores. SABP revealed several issues, including uneven program coverage, inflated self-evaluations, and weak connections between performance scores and budget decisions. To resolve these problems, MOSF introduced IESGP, which evaluates all programs annually, integrates previously separate evaluation systems, incorporates meta-assessment, and applies clearer incentives and penalties. The overall lesson is that performance evaluation must be paired with adequate capacity, credible incentives, and transparent budget links to drive improvement in government programs.
#budgeting #expenditure #evaluation
This case study aims to show the whole cycle of policy adoption, implementation, evaluation, and feedback of the Korean performance evaluation systems: the Self-Assessment of Budgetary Programs (SABP) in 2005 and the Integrated Evaluation System on Government Programs (IESGP) in 2016. The main objectives of these two policies were to evaluate the performance of all government expenditure programs and assign budget across the programs based on their performance scores. The cases are chosen because they are closely related to the fundamental question in budgetary reform and development: On what basis should governments distribute public resources across diverse projects and programs? The answer is to assign budget across programs based on their performance. Both SABP and IESGP are practical tools of the Ministry of Strategy and Finance (MOSF) of Korea to assess the performance of all expenditure programs that the central ministries launch. Through SABP and IESGP, the central ministries have evaluated themselves on the performance of their own expenditure programs. SABP was an initial endeavor of MOSF in this policy area, and IESGP was a reformative effort of MOSF to improve the problems found with SABP since SABP was implemented.
The central government of Korea introduced fundamental budgetary reforms in the late 1990s and the early 2000s under the name of the Four Major Fiscal Reforms. The reforms aimed to promote the effectiveness of performance-oriented management of expenditure programs and enhance long-term fiscal sustainability of Korea. The reforms comprise the medium-term expenditure framework known as the National Fiscal Management Plan, the Top-Down Budgeting System, the Performance Management System, and the Digital Budget Information System. The performance evaluation systems of government expenditure programs are related to the Performance Management System.
The adoption of these performance evaluation systems started in 1999 when the Ministry of Planning and Management renamed as the Ministry of Strategy and Finance launched the pilot projects with 16 selected central ministries. The ministry introduced a series of subsysne items such as the Performance Goal Management of Budgetary Programs in 2003, the SABP in 2005, and the In-Depth Evaluation of Budgetary Programs in 2006. The main objectives of the SABP were to link the performance of government expenditure programs with the amount of budget assigned to the individual expenditure programs. The MOSF amended SABP and adopted the IESGP in 2016 (Park, 2012).
In order to provide the legal foundation for performance evaluation of government expenditure programs, the central government of Korea enacted the Framework Act on Government Performance Evaluation (Act No. 6347, 2001), the National Finance Act, and the National Act Enforcement Decree. Based on these acts, MOSF evaluates the performance of all expenditure programs launched by the central government of Korea (Liu, 2017).
The central government of Korea introduced SABP as a practical tool to evaluate the performance of expenditure programs. The government benchmarked PART of the United States. Both SABP of Korea and PART of the United States are “self-assessment” programs, in which the ministries that are responsible for the expenditure programs evaluate the performance of their own expenditure programs. SABP was designed to review all major expenditure programs launched by the central government over a three-year cycle, one-third of the programs per year. SABP benchmarks PART but added some innovations. Different from PART, the Korean SABP system encourages active participation of external experts in the evaluations. For this purpose, each ministry organizes a self-assessment committee in which external experts participate. The consultation from the external experts is reflected in the ministerial evaluations. After the ministries evaluate the performance of their expenditure programs, the Ministry of Strategy and Finance double-checks the ministerial self-assessments and uses the performance information for budgeting.
One of the major objectives of these self-assessment systems was to provide performance information so that MOSF might assign budget to all expenditure programs based on their performance. After its introduction in 2005, the central government of Korea found that SABP should be amended in several ways. The central government of Korea updated SABP and introduced IESGP in 2016. The In-Depth Evaluation of Budgetary Programs was also adopted to examine the performance of government expenditure programs comprehensively. External experts were invited to examine the performance of the government expenditure programs with analytical and scientific methods (Park, 2012).
The SABP was designed to evaluate the annual performance of one-third of all expenditure programs launched by the central government. Eventually, the performance of all government expenditure programs was to be assessed over a three-year cycle. MOSF provided a checklist for evaluation for all central ministries. Based on the checklist, all central government ministries conducted standardized assessments of the performance of the expenditure programs for which they were responsible. MOSF double-checked the self-assessment results submitted by the government agencies and determined the final annual performance evaluation results. MOSF also added recommendations on how to improve the performance of the expenditure programs of the central ministries. MOSF used the final performance results in budgeting by reexamining and adjusting the size and priority of expenditure programs according to the performance results. MOSF intended to restructure the existing expenditure programs and even terminate some expenditure programs based on their performance results. In sum, the primary objective of SABP was to make central government agencies accountable for the performance of their programs (Park, 2012).
The procedures of SABP are summarized as follows. First, all central government agencies choose one-third of their expenditure programs to be evaluated by SABP. They consult MOSF for the selections. MOSF provides a guideline for the selection of target programs, reflecting “duplication or overlapping with other programs, improvement of delivery system or resource allocation process, modification of performance objectives or indicators, establishment of monitoring system on ongoing programs, program evaluation and performance analysis, and enhancement of program achievement” (Park, 2012).
Second, after selecting their target expenditure programs, the evaluation divisions of the central government agencies assess the performance of their target programs according to the guidelines provided by MOSF.
Third, the evaluation division of each government agency submits its preliminary self-assessment reports to MOSF for review. MOSF forms a committee of external experts to determine if the agencies’ self-assessment reports are consistent with the guideline that it distributed to all government agencies. The Center for Performance Evaluation and Management of KIPF and NISA participate in the committee with MOSF (Park, 2012).
Fourth, after reviewing the preliminary self-assessment reports, the committee transmits them to the Advisory Board of Evaluation of Budgetary Programs for another review. The board comprises several experts in performance evaluation from universities, research institutes, and private entities (Park, 2012).
Fifth, consulting on the comments and opinions of the Advisory Board of Evaluation of Budgetary Programs, MOSF revises the preliminary self-assessment reports of all the government agencies. After soliciting appeals from the government agencies evaluated by SABP, MOSF finalizes the SABP reports and makes them public (Park, 2012).
MOSF frequently updated the evaluation questions in the guidelines for self-assessment of the central government agencies. They provided 15 questions in 2005, 13 questions in 2007, 11 questions in 2008, and 13 questions in 2011. MOSF divided all government expenditure programs into two sectors of general administration and information technology. The programs in general administration comprised “SOC investment, equipment and facility, direct service provision, equity investment, loan, grants to private sector, and grants to local governments.” There were also two subcategories of “information system” and “supporting system for information” in the information technology sector. As of 2011, for example, the guideline suggested 12 evaluation questions commonly applied to all programs and two questions relevant only to programs in the information technology sector (Park, 2012).
The questions evaluated government expenditure programs in the areas of planning, management, and performance and feedback. In scoring, they applied different weights across the sections: 20 points, 30 points, and 50 points, respectively. The planning section examined two subsections: “adequacy of program plan (three questions)” and “adequacy of performance plan” (two questions). The management section evaluated the extent of “adequacy of program management” (three common questions and two specific questions relevant to information technology). The performance and feedback section checked the level of “accomplishment of performance objectives and feedback of evaluation results” with three questions. The guideline provided not only 13 evaluation questions but also detailed explanations of how to answer the listed questions and assign scores to each evaluation question.
The evaluated divisions of the government agencies added the scores of all evaluation questions to get the total score of each expenditure program, whose maximum was 100. Any expenditure programs with a total score of 90 or higher were categorized as “very good,” ones with a score of 80-90 were “good,” ones with a total score of 70-80 were “fair,” ones with a score of 60-70 were “unsatisfactory,” and programs with a score of 60 and less were categorized as “very unsatisfactory” (Park, 2012).
MOSF announced how to use the performance evaluation information of expenditure programs for budgeting and resource allocation across the expenditure programs. MOSF noted that any expenditure programs in the categories of “unsatisfactory” or “very unsatisfactory” should be penalized by cutting up to 10% of their previous year’s budget, although the cut was not automatic but applied after considering other conditions. In contrast, the expenditure programs whose categories were “good” or “very good” became candidates for a budget increase in the following year (Park, 2012).
Several studies found that SABP revealed multiple unexpected problems and outcomes different from the original intents. First, the number of government expenditure programs assessed by SABP has not been even across years: 555 programs in 2005, 577 programs in 2006, 585 programs in 2007, 384 programs in 2008, 346 programs in 2009, 473 programs in 2010, and 389 programs in 2011. There was a slightly decreasing trend in the number of expenditure programs under the SABP assessment. Additionally, the sectoral distributions of expenditure programs evaluated by SABP have not been balanced. Note that MOSF categorized the government expenditure programs as “SOC investment, equipment and facility, direct service provision, equity investment, loans, grants to private sector, and grants to local governments.” In the period of 2005-2011, in numbers, the expenditure programs in the category of “direct service provision” (in total, 985) received the strongest attention from the SABP assessment, followed by “grants to private sector” (957), “grants to local governments” (603), “equity investment” (284), “loans” (269), and “SOC” (173), while the expenditure programs in the category of “equipment and facility” (38) got the least attention from the SABP assessment (Park and Won, 2012).
Second, it seems that SABP did not enhance significantly the overall performance of government expenditure programs in the period of 2005- 2011, which conflicted with the expectation of MOSF. The average scores of the SABP assessment were 60.1 in 2005, 59.9 in 2006, 66.0 in 2007, 66.6 in 2008, 65.9 in 2009, 62.2 in 2010, and 61.9 in 2011. The average scores across the three subcategories (planning, management, and performance and feedback) have also fluctuated without any clear improvements (Park and Won, 2012).
Third, the scores of the SABP seemed to be inflated. The central government agencies showed a tendency to evaluate the majority of their programs as “fair,” neither too positive nor too negative. The portions of expenditure programs that were evaluated as “fair” and above were 84.3% in 2005, 88.7% in 2006, 94.7% in 2007, 73.2% in 2008, 79.5% in 2009, 75.5% in 2010, and 69.6% in 2011. Without clear reasons, the agencies became rather conservative in self-assessing their expenditure programs after 2008. Park and Won (2012) conjectured that MOSF evaluated that the central agencies were too generous in assessing their expenditure programs in the period of 2005-2007 and ordered them to be stricter in their self-assessment after 2008.
Some studies revealed the actual and potential problems of the SABP system. First, most central government agencies were incapable of producing essential performance information due to their limited technical capability. Regarding this, note that a typical human resource management practice used to place most public employees on diverse posts and rotate them across distinct tasks, which hindered most government officials from deepening their skills and enhancing their expertise in performance evaluation. Second, most central agencies struggled to identify a handful of concrete measures useful in evaluating their outcome. Third, since most government officials in Korea were not accustomed to performance evaluation systems including SABP, they were reluctant to conform to the newly introduced performance evaluation systems that intended to link the performance scores of expenditure programs to budgeting assigned to the programs (Park, 2012).
Some studies tried to evaluate the performance of SABP, although it is too early to make a clear verdict on it. First, the central government agencies favored planning and management rather than performance and feedback in evaluating their own expenditure programs. This tendency is consistent with their typical practices that focused on input-oriented management rather than outcome- and performance-oriented management (Park, 2012).
Second, a positive relationship between the performance scores of the expenditure programs and the budget allocated to the programs was not clear. The main objective of SABP was to use performance information to assign budget across government programs. SABP was expected to provide public employees and government agencies with substantial incentives so that they might be more accountable to the public and perform better. However, scholars evaluated SABP to be unsatisfactory in this area. It is not easy to find a clear association between SABP scores and changes in budget size across years. This implies that a stronger linkage between SABP and resource allocation is required to enhance the conformity of central government agencies to SABP and encourage government officials to seek higher performance (Park, 2009).
Third, the preliminary self-assessment scores by the government agencies were not consistent with the final scores modified by MOSF. Park (2012) found that government agencies tended to overestimate the performance of their expenditure programs. In contrast, MOSF behaved more conservatively in evaluating the performance of government agencies. It was found that the gaps between the preliminary SABP scores and final scores confirmed by MOSF were 25.6 in 2005, 26.8 in 2006, and 24.6 in 2007. More interesting, the majority of the evaluation gaps resulted from the discrepancies of the section of performance rather than the categories of planning and management (Lee, 2010).
Fourth, the National Assembly of Korea did not pay serious attention to the performance evaluation scores collected from SABP when it appropriated budget. The FY 2008 National Assembly assigned larger budget than requested to two expenditure programs whose SABP scores were “unsatisfactory” in 2007. In contrast, the FY 2008 National Assembly allocated lower budget than requested to 11 expenditure programs whose scores were “very good” in 2007 (Park and Park, 2008). The practice of the National Assembly failing to link SABP scores to budget allocation undermined substantially the incentives that might have encouraged the government agencies to conform to the SABP system (Park, 2012).
MOSF introduced IESGP in 2016 in order to address the problems related to the SABP. In particular, IESGP was adopted to integrate the divided evaluations across general budgetary projects, R&D projects, and regional development projects. Consistent with SABP, the primary objectives of IESGP are to enhance the accountability and responsibility of central government agencies relevant to their expenditure programs, to improve performance of central agencies by providing them with tangible incentives, and to link their performance results to their allocated budget. Different from SABP, IESGP underlines the integration of divided evaluations of general budgetary projects, R&D projects, and regional development projects.
There are a couple of IESGP features that are distinct from SABP. First, MOSF intends to evaluate all the expenditure programs of the central government agencies every year by implementing IESGP. It is different from SABP, which evaluated one-third of government expenditure programs annually uo
Second, IESGP combines the existing multiple performance evaluation systems and makes an integrated system. Prior to the introduction of IESGP in 2016, the expenditure programs launched by the central agencies were classified into general budgetary projects, fund projects, R&D projects, and regional development projects. MOSF was responsible for the evaluation of general budgetary projects and fund projects. MSIP assessed the performance of R&D projects. PCRD was in charge of evaluating all regional development projects. Since the evaluation systems were independent across the projects, they lacked consistency in terms of subject, time, and criteria for evaluation. By adopting a universal evaluation system, IESGP intends to overcome the problems that resulted from the dispersed evaluation system (GPEC, 2016a).
Third, MOSF allows more discretion and autonomy to the central ministries in assessing the performance of their expenditure programs through IESGP. Each ministry should form a master committee in charge of the whole process of ministerial self-assessment and subcommittees for sectoral self-assessment within the ministry. All expenditure programs for which each ministry is responsible are divided into general budgetary programs, R&D programs, and regional development programs. The subcommittees within each ministry are in charge of evaluating expenditure programs sector by sector. Thus, a master evaluation committee and sectoral subcommittees in each central ministry are subject to evaluate all expenditure programs for which the ministry is responsible. All central ministries are required to hold information sessions with their evaluation committee boards and the members of their expenditure programs who will become the subjects of evaluation (GPEC, 2016a).
MOSF allows all central ministries to propose a ministerial “Self-Expenditure Adjustment Plan” according to the self-assessment of their expenditure programs. MOSF directs all the central ministries to classify their expenditure programs into three groups based on their performance: “satisfactory” (less than 20% of the total), “fair” (around 65%), and “unsatisfactory” (more than 15%). The classification should be based on relative evaluations in terms of the size of budget assigned to the ministries rather than the number of programs within each ministry. MOSF adopts this system to keep the central ministries from a typical strategy; i.e., they used to classify many small-sized programs in the category of “unsatisfactory.” MOSF sets a target of the ministerial Self- Expenditure Adjustment Plan of 1% of the total budget of all programs under evaluation each year (GPEC, 2016a).
Fourth, compared to SABP, IESGP streamlined the items and criteria (indexes) of the ministerial performance evaluation. SABP evaluated program performance in three areas: planning, management, and performance and feedback. There were four evaluation items of SABP: “adequacy of program plan,” “adequacy of performance plan,” “adequacy of program management,” and “accomplishment of performance objectives and feedback of evaluation results.” SABP used 11 evaluation indexes. IESGP will evaluate central ministerial expenditure programs in the areas of management and results. The evaluation items of IESGP are “adequacy of program management” and “goal attainment and excellence in performance.” IESGP will use four evaluation indexes (GPEC, 2016a).
Fifth, IESGP adopts an additional step in performance evaluation of government expenditure programs, which is called meta-assessment. The meta-assessment determines if the ministerial self-assessment and evaluation results are appropriated. There are two components of the meta-assessment: meta-assessment based on sectoral expenditure programs and meta-assessment at the level of each ministry. For example, the IESGP system classifies all expenditure programs of the Ministry of Education (MOE) into three sectors: general budgetary programs, R&D programs, and regional development programs. MOE will self-assess the performance of its all expenditure programs. MOE’s self-assessment of its general budgetary programs will be subject to the meta-assessment of MOSF. MOE’s self-assessment of its R&D programs should pass the meta-assessment of MSIP. MOE’s self-assessment of its regional development programs will be subject to meta-assessment of the PCRD. The sectoral meta-assessment results will be transmitted to the Council of Meta-Assessment, which overhauls the sectoral meta-assessment ministry by ministry. The Council of Meta- Assessment will be made up of experts and institutions that have expertise in performance evaluation, including the Ministry of Strategy and Finance, MOSF, MSIP, PCRD, KIPF, the National Information Society Agency, the Korea Institute of S&T Evaluation and Planning, and the Korea Institute for Industrial Economics and Trade (GPEC, 2016a).
The meta-assessment should follow the guideline provided by MOSF. There are three evaluation items in the guideline: “adequacy of evaluation process (20%)”, “adequacy of evaluation results (30%)”, and “adequacy of expenditure adjustment plan (50%)”. Two evaluation indexes are assigned to examine the “adequacy of evaluation process”: “whether the organization and operation of a self-assessment board are planned properly (10%),” and “whether a self-assessment board operates properly (10%).” Three evaluation indexes are assigned to examine the “adequacy of evaluation results”: “whether the self-assessment follows performance plans relevant (10%)” and “whether the self-assessment complies with the relative evaluation scheme (20%).” Likewise, two more evaluation indexes are assigned to examine the “adequacy of expenditure adjustment plan”: “whether the expenditure adjustment meets the target assigned by the Ministry of Strategy and Finance (30%)” and “whether the details of adjustment are appropriate (20%)” (Liu, 2017).
Sixth, MOSF announces that it will provide substantial incentives to any central ministry of high performance according to the meta-assessment. MOSF promises that any ministry with high performance deserves a prize, a decrease in the amount of its expenditure adjustment in the following year, and a reduction of the proportion assigned to the evaluation category of “unsatisfactory.” MOSF will penalize any ministries with poor performance. They deserve an increase in the amount of its expenditure adjustment both in the current and following years, a decrease in their general operational costs, and an increase in the proportion assigned to the evaluation category of “unsatisfactory” in the following year. MOSF adds comments and suggestions in the final performance evaluation reports regarding how to enhance program management and performance of the expenditure programs launched by the central government ministries. MOSF will also determine if the central agencies follow the suggestions and comments in the following fiscal year (Liu, 2017).
Liu (2017) summarizes the lessons to be learned and challenges to be overcome from the case of the performance evaluation systems in Korea as follows.
The central government of Korea adopted performance-oriented reforms in order to make its ministries and agencies pay closer attention to their performance, which means to attain their planned goals and objectives. The reforms also aimed to make them responsible for their expenditures. The early stage of the performance-oriented evaluation system was SABP starting in 2005, which let the central ministries assess their expenditure programs themselves. After about ten years, MOSF developed the system into IESGP in 2016. Both SABP and IESGP provide MOSF with practical tools to evaluate the performance of all expenditure programs by the central ministries of Korea.
MOSF and other central ministries subject to performance evaluation found that it was not possible to apply some generalized evaluation criteria to all expenditure programs, particularly to programs whose performance was not easy to quantify. Most central ministries and agencies did not have sufficient knowledge and expertise to evaluate the performance of their expenditure programs quantitatively. In this perspective, MOSF was required to provide them with practical help and financial support including consultation services on how to develop their performance indexes and use them appropriately. MOSF also required to make effective incentive systems and a penalizing mechanism to increase their participation in the performance evaluation system.
MOSF found that many central ministries and agencies carried out many deceptive practices relating to the performance evaluation system. They artificially inflated the self-assessment of their expenditure programs. As for their budget adjustment, they assigned programs for which an actual budget cut would not be available for some reason. MOSF should take heed and uncover these practices by the central ministries and agencies, which impede the effective performance evaluation of government expenditure programs.
One of the most important objectives of a performance evaluation system is to link the performance evaluation scores of government expenditure programs with the budget, which is assigned to the programs in the process of budgeting. However, the Korean cases of SABP and IESGP showed that the mechanism linking performance with budgeting did not work as effectively as anticipated. There existed two phases for this linkage. First, MOSF should reflect the performance evaluation scores of all expenditure programs when it compiles the executive budget. Second, the National Assembly of Korea also should consult the performance evaluation scores of ministerial programs when it appropriates resources across the expenditure programs. Under the previous and current performance evaluation systems, the association between the performance scores of the expenditure programs and the budget assigned to the programs was not clear. When the central ministries are not sure of a clear association between performance scores and budget, they would lose motivation to comply with the performance evaluation systems. Thus, both MOSF and the National Assembly should seek out effective ways to reflect the performance evaluation scores of government expenditure programs in their budget in the course of budget compilation and appropriation, which will result in performance improvement of government expenditure programs and effective resource allocation across government programs.
This case study aims to show the whole cycle of policy adoption, implementation, evaluation, and feedback of the Korean performance evaluation systems: the Self-Assessment of Budgetary Programs (SABP) in 2005 and the Integrated Evaluation System on Government Programs (IESGP) in 2016. The main objectives of these two policies were to evaluate the performance of all government expenditure programs and assign budget across the programs based on their performance scores. The cases are chosen because they are closely related to the fundamental question in budgetary reform and development: On what basis should governments distribute public resources across diverse projects and programs? The answer is to assign budget across programs based on their performance. Both SABP and IESGP are practical tools of the Ministry of Strategy and Finance (MOSF) of Korea to assess the performance of all expenditure programs that the central ministries launch. Through SABP and IESGP, the central ministries have evaluated themselves on the performance of their own expenditure programs. SABP was an initial endeavor of MOSF in this policy area, and IESGP was a reformative effort of MOSF to improve the problems found with SABP since SABP was implemented.
The central government of Korea introduced fundamental budgetary reforms in the late 1990s and the early 2000s under the name of the Four Major Fiscal Reforms. The reforms aimed to promote the effectiveness of performance-oriented management of expenditure programs and enhance long-term fiscal sustainability of Korea. The reforms comprise the medium-term expenditure framework known as the National Fiscal Management Plan, the Top-Down Budgeting System, the Performance Management System, and the Digital Budget Information System. The performance evaluation systems of government expenditure programs are related to the Performance Management System.
The adoption of these performance evaluation systems started in 1999 when the Ministry of Planning and Management renamed as the Ministry of Strategy and Finance launched the pilot projects with 16 selected central ministries. The ministry introduced a series of subsysne items such as the Performance Goal Management of Budgetary Programs in 2003, the SABP in 2005, and the In-Depth Evaluation of Budgetary Programs in 2006. The main objectives of the SABP were to link the performance of government expenditure programs with the amount of budget assigned to the individual expenditure programs. The MOSF amended SABP and adopted the IESGP in 2016 (Park, 2012).
In order to provide the legal foundation for performance evaluation of government expenditure programs, the central government of Korea enacted the Framework Act on Government Performance Evaluation (Act No. 6347, 2001), the National Finance Act, and the National Act Enforcement Decree. Based on these acts, MOSF evaluates the performance of all expenditure programs launched by the central government of Korea (Liu, 2017).
The central government of Korea introduced SABP as a practical tool to evaluate the performance of expenditure programs. The government benchmarked PART of the United States. Both SABP of Korea and PART of the United States are “self-assessment” programs, in which the ministries that are responsible for the expenditure programs evaluate the performance of their own expenditure programs. SABP was designed to review all major expenditure programs launched by the central government over a three-year cycle, one-third of the programs per year. SABP benchmarks PART but added some innovations. Different from PART, the Korean SABP system encourages active participation of external experts in the evaluations. For this purpose, each ministry organizes a self-assessment committee in which external experts participate. The consultation from the external experts is reflected in the ministerial evaluations. After the ministries evaluate the performance of their expenditure programs, the Ministry of Strategy and Finance double-checks the ministerial self-assessments and uses the performance information for budgeting.
One of the major objectives of these self-assessment systems was to provide performance information so that MOSF might assign budget to all expenditure programs based on their performance. After its introduction in 2005, the central government of Korea found that SABP should be amended in several ways. The central government of Korea updated SABP and introduced IESGP in 2016. The In-Depth Evaluation of Budgetary Programs was also adopted to examine the performance of government expenditure programs comprehensively. External experts were invited to examine the performance of the government expenditure programs with analytical and scientific methods (Park, 2012).
The SABP was designed to evaluate the annual performance of one-third of all expenditure programs launched by the central government. Eventually, the performance of all government expenditure programs was to be assessed over a three-year cycle. MOSF provided a checklist for evaluation for all central ministries. Based on the checklist, all central government ministries conducted standardized assessments of the performance of the expenditure programs for which they were responsible. MOSF double-checked the self-assessment results submitted by the government agencies and determined the final annual performance evaluation results. MOSF also added recommendations on how to improve the performance of the expenditure programs of the central ministries. MOSF used the final performance results in budgeting by reexamining and adjusting the size and priority of expenditure programs according to the performance results. MOSF intended to restructure the existing expenditure programs and even terminate some expenditure programs based on their performance results. In sum, the primary objective of SABP was to make central government agencies accountable for the performance of their programs (Park, 2012).
The procedures of SABP are summarized as follows. First, all central government agencies choose one-third of their expenditure programs to be evaluated by SABP. They consult MOSF for the selections. MOSF provides a guideline for the selection of target programs, reflecting “duplication or overlapping with other programs, improvement of delivery system or resource allocation process, modification of performance objectives or indicators, establishment of monitoring system on ongoing programs, program evaluation and performance analysis, and enhancement of program achievement” (Park, 2012).
Second, after selecting their target expenditure programs, the evaluation divisions of the central government agencies assess the performance of their target programs according to the guidelines provided by MOSF.
Third, the evaluation division of each government agency submits its preliminary self-assessment reports to MOSF for review. MOSF forms a committee of external experts to determine if the agencies’ self-assessment reports are consistent with the guideline that it distributed to all government agencies. The Center for Performance Evaluation and Management of KIPF and NISA participate in the committee with MOSF (Park, 2012).
Fourth, after reviewing the preliminary self-assessment reports, the committee transmits them to the Advisory Board of Evaluation of Budgetary Programs for another review. The board comprises several experts in performance evaluation from universities, research institutes, and private entities (Park, 2012).
Fifth, consulting on the comments and opinions of the Advisory Board of Evaluation of Budgetary Programs, MOSF revises the preliminary self-assessment reports of all the government agencies. After soliciting appeals from the government agencies evaluated by SABP, MOSF finalizes the SABP reports and makes them public (Park, 2012).
MOSF frequently updated the evaluation questions in the guidelines for self-assessment of the central government agencies. They provided 15 questions in 2005, 13 questions in 2007, 11 questions in 2008, and 13 questions in 2011. MOSF divided all government expenditure programs into two sectors of general administration and information technology. The programs in general administration comprised “SOC investment, equipment and facility, direct service provision, equity investment, loan, grants to private sector, and grants to local governments.” There were also two subcategories of “information system” and “supporting system for information” in the information technology sector. As of 2011, for example, the guideline suggested 12 evaluation questions commonly applied to all programs and two questions relevant only to programs in the information technology sector (Park, 2012).
The questions evaluated government expenditure programs in the areas of planning, management, and performance and feedback. In scoring, they applied different weights across the sections: 20 points, 30 points, and 50 points, respectively. The planning section examined two subsections: “adequacy of program plan (three questions)” and “adequacy of performance plan” (two questions). The management section evaluated the extent of “adequacy of program management” (three common questions and two specific questions relevant to information technology). The performance and feedback section checked the level of “accomplishment of performance objectives and feedback of evaluation results” with three questions. The guideline provided not only 13 evaluation questions but also detailed explanations of how to answer the listed questions and assign scores to each evaluation question.
The evaluated divisions of the government agencies added the scores of all evaluation questions to get the total score of each expenditure program, whose maximum was 100. Any expenditure programs with a total score of 90 or higher were categorized as “very good,” ones with a score of 80-90 were “good,” ones with a total score of 70-80 were “fair,” ones with a score of 60-70 were “unsatisfactory,” and programs with a score of 60 and less were categorized as “very unsatisfactory” (Park, 2012).
MOSF announced how to use the performance evaluation information of expenditure programs for budgeting and resource allocation across the expenditure programs. MOSF noted that any expenditure programs in the categories of “unsatisfactory” or “very unsatisfactory” should be penalized by cutting up to 10% of their previous year’s budget, although the cut was not automatic but applied after considering other conditions. In contrast, the expenditure programs whose categories were “good” or “very good” became candidates for a budget increase in the following year (Park, 2012).
Several studies found that SABP revealed multiple unexpected problems and outcomes different from the original intents. First, the number of government expenditure programs assessed by SABP has not been even across years: 555 programs in 2005, 577 programs in 2006, 585 programs in 2007, 384 programs in 2008, 346 programs in 2009, 473 programs in 2010, and 389 programs in 2011. There was a slightly decreasing trend in the number of expenditure programs under the SABP assessment. Additionally, the sectoral distributions of expenditure programs evaluated by SABP have not been balanced. Note that MOSF categorized the government expenditure programs as “SOC investment, equipment and facility, direct service provision, equity investment, loans, grants to private sector, and grants to local governments.” In the period of 2005-2011, in numbers, the expenditure programs in the category of “direct service provision” (in total, 985) received the strongest attention from the SABP assessment, followed by “grants to private sector” (957), “grants to local governments” (603), “equity investment” (284), “loans” (269), and “SOC” (173), while the expenditure programs in the category of “equipment and facility” (38) got the least attention from the SABP assessment (Park and Won, 2012).
Second, it seems that SABP did not enhance significantly the overall performance of government expenditure programs in the period of 2005- 2011, which conflicted with the expectation of MOSF. The average scores of the SABP assessment were 60.1 in 2005, 59.9 in 2006, 66.0 in 2007, 66.6 in 2008, 65.9 in 2009, 62.2 in 2010, and 61.9 in 2011. The average scores across the three subcategories (planning, management, and performance and feedback) have also fluctuated without any clear improvements (Park and Won, 2012).
Third, the scores of the SABP seemed to be inflated. The central government agencies showed a tendency to evaluate the majority of their programs as “fair,” neither too positive nor too negative. The portions of expenditure programs that were evaluated as “fair” and above were 84.3% in 2005, 88.7% in 2006, 94.7% in 2007, 73.2% in 2008, 79.5% in 2009, 75.5% in 2010, and 69.6% in 2011. Without clear reasons, the agencies became rather conservative in self-assessing their expenditure programs after 2008. Park and Won (2012) conjectured that MOSF evaluated that the central agencies were too generous in assessing their expenditure programs in the period of 2005-2007 and ordered them to be stricter in their self-assessment after 2008.
Some studies revealed the actual and potential problems of the SABP system. First, most central government agencies were incapable of producing essential performance information due to their limited technical capability. Regarding this, note that a typical human resource management practice used to place most public employees on diverse posts and rotate them across distinct tasks, which hindered most government officials from deepening their skills and enhancing their expertise in performance evaluation. Second, most central agencies struggled to identify a handful of concrete measures useful in evaluating their outcome. Third, since most government officials in Korea were not accustomed to performance evaluation systems including SABP, they were reluctant to conform to the newly introduced performance evaluation systems that intended to link the performance scores of expenditure programs to budgeting assigned to the programs (Park, 2012).
Some studies tried to evaluate the performance of SABP, although it is too early to make a clear verdict on it. First, the central government agencies favored planning and management rather than performance and feedback in evaluating their own expenditure programs. This tendency is consistent with their typical practices that focused on input-oriented management rather than outcome- and performance-oriented management (Park, 2012).
Second, a positive relationship between the performance scores of the expenditure programs and the budget allocated to the programs was not clear. The main objective of SABP was to use performance information to assign budget across government programs. SABP was expected to provide public employees and government agencies with substantial incentives so that they might be more accountable to the public and perform better. However, scholars evaluated SABP to be unsatisfactory in this area. It is not easy to find a clear association between SABP scores and changes in budget size across years. This implies that a stronger linkage between SABP and resource allocation is required to enhance the conformity of central government agencies to SABP and encourage government officials to seek higher performance (Park, 2009).
Third, the preliminary self-assessment scores by the government agencies were not consistent with the final scores modified by MOSF. Park (2012) found that government agencies tended to overestimate the performance of their expenditure programs. In contrast, MOSF behaved more conservatively in evaluating the performance of government agencies. It was found that the gaps between the preliminary SABP scores and final scores confirmed by MOSF were 25.6 in 2005, 26.8 in 2006, and 24.6 in 2007. More interesting, the majority of the evaluation gaps resulted from the discrepancies of the section of performance rather than the categories of planning and management (Lee, 2010).
Fourth, the National Assembly of Korea did not pay serious attention to the performance evaluation scores collected from SABP when it appropriated budget. The FY 2008 National Assembly assigned larger budget than requested to two expenditure programs whose SABP scores were “unsatisfactory” in 2007. In contrast, the FY 2008 National Assembly allocated lower budget than requested to 11 expenditure programs whose scores were “very good” in 2007 (Park and Park, 2008). The practice of the National Assembly failing to link SABP scores to budget allocation undermined substantially the incentives that might have encouraged the government agencies to conform to the SABP system (Park, 2012).
MOSF introduced IESGP in 2016 in order to address the problems related to the SABP. In particular, IESGP was adopted to integrate the divided evaluations across general budgetary projects, R&D projects, and regional development projects. Consistent with SABP, the primary objectives of IESGP are to enhance the accountability and responsibility of central government agencies relevant to their expenditure programs, to improve performance of central agencies by providing them with tangible incentives, and to link their performance results to their allocated budget. Different from SABP, IESGP underlines the integration of divided evaluations of general budgetary projects, R&D projects, and regional development projects.
There are a couple of IESGP features that are distinct from SABP. First, MOSF intends to evaluate all the expenditure programs of the central government agencies every year by implementing IESGP. It is different from SABP, which evaluated one-third of government expenditure programs annually uo
Second, IESGP combines the existing multiple performance evaluation systems and makes an integrated system. Prior to the introduction of IESGP in 2016, the expenditure programs launched by the central agencies were classified into general budgetary projects, fund projects, R&D projects, and regional development projects. MOSF was responsible for the evaluation of general budgetary projects and fund projects. MSIP assessed the performance of R&D projects. PCRD was in charge of evaluating all regional development projects. Since the evaluation systems were independent across the projects, they lacked consistency in terms of subject, time, and criteria for evaluation. By adopting a universal evaluation system, IESGP intends to overcome the problems that resulted from the dispersed evaluation system (GPEC, 2016a).
Third, MOSF allows more discretion and autonomy to the central ministries in assessing the performance of their expenditure programs through IESGP. Each ministry should form a master committee in charge of the whole process of ministerial self-assessment and subcommittees for sectoral self-assessment within the ministry. All expenditure programs for which each ministry is responsible are divided into general budgetary programs, R&D programs, and regional development programs. The subcommittees within each ministry are in charge of evaluating expenditure programs sector by sector. Thus, a master evaluation committee and sectoral subcommittees in each central ministry are subject to evaluate all expenditure programs for which the ministry is responsible. All central ministries are required to hold information sessions with their evaluation committee boards and the members of their expenditure programs who will become the subjects of evaluation (GPEC, 2016a).
MOSF allows all central ministries to propose a ministerial “Self-Expenditure Adjustment Plan” according to the self-assessment of their expenditure programs. MOSF directs all the central ministries to classify their expenditure programs into three groups based on their performance: “satisfactory” (less than 20% of the total), “fair” (around 65%), and “unsatisfactory” (more than 15%). The classification should be based on relative evaluations in terms of the size of budget assigned to the ministries rather than the number of programs within each ministry. MOSF adopts this system to keep the central ministries from a typical strategy; i.e., they used to classify many small-sized programs in the category of “unsatisfactory.” MOSF sets a target of the ministerial Self- Expenditure Adjustment Plan of 1% of the total budget of all programs under evaluation each year (GPEC, 2016a).
Fourth, compared to SABP, IESGP streamlined the items and criteria (indexes) of the ministerial performance evaluation. SABP evaluated program performance in three areas: planning, management, and performance and feedback. There were four evaluation items of SABP: “adequacy of program plan,” “adequacy of performance plan,” “adequacy of program management,” and “accomplishment of performance objectives and feedback of evaluation results.” SABP used 11 evaluation indexes. IESGP will evaluate central ministerial expenditure programs in the areas of management and results. The evaluation items of IESGP are “adequacy of program management” and “goal attainment and excellence in performance.” IESGP will use four evaluation indexes (GPEC, 2016a).
Fifth, IESGP adopts an additional step in performance evaluation of government expenditure programs, which is called meta-assessment. The meta-assessment determines if the ministerial self-assessment and evaluation results are appropriated. There are two components of the meta-assessment: meta-assessment based on sectoral expenditure programs and meta-assessment at the level of each ministry. For example, the IESGP system classifies all expenditure programs of the Ministry of Education (MOE) into three sectors: general budgetary programs, R&D programs, and regional development programs. MOE will self-assess the performance of its all expenditure programs. MOE’s self-assessment of its general budgetary programs will be subject to the meta-assessment of MOSF. MOE’s self-assessment of its R&D programs should pass the meta-assessment of MSIP. MOE’s self-assessment of its regional development programs will be subject to meta-assessment of the PCRD. The sectoral meta-assessment results will be transmitted to the Council of Meta-Assessment, which overhauls the sectoral meta-assessment ministry by ministry. The Council of Meta- Assessment will be made up of experts and institutions that have expertise in performance evaluation, including the Ministry of Strategy and Finance, MOSF, MSIP, PCRD, KIPF, the National Information Society Agency, the Korea Institute of S&T Evaluation and Planning, and the Korea Institute for Industrial Economics and Trade (GPEC, 2016a).
The meta-assessment should follow the guideline provided by MOSF. There are three evaluation items in the guideline: “adequacy of evaluation process (20%)”, “adequacy of evaluation results (30%)”, and “adequacy of expenditure adjustment plan (50%)”. Two evaluation indexes are assigned to examine the “adequacy of evaluation process”: “whether the organization and operation of a self-assessment board are planned properly (10%),” and “whether a self-assessment board operates properly (10%).” Three evaluation indexes are assigned to examine the “adequacy of evaluation results”: “whether the self-assessment follows performance plans relevant (10%)” and “whether the self-assessment complies with the relative evaluation scheme (20%).” Likewise, two more evaluation indexes are assigned to examine the “adequacy of expenditure adjustment plan”: “whether the expenditure adjustment meets the target assigned by the Ministry of Strategy and Finance (30%)” and “whether the details of adjustment are appropriate (20%)” (Liu, 2017).
Sixth, MOSF announces that it will provide substantial incentives to any central ministry of high performance according to the meta-assessment. MOSF promises that any ministry with high performance deserves a prize, a decrease in the amount of its expenditure adjustment in the following year, and a reduction of the proportion assigned to the evaluation category of “unsatisfactory.” MOSF will penalize any ministries with poor performance. They deserve an increase in the amount of its expenditure adjustment both in the current and following years, a decrease in their general operational costs, and an increase in the proportion assigned to the evaluation category of “unsatisfactory” in the following year. MOSF adds comments and suggestions in the final performance evaluation reports regarding how to enhance program management and performance of the expenditure programs launched by the central government ministries. MOSF will also determine if the central agencies follow the suggestions and comments in the following fiscal year (Liu, 2017).
Liu (2017) summarizes the lessons to be learned and challenges to be overcome from the case of the performance evaluation systems in Korea as follows.
The central government of Korea adopted performance-oriented reforms in order to make its ministries and agencies pay closer attention to their performance, which means to attain their planned goals and objectives. The reforms also aimed to make them responsible for their expenditures. The early stage of the performance-oriented evaluation system was SABP starting in 2005, which let the central ministries assess their expenditure programs themselves. After about ten years, MOSF developed the system into IESGP in 2016. Both SABP and IESGP provide MOSF with practical tools to evaluate the performance of all expenditure programs by the central ministries of Korea.
MOSF and other central ministries subject to performance evaluation found that it was not possible to apply some generalized evaluation criteria to all expenditure programs, particularly to programs whose performance was not easy to quantify. Most central ministries and agencies did not have sufficient knowledge and expertise to evaluate the performance of their expenditure programs quantitatively. In this perspective, MOSF was required to provide them with practical help and financial support including consultation services on how to develop their performance indexes and use them appropriately. MOSF also required to make effective incentive systems and a penalizing mechanism to increase their participation in the performance evaluation system.
MOSF found that many central ministries and agencies carried out many deceptive practices relating to the performance evaluation system. They artificially inflated the self-assessment of their expenditure programs. As for their budget adjustment, they assigned programs for which an actual budget cut would not be available for some reason. MOSF should take heed and uncover these practices by the central ministries and agencies, which impede the effective performance evaluation of government expenditure programs.
One of the most important objectives of a performance evaluation system is to link the performance evaluation scores of government expenditure programs with the budget, which is assigned to the programs in the process of budgeting. However, the Korean cases of SABP and IESGP showed that the mechanism linking performance with budgeting did not work as effectively as anticipated. There existed two phases for this linkage. First, MOSF should reflect the performance evaluation scores of all expenditure programs when it compiles the executive budget. Second, the National Assembly of Korea also should consult the performance evaluation scores of ministerial programs when it appropriates resources across the expenditure programs. Under the previous and current performance evaluation systems, the association between the performance scores of the expenditure programs and the budget assigned to the programs was not clear. When the central ministries are not sure of a clear association between performance scores and budget, they would lose motivation to comply with the performance evaluation systems. Thus, both MOSF and the National Assembly should seek out effective ways to reflect the performance evaluation scores of government expenditure programs in their budget in the course of budget compilation and appropriation, which will result in performance improvement of government expenditure programs and effective resource allocation across government programs.