Ex post evaluation is not just a “pillar” of State Aid Modernisation. It is also an indispensable tool for understanding the impact and effectiveness of State aid.
But every policy measure has its costs which cannot be ignored. While decentralisation allows the Commission to focus its resources and assessment, it may also tempt Member States to grant aid in amounts, forms and for purposes not permitted by State aid rules. For this reason, the SAM has also introduced safeguards through more extensive ex post control.
This ex post control is based on three types of action:
- increased transparency,
- wider monitoring of implemented measures, and
- evaluation of large or novel measures.
Increased transparency will be put in place as of 2016. After 2016, Member States will have to publish all individual awards of aid that exceed EUR 0.5 million. The Commission hopes that the publication of awards and the names of beneficiaries will make it easier for competitors to spot incompatible aid and either initiate domestic legal proceedings or inform the Commission.
Wider monitoring will take the form of checks by the Commission of a larger sample of national measures. As Member States are submitting information sheets to the Commission concerning the measures that are adopted on the basis of the GBER, what is known is that the Commission is already earmarking the measures that it wants to subject to ex post monitoring. The Commission has not revealed the criteria it uses to select measures for monitoring and probably it will never make them public. Auditors, for example, in order to prevent the companies which they audit from hiding irregularities in their accounts choose a random sample of transactions for individual checks. It is unlikely that the Commission will choose randomly samples of national measures. It is more likely that they will concentrate on larger and more complex measures.
The third safeguard is ex post evaluation. Strictly speaking this is not about potential misuse of aid. It is more about the potential waste of public resources through legally compatible means. The Commission has been criticised by the European Court of Auditors for not evaluating the effectiveness of the measures which it approves. By not knowing whether State aid achieves its objectives, the Commission cannot know whether the aid is considered to be compatible with the internal market and is in fact capable of achieving the public policy objectives which are defined in the EU Treaty.
It should be remembered that although the Treaty prohibits in principle State aid, it exceptionally allows certain types of aid for the purpose of achieving certain public policy objectives. These policy objectives correspond to economic conditions that cannot be brought about by the normal functioning of the market. But if Member States or the Commission do not measure the effects of the State aid measures they implement, how can anyone know whether approved aid or legally compliant aid supports public policy, as the Treaty and the case law require?
Do you know we also publish a journal on State aid?
The European State Aid Law Quarterly is available online and in print, and our subscribers benefit from a reduced price for our events.
Ex post evaluation
Therefore, the Commission, belatedly in my view, has begun to request Member States to carry out ex post evaluations of three types of measures:
- measures with large budgets,
- measures with novel features, and
- measures that can impact on competition because market conditions or technology change fast.
In order to help Member States, the Commission has also adopted guidelines on different evaluation methodologies.
It appears that so far the measures that are approved on condition for ex post evaluation are those for which the GBER makes the evaluation unavoidable. The GBER provides that schemes with average annual budgets that exceed EUR 150 million can be implemented for longer than six months only if Member States submit an evaluation plan to the Commission for approval.
So far the evaluations of the following six GBER-based schemes have been approved by the Commission:
SA.40324: Spain: development of industrial technology R&D
SA.40098: Finland: R&D projects
SA.39669: Hungary: regional development
SA.39273: UK: regional aid, broadband and risk finance
SA.38830: Poland: regional development
SA.38751: Czech Republic: regional investment incentives
Public hearing at the European Economic and Social Committee
On 26 June the EESC organised a public hearing on “State Aid to Firms: Is it efficient and effective?”[1] The hearing focused on the SAM and one of the sessions was devoted to ex post evaluation. According to a Commission official, 11 Member States had already notified evaluation plans for 18 new large schemes. Those schemes had a total budgetary expenditure close to EUR 12 billion and covered regional aid, R&D&I, broadband and risk capital.
Most speakers thought that evaluation of large measures was a good idea. But there was also criticism. The main critical remarks were the following:
- Apart from the large schemes mentioned in the GBER, evaluation should not apply to smaller measures with a limited impact on competition.
- Evaluation itself is a costly process.
- The Commission guidance on evaluation mentions only certain methodologies and that appears to be limiting the options of Member States. Other, equally credible, methodologies are also available.
An assessment of the criticism
As a speaker correctly observed, evaluation is like an onion. You peel back a layer to discover yet another layer underneath. Evaluation is a complex process that has to be adjusted to the specific facts, targets and circumstances of each measure. The fact that the Commission guidance paper lists only certain methodologies does not mean that it seeks to impose a standardised approach. The discussion that followed the various presentations indeed revealed that the Commission had an open mind as to how evaluation could be carried out.
Undoubtedly, evaluation itself is not a costless process. Evaluation experts can be expensive and the process itself can take a considerable time. Even worse, in the end it may not produce unambiguous results. What this criticism misses out is that evaluation generates new knowledge. As public authorities gain a better understanding of the impact of State aid, there will be progressively less need to evaluate it or at least evaluate it in detail. Therefore, the cost of the evaluation should in principle decline as we learn more how State aid influences commercial decisions.
Even for smaller State aid measures it still makes sense from a public policy perspective to carry out what may be called an “evaluation light” exercise. Just because the amount of public money that is committed to a State aid measure is small, it does not necessarily follow that public authorities should be at liberty to waste it. As remarked by a participant at the public hearing, the Commission should concentrate in preventing cross-border distortions rather than waste of small amounts of public money that may have a negligible effect on trade.
Strictly speaking, of course, State aid rules and procedures apply only to situations that have a cross-border effect. In fact, this is the principle on which all of the internal market rules are predicated. If the effect is purely local, the rest of the European Union need not be concerned.
There are at least two counter-arguments to this view. First, is it safe to assume that for small amounts of money national procedures allow for wastage while for larger amounts all of a sudden prudence is used? I do not think so. It is more likely to believe that public officials have the same attitude on how they manage public resources at all levels of expenditure. It also makes more economic sense to support lighter evaluations for smaller amounts and smaller schemes.
Second, there is a non-negligible risk that distortions from small measures may cumulatively have a bigger impact. As the same firms can be eligible for different kinds of aid for different underlying eligible costs, and given the fact that in certain market configurations a small cost advantage may be capable of tipping the balance of competition, it cannot be excluded that smaller amounts of aid will not produce large distortions. Hence, a certain degree of discretion should be left to the Commission which should not be bound by budgetary thresholds such as that laid down in the GBER.
Possible improvements
The first evaluation plans that have been approved by the Commission indicate that the biggest difficulty concerns the issue which is at the heart of the evaluation logic. To evaluate properly, one needs to know what would have happened without the State aid. In the jargon of any evaluation methodology, this is called the definition of the “counterfactual”. Since it cannot be known how the aid recipients would have fared without the aid, normally the counterfactual ex post statistical studies is defined by comparing the aid recipients to a similar group of firms that have not received aid.
The problem is that no two firms are identical. Even if they start from identical positions and with identical resources, over time their performance will diverge as their managers make different decisions. It is the nature of the competitive process.
I expect that most evaluation plans will encounter problems in establishing a credible counterfactual. But even if it was possible to benchmark the performance of the aid recipients against that of their cohorts, experience suggests that empirical studies have to filter out many unexpected changes in the surrounding economic environment. Some evaluation techniques are expressly designed to filter out these perturbations. But not every eventuality can be thought of beforehand and taken into account. The point is that ex post evaluations may not succeed to produce clear and unambiguous results.
This unsatisfactory outcome is something that the Commission guidance paper should have taken into account. If the results are not decisive Member States will not know what to make of them and what to expect of the Commission. Remember, the Commission will authorise extension of the measures subject to evaluation only after Member States submit to it the results of the evaluation studies. Should Member States abandon measures whose evaluation will not lead to decisive outcomes?
The Commission should have defined standards of proof of increasing level of rigour. There are at least four such levels.
At the lowest level, evaluation studies should produce evidence to confirm that no aid is wasted in the sense of being granted without any incentive effect. This is the minimum that should be demanded. Several special reports of the European Court of Auditors have found that sometimes up to a third of granted aid has no incentive effect.
At a higher level of analysis, the aid measure that is evaluated should demonstrate that it has delivered by increasing the values of its immediate targets, be they employment, output, R&D, training, etc. Member States should be able to collect data on whether the targets have been reached.
At a yet higher level, the aid recipients should do better than any similar firm that does not receive aid. If they do not perform better, the aid is shown to be ineffective even if it is not wasted in the strict sense of the word. It is at this level that the counterfactual has to be established and a proper comparator group identified.
And, at the highest level, the beneficial effects of aid should be shown to outweigh the detrimental effects from distortions of competition. Evaluation at this level would not only take into account the costs from competition distortions but also the costs borne by public authorities for running aid measures and the costs incurred by beneficiaries for complying with the requirements imposed on them by granting authorities. The Commission guidance paper does not request Member States to carry out a full blown costs benefit analysis of all the aspects of their State aid measures.
Conclusions
Ex post evaluation has been presented as one of the safeguards in the SAM initiative. It is formally intended to protect the integrity of the decentralisation of State aid implementation. But it can also stand on its own. With or without decentralisation, ex post evaluation is necessary so that the Commission can know whether the aid it declares compatible with the internal market is truly capable of achieving those public policy objectives for which aid is exceptionally allowed by the Treaty and the case law.
Proper ex post evaluation is an intrinsically complex task. Given its complexity, cost and time length, at least two recommendations for improvement of the current process can be made.
First, evaluation should ideally be carried out in a light form for smaller measures and through more extensive and robust means for larger and more distortionary measures.
Second, the Commission should define different standards of proof of evaluation results. All evaluation studies should provide information on whether the aid was necessary and whether the immediate policy targets have been achieved. The Commission should also consider the margin by which the performance of the aided firms should exceed that of the unaided comparator group. Member States should know what constitutes success or failure of a State aid measure.
————————————————————————————-
[1] This is the link to the programme and documents of the public hearing:
http://www.eesc.europa.eu/?i=portal.en.events-and-activities-state-aids-firms.
