The Transport Agency assesses investment proposals in the 2015-18 National Land Transport Programme (NLTP) under three factors, being strategic fit (the alignment of the proposal's key transport issues with Government Policy Statement on Land Transport (GPS) results), effectiveness (how well the proposed solution can address the issues) and benefit and cost appraisal (how efficient resources are used to deliver benefits from the proposed solution).
The information relating to developing assessment profiles is organised by activity class groupings, e.g. road maintenance programmes, with guidance provided on each of the three factors within the groupings.
The Transport Agency assesses investment proposals under three factors, being:
The specific guidance on how each of these apply to activity class groupings is provided in relevant sections listed at the bottom of this page.
Assessment of strategic fit considers how the problem, issue or opportunity identified in the strategic case aligns with the results sought under the Government Policy Statement on Land Transport (GPS).
Strategic fit focuses on the problem, issue or opportunity being considered without regard to the possible solution.
The strategic fit assessment considers how the problem/issue/opportunity:
The need for investment comes from addressing a level of service gap. Historically, the gap has not been explicitly or consistently defined.
The One Network Road Classification(external link) (ONRC) defines nationally consistent customer levels of service. Over time, all roads in a particular category should offer an increasingly consistent fit for purpose customer level of service for users. Identifying the gap to the ONRC customer levels of service will become a key input into the assessment of strategic fit for road maintenance and improvements.
No nationally consistent level of service or classification yet exists for public transport or walking and cycling. In these activity classes we use proxies to determine the significance of the problem, issue or opportunity including the geographical classification such as main urban areas or primary routes.
Effectiveness considers the contribution that the proposed solution makes to achieving the potential benefit identified in the strategic fit assessment and to the purpose of the Land Transport Management Act 2003(external link).
Effectiveness assesses how well the proposed activity will deliver the desired results, while considering integration, and whether the solution has been correctly scoped, is affordable, timely and manages risk.
Fit for purpose
The assessment of effectiveness should be fit for purpose, taking into account the scale and complexity of the project or programme and its stage of development.
Examples:
Judgement required
Assessment of effectiveness as a whole is based on quantitative and qualitative evidence, although for each of the six criteria it may comprise solely one or the other. It requires the use of judgement rather than being a simplistic "box ticking" exercise. For large, complex programmes and projects the assessment should be moderated amongst experienced peer reviewers, to ensure different perspectives are debated to arrive at a consensus.
Sufficient evidence must be provided to support the effectiveness rating. This includes as appropriate:
Under development
On occasion, early in the development of some complex activities, e.g. before completion of the programme business case, there may be insufficient evidence to make a confident assessment of effectiveness. An interim rating of M* at the time of inclusion in the National Land Transport Programme (NLTP) indicates there is insufficient information and that further development is required.
The M* rating enables an activity to be included in the NLTP, but funding approval will not be considered until the information is developed and a firm rating of effectiveness is provided.
Low and medium ratings
A low rating means that there is insufficient evidence to show that a solution can successfully address the problem, issue or opportunity. If the effectiveness is assessed as low, the proposed solution should remain in the programme business case stage until the effectiveness criteria assessed as low have been addressed through further development.
Medium effectiveness means that an activity has not achieved the full potential identified in the strategic fit assessment. This may be due to a deficiency in the development process, e.g. sensitivity analysis should have considered more scenarios to provide a fuller view of the risks, or an acknowledged shortfall in the chosen intervention, e.g. a safety intervention that addresses only part of the crash risk. The deficiency or shortfall is not considered by the assessor as significant to the extent that the activity should not be progressed, more that its effectiveness is not as good as it could be.
Assessing effectiveness
The assessment considers all criteria. The overall effectiveness assessment is reported as the lowest rating for any criterion, i.e. an overall M rating will be given when all criteria and parts have either an M or H rating. An overall L rating will be given if any criterion is rated L.
The effectiveness assessment criteria look at how well the proposed activity or programme:
The benefit and cost appraisal considers how well the proposed solution maximises the value of what is produced from the resources used, and the timeliness of intervention.
Assessment of improvement activities uses the benefit–cost ratio as the default approach.
Cost-effectiveness and performance comparisons are used for road maintenance and public transport programmes.
Benefit Cost Ratio
The Benefit Cost Ratio (BCR) is the primary tool used to measure the efficiency of improvement activities.
All improvement activities other than Minor Improvements and specified exceptions (see below) should be supported by the provision of a robust BCR.
The Transport Agency requires that Approved Organisations and the Transport Agency (state highways) use the Transport Agency Economic Evaluation Manual procedures and templates to determine the BCR for improvement activities.
Alternatives to benefit cost ratio
For assessment of road maintenance, existing public transport programmes and road safety promotion programmes, alternative methods may be used in place of the BCR.
A brief description of these methods is provided below:
|
Present value method (PV)
|
The present value of future costs of options are determined and compared to identify the long term least cost option. The method is recommended to determine if replacement/renewal is more cost effective than on-going maintenance. In certain cases the use of benefit streams rather than future costs may be more appropriate. |
| Cost effectiveness method |
Cost effectiveness analysis is used where the objective is to compare the cost of different ways of achieving a given effect (e.g. level of service), or comparing the relative cost of different strategies with different effects. This approach is used to evaluate the benefit and cost appraisal of public transport, maintenance, or road safety promotion programmes by comparing a programme with similar programmes for other organisations. |
| Benchmarking method |
This approach involves making comparisons of programme benchmark and key performance measures against similar regions and the national average. A lack of evidence to support differences from peer group, regional and national averages may result in a lower benefit and cost appraisal rating for the programme. Alternatively it may result in a requirement for a review of the supporting business case as a condition of investment approval. Trends in benchmarking measures over time are used rather than just annual values. |
| Marginal contribution method |
The marginal contribution of programme expansions and incremental new services will be considered through benchmark and key performance measures. The cost-effectiveness of the changes will be considered as well. |
Guidance is provided to help determine benefit and cost appraisal ratings for each activity class grouping.
Non-monetised benefits and additional benefits
If a proposed solution has demonstrable non-monetised benefits, then these should be taken into account and may, if the Transport Agency considers these benefits to be significant, result in a higher rating.
Additional benefits are usually in the form of wider economic benefits that are not specifically covered by the Transport Agency’s Economic Evaluation Manual. The Transport Agency may consider additional benefits as reasonable and may determine a higher rating as a result. Alternatively, it may consider that the additional benefits should be presented as part of sensitivity analysis that will not impact the rating.
Exceptions
Benefit and cost appraisal is not required for some activities.
Activities which are not required to calculate a benefit and cost appraisal include:
Use of generic or default BCR
In specific cases generic or default BCRs may be used in lieu of a calculated BCR for the activity. These are:
No other placeholder, generic or default BCRs should be used.
Peer review
The Transport Agency reserves the right to require a peer review of benefit and cost appraisal determinations and measures, including any non-monetised/additional benefits and adverse impacts, regardless of the scope, prior to an investment decision.
Insufficient information (1* or Low*)
An activity can be included in the National Land Transport Programme (NLTP) when no benefit and cost appraisal has been made or when robust evidence is lacking to support the assessment. In such cases the rating for a benefit and cost appraisal will default to 1* for improvement activities and Low* for programmed activities. The Transport Agency represents these activities as 1* or Low* to indicate that more information is required to achieve a robust assessment profile.
An activity will not be considered for funding approval with a 1* or Low* status.
No rating
If the calculated BCR is below 1.0, then the activity is considered to be economically inefficient. In this case, no rating for benefit and cost appraisal will be given. The Transport Agency may, at its discretion, make an assessment of any non-monetised benefits to determine whether the total of monetary and non-monetary benefits outweigh costs.
Application to improvements
Assuming that the BCR is 1.0 or higher, the benefit and cost appraisal for improvements to local roads, state highways, public transport, and walking and cycling will fall into one of three bands:
BCR range 1- 3
All activities with BCR greater than or equal to 1 and below 3 are prioritised in this band.
BCR range 3 - 5
All activities with BCR greater than or equal to 3 and below 5 are prioritised in this band.
BCR range > 5
All activities with BCR greater than or equal to 5 are prioritised in this band.
Application to continuous programmes
Continuous programmes, e.g. road maintenance, existing public transport services, road safety promotion and minor improvements, are assessed with a benefit and cost appraisal rating based on their relative cost effectiveness or performance comparisons.
The ratings are:
Incremental benefit and cost appraisal
Assessment of incremental benefit and cost appraisal is required for option selection, optimisation (other than between projects within packages) and scope change proposals.
A number of criteria need to be satisfied before assessing an activity for inclusion in the NLTP for funding approval.
Requirements for NLTP inclusion
Inclusion of an activity in the NLTP requires that:
Requirements for NLTF funding
Seeking approval for funding from the National Land Transport Fund requires all of the above, plus:
The Transport Agency may consider any exceptional additional factors not otherwise captured by the 3 assessment factors, including wider economic and community benefits.
These will be specific to the activity or combination of activities being assessed and relevant to determining its overall priority and funding source in the programme.
Evidence will be required if additional factors are to be considered.
Links to guidance on developing assessment profiles for each of the activity class groupings are provided below.
