Statement of Intent 1 July 2007 - 30 June 2010
Part 1: Strategic framework (continued)
Measuring our performance - our performance framework
Measuring performance is an important part of the managing for outcomes environment. The Public Finance Act, which was amended in December 2004, sets out new requirements for information on departmental operating intentions, including specific measures for impacts, outcomes and objectives. The Ministry needs to know what difference it makes in supporting New Zealand’s security objectives and contributing to the security of other countries and whether our interventions are cost effective.
What will we do to find out about what difference we make?
We assess the impact of our outputs on:
- audit and assessment of performance
- management of equipment procurement
- policy advice.
We do this by considering whether:
- the Government and key stakeholders value advice and feedback provided by the Ministry as being relevant, timely and adding value
- improvements take place over time with respect to supporting New Zealand’s security objectives and contributing to the support of other countries.
The key performance measures and standards we use are:
- stakeholder feedback
- maintaining ISO 9001 accreditation in relevant work areas
- reviewing HR trends in the public and private sector (because with some Ministry roles we are in competition with the private sector)
- our own assessments.
Cost effectiveness of the Ministry's interventions
Section 40 (d) (ii) of the Public Finance Act requires departments to include the main measures by which they will assess their future performance including the cost effectiveness of interventions. Cost effectiveness is not defined by the Act or by the central agencies.
The provision of cost effective information poses a number of measurement challenges for the Ministry. Reporting on the cost effectiveness of interventions requires knowledge of both their costs and impact. In the context of defence activities and related policy advice, the benefits of the Ministry’s activities cannot be measured exclusively in monetary terms. Also, whilst some components of cost can be calculated, not all the costs associated with the actions or advice of the Ministry can be reliably measured (e.g. the Ministry may recommend a military solution with a humanitarian impact difficult to qualify or quantify).
Nevertheless, the Ministry is committed to a programme of developing meaningful and relevant cost effective information consistent with the expectations of its stakeholders. The Ministry is particularly interested in demonstrating to stakeholders and the public that it provides value for money for the goods and services that are delivered by the Ministry.
The Ministry also recognises that cost effectiveness cannot be considered in isolation from strategic decision making processes, performance, and risk management. Accordingly the measures and standards the Ministry intends to develop and use will involve an iterative, longer-term process that will provide ongoing opportunities to enhance the quality of the Ministry’s contribution to defence outcomes.
Key principles which underpin the Ministry’s cost effectiveness programme include:
- use of cost benefit information which is relevant, appropriate, transparent and reliable to further enhance the Ministry’s strategic contribution to key stakeholders, and
- Ministry-wide resource prioritisation is informed by robust organisational risk management disciplines and cost benefit information to improve the performance and capability of the defence organisations.
The cost effectiveness of each of the Ministry’s three output classes can be considered in the following way:
Policy
There are two levels at which cost effectiveness can be considered for the policy output class.
Costs incurred directly by the Ministry (Departmental expenditure)
The Ministry is able to influence the level of resources applied to the provision of policy advice in a number of ways. These include prioritisation of the work programme, recruitment of an effective and efficient workforce and working collaboratively with others. It is inherently difficult to determine what level of policy advice is the right level. The Ministry therefore uses quality measures as a proxy for the cost effectiveness of its work.
Costs influenced by the Ministry (Non Departmental expenditure)
The Ministry has a major role in the process, which determines the shape and size of New Zealand’s defence capability. The policy advice provided to Ministers influences the quality of decisions made in the defence sector, which have significant cost implications for New Zealand and ensures that New Zealand is secure and contributes to the security of other countries.
The Ministry has well defined protocols for ensuring that policy advice includes all relevant information to enable good quality evidence based decisions to be made. This includes ensuring that expenditure can be clearly prioritised by ensuring the advice:
- articulates government priorities
- is presented clearly in a way that facilitates easy understanding
- includes the benefits and cost implications of options presented
- includes other options and their costs, and what the trade-offs are.
Management of equipment procurement
The Ministry’s Acquisition Division is responsible for the effective and efficient management of the procurement of new or replacement military equipment or the significant modification or upgrade of existing military equipment. The Division ensures its effectiveness through:
- adopting best practice contestable procurement practices
- ensuring that the functional performance specification provided by the New Zealand Defence Force is articulated in a manner that allows an effective and contestable procurement process to be undertaken
- managing delivery risks throughout the procurement stage.
The Ministry’s procurement activities are one part of a wider capability procurement process that is focused on ensuring that value is not diminished or destroyed within that process. The procurement process is a major driver of value in selecting and delivering assets. However, the functional specification stage, which is a New Zealand Defence Force responsibility, is the main driver of the capital and operational costs of those assets.
Cost effectiveness can be considered at two levels. First, the level of resources applied in the procurement process. Second, the facilitation of a fair and transparent process, which allows the market to respond in a positive way.
Costs incurred directly by the Ministry (Departmental expenditure)
Total resources applied in the procurement process comprise the cost of Ministry personnel, associated costs such as travel for both Ministry and seconded New Zealand Defence Force personnel, technical experts, risk consultants, legal costs etc. The Ministry is managing projects worth $2.92 billion over the next 10 years. Acquisition related costs of procurement are approximately forecasted at $3.773 million for 2007 with 32 Full Time Equivalents (including New Zealand Defence Force secondees and excluding any outside contractors) working on these projects.
Given the risks involved in procuring defence equipment (including requirements to meet capital budget, key milestones and functional specifications), it is not appropriate to assess cost effectiveness on a least cost method. It is also not appropriate to assess a base ratio of procurement costs to project expenditure as different projects require significantly different levels of resources, depending on the profile of the equipment being acquired, including risk, maturity of design and manufacturing processes, complexity of the overall design, value, duration and location.
The Acquisition Division manages the resources and effort applied in the management of defence procurement by applying robust risk management principles and ensuring high quality advice is obtained throughout the procurement process. For the purposes of measuring cost effectiveness the Ministry is considering the appropriateness of developing a benchmark to indicate the level of procurement cost and resources expected to be utilised for projects of differing size and complexity. This benchmark will be indicative only and will be used as a management tool for budgeting and planning the level of resource required against the complexity of the project and forecast spend. The management tool will then allow some comparative analysis to be undertaken across years and projects to assess relative costs.
Costs influenced by the Ministry (Non Departmental expenditure)
It is not possible to assess the financial impact of advice provided by the Acquisition Division on the price achieved through the tender process but it is possible to confirm that what was delivered and paid for was consistent with what was contracted for. At a higher level, it is also possible to consider how lessons learnt from procurement projects (projects which are derived from the Defence Long-Term Development Plan) are fed back into the Defence Long-Term Development Plan.
Audit and assessment of performance
The Evaluation Division is responsible for assessment and audit of the performance of the New Zealand Defence Force and the Ministry’s procurement, replacement or repair of major military equipment. The Division also audits controls on hazardous substances under the control of the Minister.
The Division has well defined processes to ensure that its evaluation programmes are clearly prioritised and implemented by:
- meeting the expectations and requirements of the Minister
- focusing on appropriate strategic risks and issues
- following sound procedures to deliver a high quality result
- making recommendations that are designed to improve the performance and capability of the defence organisations.
The Evaluation Division seeks to provide assurance to key stakeholders about the ability of the defence organisations to meet their mandate effectively and efficiently by providing risk intelligent information, establishing performance and capability weaknesses and providing recommendations to remediate identified weaknesses.
Proposed measures
|
||||
|---|---|---|---|---|
| Output | Confirmation from the Minister that the current work programme meets expectations | Feedback from key stakeholders /sponsors as to the efficiency of response and quality of output | Monitoring the results of self review and external evaluations of performance | Internal measures in relation to quality of programmes and individual project management, resource allocation and risk management, including efficiency based quantitative measures |
| Policy |  |
|
|
|
| Management of equipment procurement | |
|
|
|
| Audit and assessment of performance | |
|
|
|
Other measures under consideration |
|||
|---|---|---|---|
| Output | Benchmarking activities and expenditure against other similar agencies | Identification and response to key risks associated with capability and performance | Level of resources applied relative to the risk profile of projects |
| Policy | |
|
|
| Management of equipment procurement | |
|
|
| Audit and assessment of performance | |
|
|
The Ministry is working on the definition of appropriate timeframes for measuring and reporting against selected measures for all its key output activities.
