Management of Lessons Learned by the NZDF
Section 3: Organisation and process
Expectation: NZDF has effective processes and systems for gathering, analysing and sharing individual knowledge and organisational experience for future use.
Expectation: Data gathered is being analysed and validated to form lessons learned.
Expectation: Lessons learned lead to changes that result in improved military or business practice.
EARLLS
3.1
The NZDF’s global lessons learned application EARLLS is an on-line system that facilitates the gathering of data to support change management and knowledge development. It allows for the banking and exchange of information related to the conduct of any activity. Information is input into the system by way of post activity reports which, as well as containing observations and issues that are analysed for lessons learned purposes, also provide general commentary on the conduct of the activity involved.
3.2
EARLLS is not an issue resolution tool for circumstances that require urgent remedies, such as a breakdown of equipment while on deployment or exercises. There are other more immediate methods established for units to report these types of incidents and seek an urgent solution.
3.3
An activity report including such incidents would nevertheless be subsequently recorded in EARLLS, along with observations. Depending on the nature of the incident the observations might simply confirm mitigation of the risk of something similar happening again. Alternatively, observations could be the means for indicating the action required to prevent such incidents taking place in the future, in which case EARLLS would be used to monitor the implementation of changes required for the particular lesson to be learned.
3.4
EARLLS resides on both the NZDF SWAN and DIXS networks.7 The SWAN network holds all activity reports and observations while DIXS provides access to those that are restricted or unclassified. Commanders/managers classify reports as they are validated and submitted to the system. Where possible, the aim is to reduce the classification level to restricted or below to maximise the availability of information and provide access to all potential users.
3.5
Any individual can submit observations to EARLLS for consideration but only commanders/managers and their delegated staff can approve and submit reports for analysis and action. This ensures that the workflow of issues is manageable for the activity, the reporting unit, and the individuals (subject matter experts) responsible for actioning any changes or modifications required. For the most part, force element units have personnel designated to manage activity related reporting procedures.
3.6
As well as enabling issues to be ‘pushed’ out for analysis and action by subject matter experts, EARLLS is a repository of information available to assist planners. When preparing an exercise or activity, planners can extract relevant information from post activity reports of previous iterations of that or similar activities. Information available may include commentary, observations, and lessons identified and learned.
3.7
EARLLS provides an automated subscription facility that enables personnel to receive all reports on specific areas or topics of professional or personal interest.
HQ NZDF
3.8
The main users of EARLLS within HQ NZDF are Strategic Commitments and Intelligence Branch and Capability Branch.
3.9
Strategic Commitments and Intelligence Branch reviews post activity reports held in EARLLS when preparing briefs for Senior National Officers appointed to overseas missions. Incoming Senior National Officers are briefed using prior activity reports and outgoing Senior National Officers are debriefed on their reports that are produced no later than seven days after prior to departure from the mission.
3.10
Strategic Commitments and Intelligence Branch personnel only infrequently receive issues for actioning as subject matter experts.
3.11
Observations or issues relating to equipment are routed to Capability Branch for analysis and possible action in relation to future procurement.
3.12
Most HQ NZDF Branches are in some way involved in learning lessons though it is most notably associated with project management activities. Internal processes rather than EARLLS are used for these purposes.
HQ JFNZ
3.13
HQ JFNZ is the application administrator and main user of EARLLS for lessons learned relating to overseas missions, major exercises, and other readiness or collective training activities. Over 70 percent of observations entered into the system relate to OLOC/DLOC generation activities.
3.14
As part of the introduction of EARLLS, HQ JFNZ established a Lessons Learned Cell consisting of three personnel. Amongst other things, the role of the Lessons Learned Cell involves:
- managing lessons learned data in EARLLS for COMJFNZ and Component Commanders,
- conducting trend analysis on observations from operational activities and reporting findings to COMJFNZ and Component Commanders,
- monitoring the lessons learned process generally to ensure the Services and others are progressing issues for resolution,
- providing assistance and advice to the Services and other users of the EARLLS application,
- promoting the use of EARLLS and delivering training to all personnel who require it.
3.15
Although the Lessons Learned Cell has wide ranging responsibilities over the use and management of EARLLS across the NZDF, its primary focus is the Chief of Staff Mission Observation Screening (COSMOS) process. This is the HQ JFNZ mechanism for handling all mission post activity reports and attached observations. This involves a specific review process undertaken under the auspices of the COSMOS Group (CoS and HQ JFNZ Divisional Heads) to identify issues and trends that are warranted for analysis and immediate action.
3.16
Improvements made to the process of reporting and actioning feedback from pre-deployment training, for Operation CRIB in Afghanistan in particular, were an example of the outcome of this screening process. A further example was the analysis of post-activity reports from units associated with repatriation from Afghanistan of the first NZDF casualties. This led to improvements in procedures before subsequent evacuations were required.
3.17
As well as administering the COSMOS process, the Lessons Learned Cell closely monitors the progress of issues generally and regularly reports to the COSMOS Group, COMJFNZ and Component Commanders on actions taken.
3.18
Three post activity reports are required for all mission rotations and deployments. They cover pre-deployment training, post arrival in theatre, and the overall conduct of the operation. The latter is submitted shortly after the rotation or deployment is completed. For major missions, such as Operation CRIB in Afghanistan, an initial report is prepared to capture observations from the theatre reconnaissance undertaken prior to each rotation.
3.19
As well as these post activity reports, situation reports are submitted on a regular basis during the mission and these may contain observations for lessons learned purposes. All post activity and situation reports are processed through EARLLS.
3.20
More recently, psychologists have begun entering the outcomes of their mission debriefing reports onto EARLLS to provide an important source of information for lessons learned purposes.
3.21
HQ JFNZ has established subject matter experts to action issues and observations in each of the staff areas of that headquarters.
Services
3.22
Force element ships, units and squadrons all have specific personnel trained in EARLLS procedures. These individuals control the flow of observations or issues to their commanding officer for inclusion in activity reports.
3.23
A requirement to process submissions through the chain of command maintains the quality of data entered into the system and ensures the effective routing of issues to appropriate area(s) for resolution. Also, because the EARLLS search facility relies on the classification of observations by topic, correct allocation of these is important when entering submissions into the system. Without this it may be laborious for users to pull the specific data they seek, or retrieve all the relevant information on a particular subject.
3.24
Each Service has nominated subject matter experts to analyse EARLLS issues associated with exercises and readiness or collective training activities. These experts are located within the Navy’s Maritime Operational Evaluation Team, the Centre for Army Lessons, and the Air Force’s 485 Wing.
3.25
Subject matter experts are also in place to analyse Service issues relating to matters such as logistics support.
3.26
For any particular major exercise or activity, post-activity reports may be completed by all of the units involved and cover many areas of interest. Some of these will be resolved and actioned directly with the unit but others will require wider consideration. For these, good communication and coordination is required between the HQ JFNZ Lessons Learned Cell and subject matter experts to ensure appropriate handling of issues.
3.27
The ability to classify observations by relevant mission essential task lists, as these become fully developed, should improve the usefulness of EARLLS for collective training purposes.
EARLLS data
Figure 3.1 shows the total number of unclassified observations entered into EARLLS by HQ JFNZ and the Services over the last five calendar years. The table shows the outcome of those observations, in particular whether they resulted in a lesson being identified, a lesson being learned, or whether the units themselves resolved the issue involved. In the case of the latter, this could be a lesson learned also, but is specific to a unit and not of wider interest.
Fig 3.1: Outcome of Unclassified Post Activity Observations submitted to EARLLS.
Observations |
|||||||
| Status | 2006 | 2007 | 2008 | 2009 | 2010 | Total | % of Total |
|---|---|---|---|---|---|---|---|
| Not started | 0 | 0 | 0 | 5 | 52 | 52 | 1% |
| Analysis Phase | 0 | 1 | 1 | 23 | 119 | 144 | 2% |
| Action Phase | 0 | 1 | 2 | 19 | 64 | 86 | 1% |
| Lesson Identified | 936 | 953 | 805 | 744 | 480 | 3,918 | 46% |
| Lesson Learned | 357 | 550 | 584 | 639 | 412 | 2,542 | 30% |
| Unit Info Only | 51 | 291 | 568 | 548 | 304 | 1,762 | 20% |
| Grand Total | 1,344 | 1,796 | 1,960 | 1,973 | 1,431 | 8,504 | 100% |
3.29
Fig 3.1 shows 8,504 validated observations were entered into EARLLS over the last five years and remain readily accessible for wider viewing. Of these, 2,542 (30%) have resulted in lessons being learned.
3.30
For HQ JFNZ and the Services the percentage of observations resulting in lessons learned was Joint 46%, Navy 19%, Army 9% and Air 26%. We note that the Army has a significantly higher number of ‘Unit Info Only’ observations than the other Services.
3.31
Figure 3.1 indicates that the total of observations submitted into EARLLS in 2010 was less that for the previous three years. Those managing EARLLS have sought no explanation for this. We heard speculative suggestions such as, ‘lower levels of collective training’, and ‘most lessons have been learned’. However, a reduction in post activity reporting might also indicate a reduction in interest in the process and the need for initiatives to regenerate awareness across the organisation.
3.32
Annex A to this report contains a small sample of lessons learned from operational and training activities, to indicate the wide nature of the issues involved.
Other lessons learned processes
3.33
Other formal lessons learned processes are in place for each Service relating to such things as training validation, logistics issue resolution, health and safety (accident prevention), security (incident management), Boards/Courts of Inquiry recommendations, and capability project management.
3.34
While the processes and systems in place to support these functions have differing purposes, the management of activities for each involves a formal process that involves the identification and analysis of issues, and the actioning of recommendations and solutions for future learning. None of these processes utilise EARLLS.
Other lessons learned sources
3.35
Outside organisations provide a crucial and significant source of operational and tactical lessons learned for the NZDF. These lessons learned can also be an important input into doctrine. Military partners, notably the Australian Defence Force, provide the most information.
3.36
There is formal exchange of lessons learned through multi-national military fora. The Naval Capability Directorate, Centre for Army Lessons, and the Air Power Development Centre, all of which are either directly responsible for, or closely aligned with, the development of Service doctrine, generally lead discussions in such groups.
3.37
Other informal sources of potential lessons learned include internal suggestion schemes and networking through established fora and communities of interest.
Findings
3.38
The NZDF has effective processes and systems in place, through EARLLS, for gathering, analysing and sharing individual knowledge and organisational experience for future use in relation to overseas missions, major exercises, and other readiness or collective training activities.
3.39
Extraction of quantitative data from the system for reporting purposes is relatively easy. However, the quality of the data is dependent on the accurate classification of activity reports and accompanying observations when entered into the system. Approval or validation procedures are in place to ensure the quality of data entered into the system.
3.40
While the various areas that monitor post activity reports analyse trends in observations to identify and validate issues that arise, there is little performance measurement undertaken to determine and report the ongoing effectiveness of EARLLS and the lesson learned process it facilitates.
3.41
Other main lessons learned processes appear to be effective for the functional areas and activities they support. However, there is little sharing of information from these across the NZDF.
Recommendation
3.42
It is recommended that:
- HQ JFNZ develops performance measures against which the ongoing effectiveness of EARLLS can be gauged.
- SWAN – Secure Wide Area Network; DIXS – Defence Information Exchange System.
