Evidence Based Training: How to improve this concept in regular line operations

Let’s talk together and exchange ideas around the main theme of digitalization in aviation – all points of view are welcome to let things move forward! We need to be reactive to keep the sky safe and give accurate answers to the global pilot shortage!

#EBT #Aviation #PilotTraining #skills #qualification

Evidence-Based Training (EBT) is today on everyone’s lips.

EBT starting point

In Europe, everything started a few year ago with directives from the EASA/IATA and the ICAO:

Also in China the CAAC, in its article 19 / 2019, gives Chinese Airlines a timescale in order to implement EBT.

EBT in reality – different approaches

Last year, while visiting different airlines, I have seen several approaches in the way EBT is introduced and applied by various companies.

Especially concerning the implementation of EBT in the simulator, there is a large spectrum in the way it is carried out. This goes from almost full freedom for the instructor in the choice of events / technical problems/airports/weather – often based on a point system, each event having a given number of points depending on the difficulties it will cause to the trainee. Other airlines still apply for a classical predetermined program, without real surprise for the candidate, except for some variations (left or right engine failure for example).

Some airlines also offer instructors the opportunity to access the file of the previous trainee’s qualifications record, in order to know his strengths and weaknesses before going into a simulator session. So a specific program can be performed based on improving the weak points.

These different approaches have their pros and cons but are fine as long as they are conducted in a constructive way!

An excellent way of improving the quality of the training and enhancing pilots’ performances is to confront trainees to the classical startle effect created by some “black swan events”. Such unexpected situations are great experiences for pilots when done in a positive environment and are also a great way to improve their resilience.

After such an unpredictable event, each pilot is in a position to realize what are his strengths and weaknesses.

When realizing his behavior and reactions during such a “black swan event”, it is possible thereafter for the pilot to mainly base his operations on his demonstrated strengths. And by recognizing where his weaknesses are, to be able to, at least, know them and at best develop strategies so that these weaknesses don’t become an obstacle in the overall crew performance.

Open issues of EBT during pilots’ check rides, line training and qualifications

 If the implementation of EBT has already well progressed in the simulator, my feeling is that in line operation it is underexploited, usually only applied during the qualification parts, without exploiting today’s possibilities to improve all the potential that EBT gives to the airlines and the trainees.

During discussions I had with several heads of training, I realized that on the qualification’s part there are still some open issues.

The basic criteria of EBT are described in the ICAO document. They are:

1. Application of Procedures
2. Communication
3. Aircraft Flight Path Management, automation
4. Aircraft Flight Path Management, manual control
5. Leadership and Teamwork
6. Problem Solving and Decision Making
7. Situation Awareness
8. Workload Management

Each of these competencies will then be assessed using the so-called behavior indicators linked to the given competencies.

Here are for example the behavior indicators (BI) linked to competency “Leadership and Teamwork”

Extract from: EBT ICAO Manual Doc 9995.en

What are the problems encountered in airlines implementing EBT?

One main point I have heard from some heads of training is that the qualification of trainees becomes somewhat too uniformized (lots of green blocks, meaning rating good to excellent, 3 or 4).

From a pure statistic purpose, in order to see the progress of a fleet, it is still ok. It is easy to show areas where improvement or degradation has happened. But on an individual basis, it makes it more difficult, based only on competencies, to realize which task might be a problem for a given trainee.

For sure, understanding and correcting the trainee about some weaknesses are mainly parts of the debriefing. But for an efficient follow-up, how can a head of training realize a weakness that repeats itself, only by analyzing the result of a competency?

An example given by an instructor, who had no comment possibility on the qualification form, was that he felt it difficult to rate an average good performance at landing (Competency: Aircraft Flight Path Management, manual control), when he experienced during a flights’ rotation 12 perfect landings in fine weather but difficulties in the only one with crosswind! Does he have to rate the performance as excellent knowing there was one exception, or does he have to give an overall bad rating due to the only failure? This is not an easy task for the instructor – and how to forward this important information into the system in order to have an eye on this specific problem?

In my airline, I find our approach convenient: We rate the competencies, then we tag up to 3 behavior indicators linked to this competency and, especially when we have a grading below top level, we have the possibility to make a comment with a written description of the encountered reason justifying such a grading. Thus, we can help the system (Chief Instructor / Fleet Chief) and also the candidate, giving him the most exact “mirror” of his performance.

What about implementing EBT in normal line operations?

In a recent schematic representation, the following components are the foundations of EBT:

• competency
• resilience
• learning from the positive
• using the data

These pillars are key elements supporting Evidence Based Training… But how to emphasize them in normal line operations?

These 4 points must be the main focus of an efficient crew-centered debriefing oriented towards EBT. As already mentioned in a previous blog article, nowadays, debriefing is usually only based on pilot’s remembering with all the known weaknesses of human memory, especially in a dynamic situation or a saturating task (such as perception issue problems, bias of the reality, etc.)

This traditional approach can now be massively improved!

Nowadays, most pilots are equipped with a computer or a tablet; the connectivity exists in airports or even in aircraft, and flight data on most modern airplanes is transmitted immediately after landing by wireless QAR. So there is the possibility to use this data in order to better understand, but also to compare the own perception with the reality of facts. A great way to learn about your own limits and situation understanding.

CEFA Aviation proposes a tool that will be of valuable help in order to increase the efficiency of Evidence Based Training, especially for line operations.

DRIVEN BY DATA (4^th pillar)

CEFA AMS is a tool enabling every pilot or instructor to create an animation based on the data from the flight recorder within minutes after landing, in order to use this animation during the debriefing as a solid base for performances’ evaluation and crew centered debriefing.

Using the flight data allows creating animations of specific parts of the flight that are useful for analysis and performances understanding during the debriefing. In the case of different perceptions, a mirror of what really happened can be used to face objective facts and see if the pilot’s perception was correct.

Compared to some tools giving only a statistical approach and some values (for example landing distance, screen height at take-off…), CEFA AMS, in addition to this statistical data, offers the comprehension of the detailed flying process leading to such a value.

DEVELOP THE RESILIENCE (2^nd pillar)

As mentioned in a previous blog article, resilience in normal operations can be enhanced by building up confidence and competencies. Confidence can be increased by allowing a better understanding of performances, especially in a challenging environment (like crosswind landing). Experience can also be improved by sharing knowledge! Animations are today the most used means to share knowledge anyway.

LEARN FROM THE POSITIVE (3^rd pillar)

For the instructor, displaying very positive elements of flight performances allows showing the strengths of a trainee, in order to improve her/his confidence. Learning from positive sequences of own performances is a confidence booster.

During the preflight phase, we also have a potential learning benefit. Why do trainee pilots have to go through the same bad experiences as their predecessors, when it is possible to show them “best practices” in a real-time 3D animation, in order to prepare difficult approaches and minimize mistakes. A detailed analysis of a good performance is an excellent tool for flight preparation.

On the trainees’ side, it brings also the possibility to analyze positive outcomes from the flight performed by the instructor, to see good practice.

Contact us to get a demonstration of this new tool, aimed at pilot training and performance understanding!

Now, what do YOU think about EBT brought to an individual level? Share your personal thoughts in the comments below!

You can also come & discuss with Pierre personally on CEFA Aviation stand, booth #404 during the APATS in Singapore from 3^rd to 4^th September 2019.

Find all the other blog articles gathered here.

You can also contact Pierre via LinkedIn or via this link. Talk to you soon!