Performance and Risk Based Oversight

Introduction

Performance is organisation’s overall capability to continuously manage its operations compliantly and safely. Traficom assesses performance with predefined assessment criteria that are tailored for the specific transport mode, domain and type of operations in question. The results of performance assessment along with justification and evidence form organisation profile. Profile is Traficom’s subjective and justified interpretation of the current state of organisation’s performance.

If organisation’s safety management performance is assessed to be high, Traficom can lighten the extensiveness of oversight for that particular organisation (in accordance with applicable oversight regulations) and instead support by other means further development of customer’s safety management. If safety management performance is assessed to be poor, Traficom prioritises that organisation in its oversight programme and focuses special attention to those areas of operations that have most potential for development.

Based on performance assessment results i.e. organisation profiles Traficom manages the following areas of its operations:

• to which organisations oversight actions should primarily focus on,
• with what type of oversight actions the most safety impact would be achieved,
• what is the scope and depth of different oversight actions,
• when oversight actions should be applied,
• how often different organisations are overseen,
• how much inspector work the above mentioned require.

Application of performance and risk based oversight model

Performance and risk based oversight model is applied to Traficom’s aviation, maritime and railway transport customer organisations. The general assessment process and tools are the same for all three transport modes. Assessment criteria and detailed oversight plan specifics vary with each transport mode and specific domains within them. For example assessment criteria for aerodromes are different than those for airlines and criteria for railways are different than those for ship owners.

Aviation

In aviation sector EU regulations have required Civil Aviation Authorities to implement risk based approach to oversight since 2012 (e.g. Reg 965/2012). The different types of operations that Traficom oversees based on performance and risk levels are:

• flight operations,
• aerodrome operations (ADR and SEC domains),
• air traffic management and air navigation services (ATM/ANS),
• flight training (ATO and FSTD domains).

Additionally risk based oversight is used in ACAM (Aircraft Continuing Airworthiness Monitoring) and ramp inspections (Safety Assessment of Foreign/Community/National Aircraft). Current development suggests that regulatory requirements for risk and safety performance based oversight will be gradually introduced to other aviation domains as well in the near future.

Maritime

In maritime sector national Ship Safety Control Act (370/1995) states that “Ship safety inspections must be carried out as frequently and efficiently as required for control purposes---”. In order to define and eventually justify adequate inspection frequency and efficiency Traficom applies performance and risk based oversight model to domestic ship owner companies in Finland. The approach is similar to Port State Control inspections which are risk based and applied to international maritime traffic.

Railways

Out of the three transport modes regulatory requirements for performance and risk based oversight have been in railway legislation for the longest period, starting in 2010. In railway transport Traficom applies its performance and risk based oversight model to the following operations:

• holders of safety certificates,
• holders of safety authorisations,
• railway training organisations.

The aforementioned also include heritage operators and National rail infrastructure manager.

How do we assess performance?

Traficom’s performance assessment emphasises practical implementation of safety management and organisation’s safety management system’s overall ability to manage safety (excl. maritime sector where SMS is not required from all domestic ship owners). Verification of compliance with regulations is mainly assessed during initial certification process and as a part of oversight activities. In contrast performance assessment also pays special attention to the following points among others:

• efficiency of processes and their different stages including information gathering, decision making, use of results and follow-up on actions done,
• adequacy of process results (e.g. risk assessment),
• utilization of information and knowledge to improve safety (incl. continuous improvement of safety management and internal oversight done by organisation),
• aspects of safety, organisation and just cultures,
• comprehensive and consistent evidence of the above,
• past, current or near future changes in operations that may require follow up oversight,
• scope and special characteristics of operations.

Performance assessment is an internal process done by Traficom’s inspectors. It aims to gather all available relevant knowledge and information on organisation acquired by Traficom’s different units such as:

• observations and results of previous oversight (organisation audits, inspections, desk top evaluations, technical inspections of means of transport such as aircraft ramp inspection or ship inspection results),
• observations during initial certification process and in later stages for example during special and additional approval processes,
• incident and other safety occurrence data and analysis,
• data received from customer (safety discussion meetings, conversations, e-mail exchange etc.).

This data is used methodically to assess organisation’s performance and risk level into organisation profile and acts as back up and justification for assessment. Assessment is done by expert group (when applicable), for example in aviation process oversight and certification inspectors as well as safety analysts and safety management experts that know organisation in question attend assessment meetings.

After initial organisation profile is comprised, it is kept up to date by review sessions which are done at least twice in a calendar year. However profile updates are also done whenever there is new information available that could potentially affect organisation’s performance and consequently oversight. Additionally, results of organisation profiles are used in the end of calendar year to update and review next year’s oversight plan.

What benefits does it bring? Why use performance and risk based approach?

In Traficom’s point of view compliance solely does not guarantee safety. Traficom implements performance and risk based oversight in order to achieve the most impact with its activities. “We want to be where we are needed the most” is the slogan used for Traficom’s performance and risk based oversight approach. It enables more efficient and effective use of safety aurhority’s resources. Even more importantly it enables Traficom to better support its customers to achieve safety by means of partnership management that is tailored specifically for each organisation’s needs. This is supported also through so called safety discussions with customers where Traficom introduces them with their own organisation profile and encourages organisation to use the results to improve their safety management proactively (i.e. well before and outside of traditional oversight activities). The emphasis is on open and co-operative partnership between the Safety Authority and customer organisation.

From customer’s point of view this also means more opportunity to affect the oversight services they receive: when organisation is able to demonstrate high performance, the extensiveness of oversight may be reduced which saves their time and human resources as well. For organisations whose performance has room for improvement, organisation profile provides a clear and justified picture of what type of oversight to expect. Additionally organisations are given more ways to demonstrate sufficient safety management since performance assessment focuses on practical arrangements. This in turn adds to their commitment to safety because each organisation is encouraged to adapt safety procedures to their specific operations (even though they have already achieved minimum level of compliance with regulations).

Traficom also regularly reviews organisation profiles within the same type of operations in parallel in order to see on-going or emerging trends among organisations’ performance. For example if assessment results show that all or the majority of organisations struggle in the same areas of safety management, Traficom can organize training and/or information events or publish additional guidelines on these subjects.

Lastly, regulations in railway, maritime and aviation sectors require that national authorities plan their oversight based on performance and risk assessments.