Traficom works to ensure high-quality subtitling services – new quality assessment scheduled for Nelonen

The Finnish Transport and Communications Agency Traficom will continue to monitor the quality of the subtitling service on the TV channel Nelonen. Traficom has previously reprimanded Sanoma Media Finland Ltd for its failure to meet the quality requirements. The company has sent a response to Traficom clarifying its views on the matter. Based on the information received, Traficom will initiate a new quality assessment towards the end of the year. Traficom has the duty to promote the availability of accessible media services for special groups.

In the spring of 2022, the Finnish Transport and Communications Agency Traficom reprimanded Sanoma Media Finland Ltd for its failure to meet the quality requirements concerning the obligation to provide subtitling services on the television channel Nelonen. According to Traficom, the subtitling service on Nelonen did not meet the statutory requirements of being sufficiently clear and easy to understand.

Traficom required Sanoma Media Finland Ltd to ensure that in future the subtitling service on its channel Nelonen is of high quality. Traficom also required the company to report by 5 August 2022 which measures it has taken following the decision, how the measures have improved the quality of subtitles in practice and how the changes have affected the production of programmes and thereby the quality of the service from the viewers’ perspective.

Information provided by Sanoma Media Finland Ltd

In its response, Sanoma Media Finland Ltd presents its own view of the assessment concerning the quality of the subtitling for the deaf and hard of hearing. According to the company, the requirement of being “sufficiently” clear and easy to understand, as laid down in the Act on Electronic Communications Services, should be interpreted as a less strict and more reasonable requirement than being “flawless”. 

The company also notes that the quality criteria set by Traficom guide the development of subtitling services but go considerably beyond the requirements laid down by law. Therefore, the company considers Traficom’s quality criteria to be mere recommendations.

The company says that its subtitling service is based on an application employing artificial intelligence (AI) and that it continuously develops the service further.

According to the company’s own observations, the quality of subtitling on Nelonen has improved significantly.

Traficom monitors closely the development of subtitling on Nelonen

Traficom has met with the representatives of Sanoma Media Finland Ltd to discuss the matter, and Sanoma has demonstrated the new version of its subtitling application to Traficom with two short programme samples. The company selects in advance those programmes and series that require human subtitling or human intervention in the subtitling process based on the company’s previous experience of the programme type in question. However, the majority of subtitles are produced with the AI application.  

In Traficom’s view, the quality of subtitles for the deaf and hard of hearing produced with the new application version used by Sanoma Media Finland Ltd is much higher than before. However, based on the samples presented, Traficom’s preliminary assessment is that the subtitles still fall short of meeting the quality requirement of being sufficiently clear and easy to understand.

• The AI application converts speech into subtitles word for word instead of following speech and dialogue in the most appropriate manner possible, summarising and clarifying content where necessary to ensure that subtitles are comprehensible. Thus, the outcome does not even correspond to the company’s own view of the application producing subtitles that include the content essential to follow the programme.
• The application is also unable to identify speaker changes for example by ending a sentence and beginning a new one or by dividing sections by different speakers on different lines, which makes the subtitles more difficult to understand.
• The application still fails to recognise some words, such as names, even though the number of obvious mistakes has decreased. 

Traficom will re-assess the quality of the subtitling service

Traficom has the statutory duty of assessing the quality of subtitling for the deaf and hard of hearing. To this end, Traficom has issued guidelines on the quality assessment criteria it uses. The criteria are based on a national recommendation on the quality of subtitling for the deaf and hard of hearing. Under the law, Traficom may take into account in its assessments any standards and recommendations concerning the quality of subtitling and technical developments in subtitling services. The national recommendation on the quality of subtitling for the deaf and hard of hearing was prepared in a working group coordinated by the trade union Kieliasiantuntijat ry (Language Experts). Two out of the three television companies subject to the quality requirement participated in the working group and signed the recommendation.

Traficom will continue to monitor the development of quality in the subtitles provided on the channel Nelonen and will launch a new quality assessment towards the end of 2022. As in previous assessments, Traficom will select several different types of programmes and compare the quality of the subtitles provided for them with the quality assessment criteria included in its guidelines.