Overview

In the last 20 years, research on automated image analysis in dermatology has been focused on diagnosis support, mainly on the diagnosis of melanoma.

Other potential application areas of automated image analysis in dermatology, such as the automated assessment of image quality and adherence to examination and image acquisition protocols have been neglected. Such quality and adherence assessments are, however, of capital importance in asynchronous tele-dermatology settings, where examinations and image acquisitions are performed on an eventual basis by primary healthcare personnel and not by dermatologists or trained dermatology technicians. Even if adequately trained, tele-dermatology examinations acquired by primary healthcare personnel will be much more prone to protocol and photographic errors. This will lead the reading tele-dermatologist to invalidate the examination and request the primary healthcare facility to repeat the examination. When this quality assessment is performed manually and assynchronously by the tele-dermatologist, the staff at the primary healthcare facility will receive this information that the examination is invalid only hours or days after the patient’s visit. The primary healthcare facility staff will have to contact and ask the patient to return for this new examination. These errors delay diagnosis and cause costs to the healthcare system and discomfort to the patient.

An automated validation of a tele-dermatology examination that checks image quality and adherence to the examination and image acquisition protocols which should have been employed for a skin cancer, leprosis, psoriasis or other specific examination would be of enormous help. It would ideally provide immediate feedback on the correctness of the execution of that examination and point protocol and image quality errors. This would allow an examination to be repeated while the patient is still at the primary healthcare facility and would relieve the tele-dermatologist of the task of providing image quality reviews.

In the State of Santa Catarina in Brazil, because of the State’s demographic composition, skin cancer represents a disease with morbidity rates of high epidemiologic concern. In 2018, accordingly to the Brazilian National Cancer Institute, more than 36% of the new cancer cases will be skin neoplasms. The estimate of incidence for 2018 is 140.8 new cases per 100.000 inhabitants. Melanoma and non-melanoma skin neoplasms are, together, the most common cancer types in Santa Catarina. Fortunately, Santa Catarina has a large-scale asynchronous telemedicine network that covers 100% of the State and has been operating for more than 14 years, integrating primary, secondary and tertiary healthcare in a single infrastructure, the Santa Catarina State Integrated Telemedicine and Telehealth System (STT/SC).

The existence of this system facilitated the development of an extended tele-dermatology model, capable of integrating the entire dermatologic care process into this telemedicine and telehealth infrastructure. Between January, 2014 and June, 2018, the STT/SC processed 83,100 tele-dermatologic examinations that were performed at primary healthcare facilities and required tele-dermatological patient triage. A total of 75,832 (91.25%) of these examinations were considered correctly performed from the point of view of image quality and acquisition protocol. This means that between January, 2014 and June, 2018 a total of 7,268 tele-dermatology examinations were invalidated and had to be repeated.

Objectives

The objective of this project is to develop a software technology capable of automatically assessing the (a) quality and (b) protocol adherence of teledermatology examinations. It should provide primary healthcare personnel performing these examinations with fast feedback on the technical adequacy of the dermatologic examinations that were acquired and uploaded by them.

This automated examination quality assessment software technology should be able to be transparently integrated into the examination submission process of a large scale assynchronous teledermatology system and should provide feedback on the adequacy of the examinations in a time-frame of no more than a few minutes when running as a process on a typical dektop computer or virtual machine on a server.

This project will develop and field-validate automated image analysis and interpretation methods based upon deep learning technologies that support the automated assessment of image quality and image acquisition protocol adherence for teledermatology examinations in large-scale asynchronous telemedicine infrastructures. Research on image analysis in dermatology has been focused on diagnosis support. When considering large-scale teledermatological patient triage and clinical management, however, examination quality and protocol adherence assessment is of key importance: Examinations performed by non-specialized primary healthcare personnel are more prone to protocol and photographic errors, leading to the need to repeat them, which delays diagnosis and causes costs to the healthcare system and discomfort to the patient. Modern deep learning-based image classification, object and pose detection and focus analysis techniques can be employed to develop automated methods that check compliance to all aspects of examination and image acquisition protocols and can provide immediate feedback to primary healthcare personnel.

Vision statement

In the short term we intend to develop and field-validate image analysis and interpretation methods based upon deep learning technologies that support the automated assessment of image quality and image acquisition protocol adherence for teledermatology examinations in large-scale asynchronous telemedicine infrastructures. This R&D will base upon well-established state-of-the-art deep learning technologies, which will be adapted to our application. For this purpose we need to build an R&D team, composed of graduate and undergraduate students and experienced researchers, and acquire a specialized GPU server for training deep learning solutions.

In the long term we will integrate these technologies in the production environment of our statewide tele-dermatology infrastructure, which is expanding in order to operate nationwide, and also disseminate and offer them to other similar initiatives. This means that this project has the potential to have a direct, real impact on healthcare activities occurring at a nationwide level, changing the tele-dermatology process in order to support automated feedback on examination quality and protocol adherence. The GPU server will have a role in this phase also, providing the means for online image analysis and feedback.

We expect to gradually reduce the rate of invalid examinations through automated feedback to primary healthcare teams.

Potential impact

Research on image analysis in dermatology has been ongoing for more than 20 years and started at the same research group around Stolz et.al. at the University of Munich who, in the mid-1990’s developed the 10x contact dermoscope and the ABCD-rule used today as standards in dermatological diagnosis [13,14]. Recent research, with the launching of the Kaggle IMA205 Skin Cancer Challenge, has been targeted on using deep learning techniques. Al relevant research in computer-aided-dermatology these 20 years has been focused on automated dermatological diagnosis support

Diagnosis support is, however, not the main need in dermatology: in more than seven years operation of a large-scale tele-dermatology network that performed more than 99,000 examinations, we identified that the main need here is wideaccess to quality examinations.

Dermatological care for pathologies such as leprosis or skin cancer is difficult to provide at primary healthcare facilities in remote locations: it depends on a medical speciality. Telemedicine-based support for patient triage and telehealth-based clinical management can be provided, even on a large-scale basis, when the process of performing the examination and photographic registration of the patient and her lesions is outsourced to the local primary healthcare staff. Our initiative, with only six tele-dermatologists working part-time was able to supply the demand of a whole State with more than 6 mio. inh. We reduced waiting times for a dermatology consultation that were up to 6 months to about 72 hours and eliminated the waiting lists.

The STT/SC presently performs more than 3,000 tele-dematological examinations/month in more than 300 primary healthcare facilities. From these, 200-300 examinations are invalidated each month due to photographic problems or lack of adherence to the examination protocols. The assessment of the examination’s technical adequacy is presently performed manually by the dermatology team. This means that each year thousands of patients have to be called back to the primary healthcare facility in order to repeat the examination, which can mean having to travel a hundred or more kilometers on precarious roads.

An automated validation that checks image quality and adherence to the examination and image acquisition protocols which should have been employed for a skin cancer, leprosis, psoriasis or other examination would be of enormous help. It would provide immediate feedback on the correctness of the execution of that examination and point protocol and image quality errors. This would allow an examination to be repeated while the patient is still at the primary healthcare facility and would relieve the tele-dermatologist of the task of providing image quality reviews.,,,,,

Details of the project

Our statewide teledermatology infrastructure has been operating for more than seven years and has, until November, 2018, supported the execution of more than 99,000 tele-dermatology examinations. The service is offered at 313 primary healthcare facilities distributed among 265 municipalities. It supports not only tele-diagnostics and patient triaging, but also integrated support for operational protocols and clinical management processes. These protocols guide the primary healthcare facility staff and the specialist through the complete patient care process, support findings reporting, risk assessment and clinical management. Our intiiative has successfully addressed two key issues:

(a) Definition of Methodologies and Processes, including protocols, results, examination guidelines, the development of training courses for healthcare teams in the execution of tele-dermatologic examinations and the structuring of primary healthcare dermatology services.

(b) Development of a Technological Infrastructure that supportd the dermatologic diagnostic and care process and integrates into the existing statewide telemedicine infrastructure.

A detailed report of our results is given in [JTEH].

Methods and Processes

We developed two groups of Telemedicine-Oriented Dermatologic Protocols and Clinical Management Guidelines:

(a.1) individual Examination Protocols and Image Acquisition Protocols for lesions presenting suspicions of (a) skin cancer, (b) psoriasis, (c) leprosy and (d) other dermatoses, compiled and formalized into a Protocol for Dermatologic Photographic Recording (see Additional Material for a graphic); and

(a.2) tele-Dermatologic Patient Referral and Management Protocols, formalized in the form of: a workflow (integrated into STT/SC), a risk assessment protocol and a clinical management protocol.

We also developed primary healthcare staff Training Strategies for:

(a.3) recognition of elementary lesions;

(a.4) drafting symptomatic diagnostics focusing on: skin cancer, pre-malignant diseases, psoriasis, leprosy, superficial dermatoses and mycoses;

(a.5) execution of image acquisition procedures, including contact dermatoscopy, panoramic and close up patient photographs.

Technology Infrastructure

In order to support this set of processes we developed:

(b.1) a web-based teledermatology module integrated into the STT/SC portal supporting the defined workflow17;

(b.2) web-based image acquisition and dermatologic consultation registration tools integrated into the STT/SC complying with the defined protocols17;

(b.3) templates for structured dermatologic findings reports adherent to the protocols defined in17, supporting the protocols and encouraging the standardization of the descriptions of lesions;

(b.4) controlled vocabularies used with structured reports, aiming at the standardization of lesion descriptions, enabling real-time indexation of reports and generation of detailed epidemiological data; and

(b.5) a mobile application for easier access and execution of tele-consulting activities performed between primary healthcare staff and telehealth-based specialists.

The teledermatology process we developed was submitted to and approved by the State Health Commission (CIB/SC) in their 179^th ordinary session in 2013. It defined the protocols for the triaging of dermatology through telemedicine: (i) workflow, (ii) risk assessment, (iii) clinical management, (iv) photographic image acquisition and (v) the electronic form for the documentation of dermatologic examinations. All these protocols were implemented through the STT/SC.

Evidence

As of November, 26^th, 2018, 8,832 tele-dermatology examinations have been invalidated due to technical reasons and the impossibility to be used to perform patient triage. Evidence we published in [JTEH] shows that invalidation rates, which were 44% at the beginning of our project, have reduced dramatically, but have stabilized around 7% to 8% of the examinations, even with our continuing distance education efforts (October 2018 data: 220 invalid in a total of 3,291 tele-dermatology examinations).

Image processing approaches employing convolutional neural networks and deep learning techniques have, however, evolved very fast in the last four years. The development of a set of image processing modules able to pre-process tele-dermatological examinations in order to assess adherence to individual image acquisition protocols and provide immediate feedback on technical examination quality has become possible. These modules could verify if examinations are (a) in focus, (b) depict the expected anatomic structures and (c) contain the picture elements foreseen in the protocols, such as lesion-tags and rulers.