Validation of a screening tool for autism spectrum disorder in adults – a study protocol

The global prevalence of autism spectrum disorder (ASD) is estimated to be 1% [1]. However, the prevalence estimates show great variation worldwide, and even in Europe estimates vary 0.4-3.1% [2]. Danish data suggest a prevalence of 1.26% in 2015 [3], 2.1% in 2020 [4], and estimates of a cumulative incidence reach 2.8% [5]. The considerable heterogeneity of the clinical presentation of autism and the presence of mental and physical health comorbidities may present challenges in the assessment. The high prevalence of comorbidities like depression (50%), anxiety (40%), Attention deficit hyperactive disorder/attention deficit disorder (ADHD/ADD) (40-60%) and personality disorders (40-60%) [6] explains some of the challenges associated with the assessment of ASD. Additionally, social phobia and avoidant personality disorder impose considerable differential diagnostic challenges. Furthermore, ASD at the symptomatic and subclinical level alike shares considerable core features with schizophrenia spectrum disorder (SSD) [7, 8], and 23% of patients with SSD also satisfy the criteria for ASD [9]. Within SSD, the International Classification of Diseases, tenth version, (ICD-10) the diagnoses of schizophrenia simplex and schizotypal disorder are characterized by a significant overlap with ASD in terms of their clinical features with a considerable burden of negative symptoms. These differential diagnostic challenges and ample overlapping features impose considerable difficulties on health professionals in general medicine and in psychiatry who need to make correct case identifications and referrals for ASD assessment at specialised clinics.

Correct case identification in the initial phases is becoming more relevant as the number of referrals to specialised clinics is rising. The ADHD & Autism Mental Health Clinic in the Capital Region of Copenhagen has seen a rise in referrals from 110 in 2016 to 650 in 2021 (official data publically availably, by request to corresponding author); see Figure 1. The increasing number of referrals and the difficulties associated with ASD case identification in the general population highlight the need for screening tools of a high validity and reliability. Therefore, the aim of the present study is to investigate the psychometric properties and clinical features of a Danish version of the Ritvo Autism Asperger Diagnostic Scale – Revised (RAADS-R), a self-report questionnaire designed for screening of ASD in adults, which has been widely used in Denmark for a decade.

We hypothesise that the Danish translation will show a sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) comparable to that of the original English version when compared with a control group from the general population. Furthermore, when compared with a control group with other psychiatric disorders (and no ASD), we hypothesised that the above-mentioned parameters will display clinically significant values. Furthermore, a factor analysis will show that subscales of the RAADS-R will converge on a two-factor solution and thereby fit the two-factor construct of Diagnostic and Statistical Manual of Mental Disorders (DSM)-5.

METHODS

Recruitment

A total of 200 patients will be recruited for investigation from the pool referred to our outpatient clinic for diagnostic assessment of ASD (infantile autism, atypical autism and Asperger syndrome).

A control group counting 200 psychiatric patients without ASD = non-ASD (bipolar disorder, depressive disorder, anxiety, post-traumatic stress disorder (PTSD), personality disorders and psychotic disorders) will be recruited through psychiatric outpatient clinics from Mental Health Centre Glostrup, Denmark. A second control group counting 200 patients from the general population without any psychiatric diagnoses will be recruited through market analysis investigators. See Table 1 for inclusion and exclusion criteria.

The Ritvo Autism Asperger Diagnostic Scale – Revised

In 2011, a revised version of the 80-item self-report Ritvo Autism Asperger Diagnostic Scale (RAADS) [10] was developed and shown to be a valid and reliable instrument to assist the diagnosis of adults with ASD. In their systematic review of diagnostic tools for autism spectrum adults of mean normal intelligence, Baghdadli et al. [11] reported that among the screening tools available, only the Autism-Spectrum Quotient questionnaire (AQ)-50, the AQ-S, the RAADS-R and the RAADS-14 had satisfactory psychometric properties. The potential advantage of the RAADS-R over other instruments such as the AQ [12] may be that it includes items referring to hypo- and hypersensitivity in line with the new diagnostic criteria of the DSM-5 and the ICD-11.

A Dutch study [13] examined RAADS-R and two short versions of the AQ (the AQ-28 and the AQ-10) in 210 patients referred for ASD assessment and in 63 controls. Among the 210 patients, 139 received an ASD diagnosis and 71 received another psychiatric diagnosis. The PPV figures indicated that these tests correctly identified ASD patients in almost 80% of the referred cases. However, the NPV figures suggested that only half of the referred patients without ASD had been correctly identified. The sensitivity and specificity of each of these instruments were much lower than the values reported in the literature. In their study, the sensitivity of the RAADS-R was the highest (73%), whereas the AQ short forms had the highest specificity (70% and 72%). However, due to the low number of controls, the power of this study was low.

A French study [14] performed a psychometric validation and diagnostic accuracy study of the French version of the RAADS-R on a sample of 305 adults: 105 with ASD without intelligence disorder (ID), 99 with psychiatric disorders and 103 non-psychiatric controls. The French version of the RAADS-R demonstrated good reliability and diagnostic validity, suggesting that it may help clinicians in the diagnostic process in adults with ASD without ID. However, the finding that a two-factor structure better fits the results requires further validation. The study also identified a need for further studies of the RAADS-R in the different psychiatric disorders due to the relatively high false-positive rate (55.6%) of ASD.

Danish version

For this study, a Danish version of the RAADS-R (RAADS-R-DK) was prepared that is semantically and culturally equivalent to the original version (see supplemental material A02220118_-_supplementary.pdf). Three native Danish clinicians (one doctor and two psychologists with combined > 10 years of clinical experience in ASD) with academic level English/Danish bilingual capacities translated the RAADS-R independently. Differences in the translations between the three clinicians were discussed at open meetings where all three agreed to a first Danish version of the RAADS-R. Version 1 was then presented to another team of clinicians (one senior consultant, two specialists in clinical psychology and two nurses) from our specialised clinic, leading to further refinement of the translation. Having cleared the translation in the clinical group, a Danish version was then presented to a native English psychologist with a professional career in Danish/English translation. He was instructed not to consult the original English RAADS-R version and asked to back translate our Danish version into English. Differences between the original RAADS-R and the back translation were discussed between authors and translators until a consensual version had been achieved.

Autism-Spectrum Quotient questionnaire

The AQ is a widely used questionnaire for the assessment of autism in adults [12]. Designed to be short, easy to use and score, it contains 50 questions with five sub-scales: social skill, attention switching, attention to detail and communication [15]. The AQ has not been validated in Denmark even though referrals are regularly based on the AQ. It is therefore relevant to compare the psychometric properties of the AQ with those of the RAADS-R-DK in the clinical sample of patients with confirmed autism.

The DSM-5 and the ICD-11 incorporate the severity of symptoms as a specifier in the diagnosis, thus emphasising the need for measuring the symptom severity in terms of disability. We propose the World Health Organization – Disability Assessment Schedule (WHO-DAS) 2.0 and the five-item World Health Organization Well-Being Index (WHO-5) to quantify this.

The World Health Organization – Disability Assessment Schedule 2.0 and five-item World Health Organization Well-Being Index

WHO-DAS 2.0 [15] is a generic assessment instrument for health and disability. It is a tool to produce standardised disability levels and profiles. It is applicable across cultures in all adult populations and is directly linked at the conceptual level to the International Classification of Functioning, Disability and Health. The WHO-5 [16] is among the most widely used questionnaires assessing subjective psychological well-being.

See Figure 2 for an overview of the inclusion and investigations.

Considerations related to the statistical analysis

Sample size

We estimated the minimum sample size based on a power analysis. We aimed at a power value of 0.8, meaning an 80% chance of correctly rejecting the null hypothesis that the sample means are identical. Based on a Swedish study [16], the mean (± standard deviation (SD)) RAADS-R total for the control sample was 33.8 (± 27.6) and the mean (± SD) RAADS-R total for the clinical sample was 118.7 (± 38.8). We used G*Power to calculate the minimum sample size necessary to correctly reject the null-hypothesis with t-test based on difference between two independent means based on an alpha value of 0.05. The recommended minimum sample size was found to be five. The actual power for the estimate was found to be 0.976, which is well above the minimum of 0.8. The sample size consists of 600 individuals divided into three groups: the potential ASD sample, a group of other psychiatric patients and a healthy control group. RAADS-R scores will be collected from all participants in the three groups of the study, blinding patients, the primary research investigators and the clinical investigators. The scores will be unblinded after completion of all clinical evaluations.

Validity and reliability

The evaluation of the validity of the construct that we measure with the RAADS-R-DK will be done through various approaches. We will evaluate whether the RAADS-R-DK, which was originally designed for compliance with DSM-4-TR, is also relevant for the DSM-5.

We will compare four different factor models to the data. The RAADS-R comprises four intended subscales. However, earlier factor analysis has converged on a two-factor solution, which suggests a good fit with the two-factor construct of the DSM-5. We will try to fit a three-factor solution based on the original RAADS-R and a one-factor solution as well. Multiple goodness-of-fit-indexes will be used.

To be able to decide on a cut-off value we will explore the relationship between true positives and false positives with receiver operating characteristics. Furthermore, the positive and negative predictive power of the RAADS-R-DK will be evaluated as a measure of its usefulness in clinical decisions.

Ethical considerations

The study will be conducted in accordance with the Helsinki Declaration of 1964, including any subsequent revisions. Participants will be included only after having signed an informed consent form based on oral and written information. The participants may at any time choose to withdraw from the study without being required to explain why and without this affecting the person's future treatment. The study is approved by the Ethical Committee of the Capital Region of Denmark (approval number H-21039423), and the project will be reported to the Danish Data Protection Agency. The study investigators are under the impression that the questionnaires and investigations will not lead to any discomfort for the subjects. There are no known short-term or long-term risks associated with the present study.

Outcome

The primary outcome is the value of the total RAADS-R scores ranging from 0 to 240. We expect to find a high specificity and sensitivity (more than 90%) at a cut-off of 65 compared with the control group. We further expect to find significant differences in means and variations in test scores between the control group of psychiatric patients without ASD and the ASD group. We also expect to find significant differences between the control group of psychiatric patients without ASD and the control group without any psychiatric disorder. Furthermore, we expect the Danish translation to show a sensitivity, specificity, PPV and NPV comparable to those of the original English version. In addition, we expect that a factor analysis will show that subscales of the RAADS-R converge on a two-factor solution and thereby fit the two-factor construct of the DSM-5. With respect to the secondary outcome, we expect that the positive predictive values of the RAADS-R-DK will provide a strong clinical indication with regards to the selection of patients/individuals in need of further diagnostic examination in a specialised clinic.

Trial registration: ClinicalTrials.gov NCT05213286

DISCUSSION

Due to its heterogeneity and complexity, the assessment of ASD is a time-consuming process even when undertaken by experienced clinicians. The high prevalence and under-dimensioned health system in place to serve this demand emphasize the need for advancement of the diagnostic process. The need for a more efficient case identification methods is evident.

Only two tools had satisfactory psychometric properties (the AQ and the RAADS-R) in a screening process, but a concern exists as to the applicability of the diagnostic properties. Although the RAADS-R has been translated into Danish, none of the previous translations have been validated, and the original version has never been tested in a Danish population. Furthermore, the clinical value has not been tested worldwide in high-powered studies in an outpatient clinical sample. The need for studies investigating the sensitivity, specificity and PPV of psychometric tools to aid in the diagnostic assessment in a Danish clinical sample is thus apparent. As the RAADS-R is not a diagnostic tool, its relevance in the diagnostic process needs to be determined. The present study aims to overcome the above-mentioned limitations of previous studies and follow their recommendations and is expected to form the basis for future research into diagnostic tools in Denmark for adults with ASD. As such, the study is important for the development of future screening and diagnostic tools.

Correspondence Rizwan Parvaiz. E-mail: rizwan.parvaiz@regionh.dk

Accepted 12 October 2022

Conflicts of interest Potential conflicts of interest have been declared. Disclosure forms provided by the authors are available with the article at ugeskriftet.dk/dmj

Cite this as Dan Med J 2022;69(12):A02220118