WCG Clinical reports that failure rates for clinical trials involving the central nervous system (CNS) exceed 85% during Phases II and III. Reasons for this are partly because understanding the functional implications of cognitive impairment demands complex questions that can be subjective and require a clinician’s judgement to score.

These high failure rates raise questions about the current processes for quality assurance. Integrating speech analysis for automated quality assurance (AQUA) provides an effective approach for identifying assessments at risk of quality inconsistencies, allowing for targeted examination and increasing trial efficiency.

The challenge for CNS trials

When considering clinical instruments utilised in CNS trials, it is evident that they play a crucial role in assessing patient outcomes. However, they also present their own set of challenges and obstacles. There is currently no accepted standard for the clinical research industry to follow when selecting and training raters to administer rating scales. Training and certification methodologies used vary in their level of rigor and have diverse cultural and language differences that may impact accurate administration country to country2.

Utilising expert reviewers to assess a small portion of evaluations can indeed enhance the quality of data. However, this strategy is costly and difficult to expand. Numerous instruments include subjective clinical judgment and intricate administration, leading to potential inconsistencies in trial data with even slight scoring discrepancies notably affecting result accuracy. Additionally, raters are not 100% consistent due to being inherently human, a factor not fully mitigated by relying on a limited number of assessments for quality assurance. These human factors bear significant implications for regulatory approval, clinical decision-making, and patient care.

To solve this challenge, AI-driven software solutions such as AQUA, automatically reviews assessment audio to produce high quality QA data in a more cost-effective manner, ensuring that clinical trials have the best possible chance for success.

The solution for more efficient CNS clinical studies

The use of a tool such as AQUA allows for the real-time transcription or retrospective entry of the entire clinical assessment, incorporating instruments like the ADAS-COG or the CDR. This solution crucially distinguishes between the clinician’s instructions, the patients’ responses, and any other speakers heard in the recording. Subsequently, each section of the instrument undergoes further subsection analysis, such as word recall prompts between patient and clinician or instructions provided during a command section (Figure 1).

The alignment between the recording and the transcript is confirmed and a customised quality report is generated for each assessment. This detailed breakdown offers the opportunity for precise review of the administration process.

Linguistics and acoustics: A distinctive advantage in CNS trials

When evaluating quality assurance tools, it’s essential to prioritise those that offer advanced and robust options. The software will need to be able to automatically process the subtle nuances of speech, encompassing both linguistic and acoustic elements. This capability significantly enhances the likelihood of identifying any discrepancies within the assessment.

Linguistics

Software tools such as AQUA not only analyse the literal content of speech but also evaluate elements like emotional tone, verb frequency, pronoun usage to develop a comprehensive understanding of response structures. This methodology guarantees that all raters are evaluated based on uniform, objective criteria, thereby reducing the impact of individual biases or subjective interpretations that may stem from their delivery style.

Acoustics

Voice quality assurance tools offer an objective, fast method for evaluating the rater’s interview techniques and communication skills. By analysing parameters such as tone, pitch, and speech patterns, these tools can assess the clarity, effectiveness, and consistency of the rater’s communication with trial participants. Additionally, they are essential for detecting, solely based on voice, whether a speaker’s identity remains the same or differs.

Automated objective processes can also serve to pinpoint training or remediation requirements. By examining the rater’s interviews, voice quality assurance tools can detect areas necessitating additional training or support. For instance, if a rater consistently demonstrates unclear speech or lacks empathy during interviews, tailored training interventions can be introduced to rectify these shortcomings and enhance overall interview quality.

The results: Eliminating failure rates in CNS trials

The final produced report is comprehensive, flagging major and minor issues (Figure 1; Figure 2). With users able to participate in setting the threshold for what requires remediation, or actively participating in issue clarification, enabling a tailored approach according to different trial requirements and priorities. Analyses have demonstrated that even individuals without clinical expertise can effectively pinpoint quality concerns, meaning the software can also help reduce costs as the reliance on clinical experts can be decreased.

In a recent meta-analysis it was shown that AQUA can reduce the expense of data quality review by 60%, increasing the return on investment (ROI) for trials3. Designed for seamless integration and scalability, the platform can work with trials of any scale and can be integrated directly into Cambridge Cognition’s eCOA service or a third party eCOA via API. Today, products like AQUA are the key to unlocking the full potential of clinical instruments, and furthermore greater success in CNS clinical trials.

Figure 1 (top): An example of a comprehensive report showing a section of output from running AQUA software.
Figure 2 (bottom): An example from a comprehensive report showing a section of output from AQUA software.

References:
1. CNS Trial Failure Rates High As Need for New Drugs Grows | WCG (wcgclinical.com)
2. CNS SUMMIT RATER TRAINING AND CERTIFICATION COMMITTEE:; West MD, Daniel DG, Opler M, Wise-Rankovic A, Kalali A. Consensus recommendations on rater training and certification. Innov Clin Neurosci. 2014 Nov-Dec;11(11-12):10-3. PMID: 25621182; PMCID: PMC4301026.
3. Based on comparison of similar services by similar providers.