The Extensible Clinical Protocol: Setting Up Clinical Trials for Successful Execution

Role and Importance of the Clinical Protocol in Drug Development
The protocol contains a high-level description of, the reasoning behind, and the schedule for every activity that will occur during a clinical study.  It incorporates input from various stakeholders, including scientific, operational, regulatory, and marketing constituencies.  The protocol is used not only by study investigators and coordinators but also by numerous individuals and groups at the sponsor organization, the sponsor's vendors, the research institution, regulatory bodies, etc. 

Most importantly to the ultimate success of a study, the clinical research protocol serves as the origin for all data generated during the study and therefore for all knowledge resulting from such data.  Data collection systems (EDC), data and trial management systems (CDMS and CTMS), and data analysis and reporting systems (SMS) are all informed directly or indirectly by the clinical research protocol.  Therefore, if an error is contained in the protocol it is easily propagated throughout all information systems, affecting all research operations. 

A couple of examples illustrate the importance of an operational error-free protocol.  A complex Phase 3 research study asks whether health-related quality of life (HRQOL) is affected by therapy.  HRQOL is included as a secondary endpoint in the protocol.  Suppose that the protocol schedule of events includes administration of a validated HRQOL instrument at baseline, but that, because of an oversight, administration of the instrument is not scheduled during the therapy period.  The procedure omission could easily be missed during EDC setup.  No data would be collected after baseline, and the secondary endpoint could not be evaluated.

Subtle errors may be just as damaging.  Consider a protocol in which the schedule of events narrative (text) indicates that “a brief physical examination with vitals will be performed at each visit.”  For brevity, however, the schedule-of-events table in the protocol (which is also copied over into the protocol synopsis, the investigator's procedural training manual, the statistical analysis plan, and the clinical study report shell) simply indicates “Vitals” at each visit.  Suppose that EDC is used, and that the EDC screen for each visit (which was setup by a data manager who relied on the protocol’s schedule-of-events table as her primary source) has defined user input for vital signs at each visit screen but none for physical exam findings.  It is likely that investigators will not question the EDC screens, and a key component of safety evaluations will have been overlooked, perhaps until a sponsor’s site auditor notices the protocol ambiguity.

Initial protocol development usually is not rate-limiting to study startup.  However, amendments to protocols—depending on their magnitude—essentially "stop the presses," impacting all downstream systems and processes, with important implications for operational productivity.  Although solid evidence is lacking, many of these amendments are believed to be avoidable.  Calculating the total cost of a single protocol amendment is difficult and subject to disagreement.  It includes direct (out-of-pocket) costs, such as publishing and disseminating the amendment to investigators, regulators, and IRBs, retraining investigators and renegotiating contracts, reprogramming the EDC system (or republishing and redistributing CRF workbooks) and the downstream data system to collect different data, and implementing any new study-related procedures.  Amendments also have hidden costs, such as sponsor personnel being redirected away from critical-path activities in the amended study (such as enrollment support), and the opportunity costs that arise from postponing activities outside of the amended study under situations of resource constraint.  The direct costs alone of a single protocol amendment in phase 3 have been pegged at $200,000 by one industry consultant (Zuckerman 2005).  Short of amendments, are avoidable operational protocol issues that consume sponsor resources and potentially lengthen study cycle times.  Issues such as ambiguous eligibility criteria, for example, can result in protocol violations (deviations), potentially delaying enrollment and typically requiring multiple communication cycles between investigators and sponsors to dispatch.

Problems with Traditional Protocols
The state-of-the-art in clinical study startup today is reminiscent of the state-of-the-art in complex engineering projects of twenty years ago.  Today's clinical research protocols are created using "dumb" word processing tools that do not themselves ascribe meaning to words and thus offer no ready means of automated content review, updating, reuse, or repurposing.  Make a conceptual change in a protocol today and a human expert must search the protocol in its entirety for each instance where the change might affect the original meaning.  Make a change affecting the schedule of activities table late in the authoring process, and the protocol writer might have your head for lunch. 

Because there are no standard meanings behind protocol concepts and no standard rules for structuring protocol information, today’s traditional protocols are information highway dead-ends.  To make further use of paper- or word processor-bound protocol information requires human interpretation and manipulation of protocol-sourced information, manual data entry into information systems, and human-expert quality systems to ensure that human data management has not corrupted the original protocol intent.  Quality control of the protocol itself must also be accomplished manually, via human expert reading and re-reading of completed protocols, with the goal of diminishing errors at each review cycle.  Word processor spelling- and grammar-checkers are the most sophisticated error-checking tools commonly used, and they cannot reliably distinguish between "two visits" and "too visits". 

Protocols should be organs of knowledge, yet developed traditionally, protocols rely on crude and ineffective methods to enforce quality standards and propagate institutional knowledge:  Approved protocols are used as starting material; modifiable templates offer preferred styles, formatting and guide text; archetypal text blocks are cut and pasted, checklists suggest content, etc.  But when there is a will there is a way, as the cliché goes, and when authors are willing to circumvent the crude processes intended to enforce institutional standards or to diffuse knowledge longitudinally, for whatever reasons, today’s authoring and document management tools fail to prevent it.  People leave a company; an organization updates its references or its SOPs; an institution gathers new knowledge of what makes a protocol “work” within that organization; a new technology prompts a company to venture into a new therapeutic area; firms merge to become new entities.  Each of these events is an opportunity for institutional learning to occur or for knowledge to be lost.  Traditional, paper-based protocol tools fail to retain institutional knowledge and promote its diffusion.  For that, organizations require more than just authoring tools; they require institutional learning systems.

The Solution: Extensible Clinical Protocols
Use of a robust extensible clinical protocol tool potentially addresses all of the shortcomings of paper-constrained protocols described above.  I refer to an extensible clinical protocol as any protocol that meets two criteria:  (1) It uses standard-language, tagged elements to represent common protocol concepts and their associated data values and (2) It uses a standard structure and semantic (meaning-based) rules to allow information interchange between disparate information systems.  Because concepts in an extensible protocol are represented as standard-language, tagged data elements, they can be shuttled to multiple locations within or even between documents.  When the value associated with a concept changes, the change can be reflected in all instances where the concept appears in a document (or theoretically in multiple linked documents).  For example, if a study-drug dosage changes from 75 mg daily to 150 mg daily, all instances where the study-drug dosage was mentioned in a document will likewise change from 75 to 150 mg, whether the dosage was mentioned in body text, in a table, a footnote, or in a dynamically drawn figure.

When the standard language of an extensible protocol is coupled with an industry standard structure for interpreting and transferring concepts, unrelated information systems can essentially "speak" with one another, and each system can "know" exactly what is meant by the other.  Thus, the protocol can communicate directly with an EDC system, telling it the number and timing of visits, the tasks to be performed at each visit, the measures and variables associated with each task, etc. 

An extensible clinical protocol provides a wealth of opportunities for improving operational efficiency and quality that do not exist in today's paper-constrained protocol environment.  For instance, Fast Track Systems' solution for creating extensible clinical protocols, TrialSpace Designer XCPTM (eXtensible Clinical Protocol), uses a configurable, embedded expert system for improving protocol quality that checks for inconsistencies, ambiguities, missing elements, unused elements, etc. and provides authors and reviewers with advisories to alert them to discovered issues.  This system can essentially supplant human quality assurance reviews of the protocol and allows expert reviewers to focus on science without worrying about the logical structure of a protocol.  Because the expert system is configurable, sponsor-specific and group-specific rules may be edited or created whenever changes are needed, for instance, when a new regulatory requirement demands certain protocol language or study procedures.   

Such an expert quality system is practical only when key protocol concepts can be "understood" readily by computers.  Designer XCP does not impose restrictions as to how these concepts are represented in natural-language documents (e.g. task vs. procedure, outcome vs. endpoint, drug vs. medication).  Rather, it associates a single, unique meaning with each conceptual natural-language representation, that is, with each protocol element.  The protocol elements, together with the expert system and other components form the basis of a modeled protocol that can be used to create a protocol document, a related non-protocol document (e.g. Informed Consent Document), or it can be exported into an information system.  The XCP application itself is embedded within Microsoft’s Word application, so the appearance of the protocol document as it is created is immediately familiar to authors and reviewers, again facilitating scientifically focused authoring and review. 

With Designer XCP and the CDISC standards, the tools for creating extensible clinical protocols and the industry standards needed to take full advantage of them are now realities.  In May 2007, a consortium of vendors participated in an end to end workshop intended to test the abilities of disparate information systems to communicate directly with one another using CDISC standards.  Fast Track Systems served as the extensible clinical protocol vendor.  Designer XCP was used to create an XML file containing the contents of an actual Phase 3 clinical protocol.  The export file was structured by XCP in conformance with CDISC’s ODM (operational data model) standard.  Five unrelated data capture systems that are equipped to accept ODM-compliant XML imported the file directly, without technical assistance from Fast Track Systems or communication between EDC vendors.  In each system, the file was able to set up the study forms for data collection.  Although a complete study setup in EDC continues to require manual programming, one major EDC vendor estimated that at least 50%, and perhaps up to 80%, of the information required to populate the EDC system for data collection can be imported automatically within seconds from the protocol itself, without rework or human intervention and its attendant risks of error introduction.

Large pharma’s early adopters are already implementing extensible clinical protocols, fueled primarily by their interests in improving research quality and attempting to create the best possible study design the first time.  With the industry’s early majority expected to test these waters soon, there is reason for optimism that the head-end of clinical research will at last lead the way towards highly efficient clinical development operations.

Reference

Zuckerman, D. Towards a painless protocol. Pharmaceutical Executive; September 01, 2005.