1: CRF design teams were designated and followed a defined process for end-user acceptance testing to ensure it aligns with the workflow and research milestones.

2: Design and testing included experts in areas under study such as database design and curation; statistics, neuroimaging; biospecimen management; outcome assessments; and clinical assessments etc.

3: Before data entry was started, the CRF forms were reviewed by an independent researcher with appropriate expertise. 

1: Data governance team should include all the important and key stakeholders such as: data curator, steward, analysts.

2: Subject matter experts should be involved from the outset (study design time) to ensure that the data is sufficient and aligned to fullfill the study objectives.

3: Technical experts should be involved from the outset (study design time) to ensure that the variables to be collected are appropriately specified.

1: Data base fields conform as appropriate to FITBIR NINDS Common Data Elements for Traumatic Brain Injury; CDISC; CDASH; and Abbreviated Injury Scores.

2: Pre-specified data ontology at design-time.

3: Published data elements used as far as possible. However, when not possible, deviations from published specifications clearly documented and have robust scientific justification.

1: Specifications for floating point, integer, factor or free text

1: When data is extracted from the database, it should be easy to recognize what the variable names refer to.

2: When there are many variables of the same kind, the names follow a pattern that is consistent.

3: The same name should be used across the CRF and database design so that confusion is minimised. 

1: Similar data will be collected in one table.

2: Repeating data is not collected in separate fields, but rather identified by a time point indicator. 

1: The GUID (Global Unique IDentifier) or the GUPI (Generated Unique Patient Identifier) is a method of ensuring that patients are not duplicated across datasets. 

1: Duplication identification process in place to assure single entry of codes assigned to unique subjects by institution across repository data sets (e.g., blood samples, imaging sequences and other assessments).

2: Audits are conducted to reconcile record linkages and assure no duplicates are entered (e.g., image sequence files not submitted twice; subject CRF only entered once in database; or biospecimen have unique IDs at different timepoints and only entered once).

3: For data across multiple platforms assure same subject ID is used in the database. For example, Global unique identifier (GUID). 

1: Mandatory elements require a value or explanation for reason missing.

2: Curation team is responsible for reviewing and accepting or rejecting explanations.

3: Data completeness for key variables is checked against pre-specified study design goals and minimum standards for data completeness in key areas are met.

4: Quality control is in place to ensure that completed clinical measurements or investigations such as imaging meet the specifications in the study protocol.

1: Definitions are agreed upon at design.

2: Codes are defined as appropriate to study settings: missing data from hospital record (e.g. not recorded) versus missing data from a study appointment (e.g. subject did not return etc.). 

1: Avoid free text fields when possible.

2: eCRF has automated error flags to prompt immediate alert to review entries that don’t pass validation with option to override when appropriate. 

1: ‘Other’ checkbox included in field response options as appropriate with free field text for description.

2: Free-text fields set as containing PHI to avoid inadvertent export/release of personal information. 

1: Scheduled queries are run for data checks across tables/CRFs for variables prone to conflict.

2: There should be rules to ensure that incompatible choices are excluded both within a data element (e.g. cannot be both male and female) and between related elements (e.g. male cannot be pregnant).

1: Database has flags for identification of when CRF completion or a study visit is overdue.

2: Automated weekly messages sent to coordinators listing overdue eCRFs.

3: Central study staff responsible for monitoring overdue CRFs. 

1: For data entry: cells for skipped questions are closed.

2: Clear directions on CRFs for when to skip questions. 

1: Avoid enrollment errors by requiring completion of a CRF with checklist confirming that eligibility criteria have been met for enrolled subjects.

2: For subjects that need to be enrolled within specific window of time there are timestamp checks in the eCRF in place to verify the eligibility window.

3: Date of birth falls within the enrollment age criteria. 

1: Where decimal places are to be limited, it should be clear to the user and prevent them from entering the value rather than a truncation performed with the user being unaware.

1: ETL process logged.

2: Permissible value ranges in the ETL software are study specific rather than relying on institution permissible value logic. 

1: Systematic approach to translation and cross-cultural validation of study material in multi-language settings.

2: Psychometric transformations considered. 

1: Manuals compiled by multidisciplinary team of stakeholders and involve data curation expertise. For example,   Imaging studies and biospecimen collection procedures are detailed for study personnel and end-users.

2: Manuals specify how eCRF data is collected, when, by what metrics and under what conditions.

3: CRF fields have links to relevant data manual sections. 

1: Training materials developed and disseminated via print, webinar or in-person training.

2: Study implementation checklist includes documentation for training requirements according to study personnel roles.

3: Testing process in place: written exams with scenarios for coding data; video demonstrations for examiner competency to administer of outcome assessments; competency for performing research imaging sequences; or testing with mock patient records for complex forms.

4: Post-launch procedures for assuring new project personnel also fulfill training requirements.

1: Layout of paper and eCRFs are easy to navigate with simple wording.

2: Examples are provided for coding of complex response choices.

3: The CRF is designed and validated to be unambiguous to different users.

4: eCRF forms are designed to be presented based on rules to appear at times that are appropriate depending on the study design: Hiding irrelevant forms and to function as a prompt to promote timely data collection. 

1: Random audits of clinical source documents against data entered on hard copy and eCRFs.

2: Double data entry audits of selected forms that are known to be prone to errors.

3: Audit of eCRF and hard copy logs for biospecimens; audit of complex assessments by experts.

4: Training and on-line documentation of how to handle exceptions.

1: Certification processes specified in protocols.

1: Therapists administering the Functional Independence Measure (FIM) tool do not know who was given an experimental treatment or a placebo.

1: Change logs maintained and audited.

2: Workflows and ETL (extract / transform / load) processes are described. 

1: Table and study form fields relationships are clearly identified in the data dictionary. 

1: Uniform collection standards are in place.

2: When uniform measures are not used by hospital units or labs, the CRF provides conversion guidance.

3: Inter-site sampling differences are described in the data dictionary. 

1: Phantom standardization of MRI scanners across study sites.

2: Appropriate assurance of quality and comparability between sites and within sites and assurance of ongoing compliance over time.

3: SOP aligned with agreed imaging acquisition standards.

4: Adequate training of technical personnel across sites.

1: Clear Biospecimen Standard Operating Procedure (SOP) for sample preparation.

2: Lab SOP aligned with agreed sample handling standards.

3: Adequate training of lab personnel across sites.

4: Published / national / international standards should be specified and adhered to where possible.

5: Audit trail of sample processing and storage.

6: Protocol deviations filed when protocol not followed.

7: Audits of time intervals for drawing, processing and freezing are within protocol windows for each specimen type. 

1: Normal ranges are determined for each assay.

2: Assays are standardised as far as possible.

3: Assay substrate are defined (e.g. ionised vs. unionised fractions, whole blood vs serum, total vs protein bound as appropriate).

4: Normal ranges should consider/be customised to pathology or subject group in question. 

1: Vial labels IDs entered into bio-repository logs via bar code scanner. 

1: Anticipated and unanticipated data quality issues are identified and appropriate solutions/mitigation strategies are devised in a timely manner.

2: There is a process in place for ongoing systematic quality checking and analysis of data during data acquisition to identify unexpected data quality issues in order to remedy them in a timely manner. 

1: Protocol is designed to allow for surrogate to respond to selected questions when patient is too impaired to respond. CRF indicates when surrogate is responding.

1: For example, testing is carried out if fields such as ‘years of education’ and ‘level of education’ are automatically combined into a new variable summarising educational status.

2: Coding is tested on a subset of data, ideally by multiple members of the study team.

3: There is a centralised log / version management of all transformations used. 

1: Rules in place to check whether the apparently missing element should be completed or not for a particular study stratum or time point.

1: Schedule of standard reports for data quality checks.

2: Ad hoc reports for identification of new issues. 

1: Site visits for audit of subject hospital records against CRF and eCRF entries.

2: Submission of redacted hospital records for audit against eCRF entries.

3: Independent double entry of any data transferred to the database from paper form, either for full dataset or for representative subset. 

1: Database rule checks in place to identify conflicts in data entries for related or dependent data collected in different CRFs or sources.

2: Inter-rater scoring of MRIs is done independently by 3 experts on 10 cases per month. 

1: Queries in place to identify the following types of scenarios: For 9 of 10 study sites a mandatory and critical variable is only missing in 5% of the subjects. For site 10 data is found to be missing in 30% of the subjects. For a single site study, several personnel have been collecting longitudinal study data with low numbers of missing data. A pattern of increasing numbers of missing data has been identified with a newly hired data collector.

2: Variables known to be problematic are regularly monitored by curation team and delays in site resolution of issues are reported to governing committee.

3. Aggregate data are examined for inconsistencies such as lower average mortality in patients with GCS 3 than GCS 4-5.

1: Criteria in place for closing and locking eCRFs in the database. 

1: Database platform has feature for export of CRF field definitions and schema of table relations.

2: Dictionary has annotations as needed to provide guidance on coding decisions.

3: Dictionary cites primary sources where appropriate for uniform data collection (e.g., NINDS TBI CDEs)