Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The importance of structured reporting in healthcare, specifically in the context of pathology reporting. It highlights the benefits of synoptic reporting, such as improved accuracy of coded renderings in standard vocabularies like SNOMED CT, increased completeness of reporting, and facilitation of research. The document also touches upon the challenges of implementing structured reporting and the role of standard terminologies in ensuring interoperability and data integrity.
Typology: Slides
1 / 51
This page cannot be seen from the preview
Don't miss anything!
The ASPE Expert Panel on Cancer Reporting Information Technology
College of American Pathologists
and
The Altarum Institute
Washington, DC
September 2009
Acknowledgements
We would like to acknowledge the following experts who participated in the panel meetings and contributed their valuable information and insight.
Dr. Monica de Baca , Physicians Laboratory & Avera McKennan Hospital
Mr. Christopher Carr , Radiologic Society of North America
Dr. Anni Coden , IBM Research
Ms Kate Flore , Altarum Institute
Dr. James Ostell , National Center for Biotechnology Information, National Institutes of Health
Mr. Eric Prud’hommeaux , World Wide Web Consortium (W3C)
Professor Munindar Singh , Department of Computer Science, North Carolina State University
Drs. John Sowa & Arun K. Majumdar , VivoMind Systems
Professor Jun Yang , Duke University
Executive Summary
Cancer is the leading cause of death in our nation; at any given time there are approximately 16 million patients with a cancer diagnosis in the United States^1 [Horner 2006]. The administration has set an ambitious goal for nationwide implementation of interoperable health records, with the aims of reducing medical error, controlling cost, streamlining communication among providers, and improving care quality. Enhancing management of cancer care ranks among the top challenges.
Patient records today—including existing electronic records—are often transmitted, and typically stored, as free text documents: the physician’s traditional “reports” and “notes”. But our national vision foresees complex data management, aggregation and processing tasks, for which unstructured free text is a poor starting point. For data-aware applications, physician observations, individual test results, diagnostic impressions, and the like must be represented as discrete, computer-processable values. Information must be available in “granular” form.
The challenges and effects of moving to granular data representation—referred to variously as “structured”, “synoptic” (from “synopsis”), or “templated” reporting—is best investigated on the basis of medical document types in which the underlying structure is already fairly predictable: medical reports in which the necessary and optional data content that constitutes the standard-of-reporting-practice have thoroughly been characterized by medical experts. Surprisingly, there are few such report types in medicine. At present, one of the best developed structured reporting domains is cancer diagnostic reporting in Pathology. This development is due to the efforts of the College of American Pathologists (CAP) and American College of Surgeons (ACoS). The efforts to create electronic versions of these checklists have been generously and consistently supported by the Centers for Disease Control (CDC).^2 In Section 1, we review the development of this structured reporting standard, its current implementation status in electronic reporting frameworks, and the “life-cycle” of documents that use these standards.
There is a widely held belief that adoption of records entry and storage systems that support discrete data entry improve provider communication and saves effort and expense. Yet published evidence for these supposed benefits is often difficult to adduce. In Section 2, we review the evidence for beneficial—or possibly detrimental—effects of adopting structured reporting in Pathology.
The computer-processable representation of free text source as discrete values is intimately related to the use of “medical vocabularies” or “controlled terminologies” to encode medical concepts. Any translation of a medical document into a fixed vocabulary necessarily ignores some contextual information and typically leaves some original intent of the physician’s meaning underspecified. The importance of such under specification varies depending upon the intended application. For search and retrieval of archived documents it may be unimportant. For medical decision support or automated quality assurance, it may be of critical importance. In Section 3, we discuss the status of standardized medical terminologies based on the example of Pathology, and comment on the interoperability problems raised by standardized terminology in document-oriented domains such as medicine.
Shared vocabularies are crucial for encoding the conceptual entities to which reports make reference; but equally important is a shared, easily deployed, and stable system of identifiers for the concrete entities to which medical reports refer: the particular patients, specimens, doctors, family members, and so on that are the tangible subjects of medical reports. In Section 4, we discuss shortcomings of existing universal identifier schemes, and suggest improvements and alternatives to address these shortcomings.
Finally, the effective dissemination of structured reports depends upon their availability within a linked network of healthcare providers who share certain quite specific expectations about data quality and reliability. In effect, the supporting network must provide a trust infrastructure that guarantees that structured data will be presented on the network in a fashion that does not adversely affect the semantic integrity of the data. Such trust can only exist on networks that are designed in awareness of the implicit rules governing data integrity that accompany data sharing between individuals behaving in certain roles within the network community. In Section 5, we discuss the computer professional’s emerging notion of trust-based services. We did not however conduct a detailed review of the larger issues related to the privacy and security of medical records in general.
The range of downstream data consumers for a pathologic diagnosis of cancer is a cross-section of the entire healthcare ecosystem, and as such, this use case epitomizes the full range of challenges in interoperable data sharing. As shown in Figure 1, the initial data consumer of the cancer report is the oncologist, whose context is treatment management; and the operating surgeon, whose interest is determination of the adequacy of the resection. Beyond this point, a large number of secondary people become consumers of the pathology diagnostic data. Examples include:
A cancer clinical trials nurse , will use the data to determine whether the patient qualifies for any experimental treatment protocols.
Reimbursement specialists , both in the hospital billing office and an insurer, will use the diagnostic information to determine the appropriate DRG’s for inpatient billing or appropriate outpatient billing.
Quality improvement managers and researchers (with patient agreement to use their information); will find added value in the diagnostic data that may, in part, become linked to a tissue banking or outcomes database.
The diagnosis becomes part of the quality assurance process of the hospital ; and may be used in various ways to contribute information for quality metrics. The report itself may subsequently be subject to quality review for completeness and for correlation with other findings for the same patient.
Since cancer is a reportable disease in most state jurisdictions , the diagnosis becomes a data input for the hospital cancer registry. At the local registry the data undergoes further processing and correlation with other clinical records, and is then forwarded to a state or regional cancer registry, and from there to one or more national public health cancer registries [Wingo 2005].
Ultimately insofar as it is available for research , the diagnosis feeds back into the professional standards setting process , public health reports, outcomes studies and quality assurance.
Because of the importance of the pathologic cancer diagnosis to the patient’s treatment and to myriad other healthcare functions, systematization of the content of the report has been a high priority of pathologists. The College of American Pathologists (CAP), the U.S. medical specialty organization of pathologists, has periodically convened a committee of cancer experts to assess the state of the scientific evidence regarding cancers of various sites and types, and to recommend requisite and optional data items for inclusion by pathologists into their reports [Anonymous 2009a] “Staging” is the method by which pathologists, oncologists, radiologist and public health workers communicate how advanced a patient’s cancer has become, measured according to a standard scheme based on a combination of clinical and pathologic assessment of a variety of parameters. The CAP templates are also closely aligned with standards promulgated by the American Joint Commission on Cancer (AJCC), the North American Association of Central Cancer Registries (NAACCR), the CDC National Program of Cancer Registries (NPCR) and the National Cancer Institute’s Surveillance Epidemiology and End Results (SEER) programs. These recommendations are published in periodically updated reports available to pathologists through the CAP website or as a printed document. These templates are one input to pathologists’ best practices, and a particularly authoritative one. The sister specialty group, the American College of Surgeons (ACoS) has therefore adopted adherence to these templates as a criterion for assessing whether cancer treatment centers shall win accreditation as “Comprehensive Cancer Centers” under the ACoS’s quality assessment program.
The complexity of medical reports for future national health information infrastructure spans a spectrum. At one extreme, clinical laboratory reports (e.g. blood chemistries or cell counts) have a relatively predictable structure—making such tests the “low hanging fruit” of electronic reporting. But this predictability makes these inadequate models for highly complex and variable medical report types with substantial amounts of unstructured, free text content: reports such as progress notes and physical examinations.
Because the data elements appropriate to a cancer report are explicitly defined by the CAP and AJCC standards, these heavily structured, yet quite complex reports have long been seen as particularly suitable “mid-spectrum” model for investigating issues involved in translating complex medical reports into an electronically sharable and searchable formats.
To make this concrete, we exhibit here a fictionalized, but otherwise accurate, specimen of a traditional (pre-CAP template) cancer report (Appendix 1). Note that the report, including the diagnosis, is rendered in a free-text style throughout. The report has a sectional structure, to be sure, but the section headers merely adumbrate the content. The substance of the report is human-readable only; it is not machine-processable, though it may be stored as electronic text.
We next exhibit a pathology cancer report formatted according to CAP template standards (Appendix 2). In this version of the report, data items are individually identified in a “question-and-answer” format, with one information item to a line. Two substantive differences from the narrative version are significant. The underlying content model is more precisely specified: atomic data items such as histologic type, tumor grade and so forth are explicitly specified in the model and a list of possible valid responses are given in the guidance document. Furthermore (not shown here), some items are explicitly stated to be required, while others are stated to be optional.
A limitation of the rendition described immediately above is that it remains a text document. Next, we exhibit a possible electronic extract of such a report, rendered in the XML format proposed by the Pathology Electronic Reporting Taskforce (PERT) of the CAP [Madden 2009] (Appendix 3). In this version, each of the elements is designated by a formally defined XML element associated with the CAP by means of its XML namespace, and [Madden 2009] in a defined syntax available in publicly accessible XML schemas. The XML markup renders the individual data items computer-identifiable, retrievable and processable.
Even this XML version has its limitations. Such a report would normally be stored in a local clinical database. While the retained information is locally meaningful, it is not immediately sharable with another hospital’s database. In the second database, these data items would likely be stored in differently named database tables and columns, peculiar to the other institution’s database logical design. The use of a
Because of its prominence as a standard terminology in the pathology domain, we will frequently refer to SNOMED CT as a terminology, beginning in the next section. For this reason, it may assist the reader to provide some background information about this particular controlled vocabulary.
SNOMED CT [Anonymous 2009b] is a comprehensive, multilingual clinical healthcare terminology. It contains more than 311,000 active concepts with unique meanings and formal logic-based definitions organized into hierarchies. SNOMED CT also works to provide explicit links (cross maps) to health-related classifications and coding schemes such as ICD-9-CM, ICD-O3, as well as the OPCS-4 classification of interventions. Today, SNOMED CT is available in U.S, English, U.K. English, and Spanish. SNOMED CT is one of a suite of designated standards for use in U.S. Federal Government systems for the electronic exchange of clinical health information and is also a required standard in interoperability specifications of the U.S. Healthcare Information Technology Standards Panel (HITSP).
Section 2. Evidence of Value
The literature suggests that control of reporting variability reduces clinical errors. In pathology reporting of cancer, there is net evidence of such value in a synoptic report process.^3 In most of the studies assessing the pathology reporting process, investigators have defined synoptic reporting as the presentation of information in a tabular rather than a descriptive format. The expected benefits of synoptic reporting are the guarantee that all information required for patient management is accurately included. The ease of extraction of data from synoptic reports is considered a secondary advantage.
Standardization of cancer reporting may reduce typographic errors as well as errors of omission. In one of the most referenced articles on the use of protocols to reduce medical practice variations [James 2000], the authors demonstrated improvement in reporting critical elements using a synoptic style. In anatomic pathology, the use of synoptic reporting for cancer cases reduced the amount of missing information with an increase in oncologist satisfaction and facilitation in patient care. The authors concluded that as a result, such reports offered a clearer path of communication between pathologists and other clinicians.
In [Karim 2008], the authors reviewed almost 1700 pathology reports with a diagnosis of primary cutaneous melanoma (PCM) to determine whether synoptic formatting increased the frequency with which pathologic features that influence prognosis and management were documented. They found that synoptic pathology reports were more often complete than non-synoptic reports with the same diagnosis, even in a specialized melanoma center.
[Cross 1998] audited the content of reports at a single hospital over a four year period after four different interventions. Initially reports were free text with no standardized guidelines for reporting. Subsequent audit periods introduced test guidelines, flow diagrams and synoptic reports. While each intervention increased completeness of feature reporting, only synoptic reports brought compliance to 100% for the data items audited.
Of note, the consistent recording of negative findings in both studies was deemed a significant outcome. By requiring an answer, whether negative or positive, the clinician has the added knowledge that a feature has been assessed. It was found that non-synoptic reports did not always document this distinction.
Most authors recognized what the CAP standards also emphasize: that all synoptic reports must include the facility for free text to express degrees of uncertainty, among other reasons. If free text is an inevitable component of even well-designed structured reporting systems, then the problem of accurate textual encoding is unavoidable, and worrisome results regarding coding reproducibility (reviewed in Section 4) are relevant.
No statistically rigorous studies addressed differences of interpretation by clinicians of pathology cancer reports in the narrative versus the synoptic style. [Thompson 2004], in an editorial, stated that clear and free exchange of information was the most important aspect of interaction between pathologist and oncologist. This included a pathology report containing sufficient information to allow evidence-based patient management. The use of synoptic formats in pathology ensured all essential information was reported. However, pathologists must make clear if diagnostic uncertainty is present, and pathologists should refrain from making management recommendations. The use of synoptic report was viewed as a great value to the surgeon because it provided the necessary information to make management decisions and perform accurate staging. If this information was missing, the accuracy of staging and prognostic estimates was compromised.
The literature recognizes fundamental incompatibilities between structured reporting and present-day
Compensating workflow dislocation are the benefits cited above: error reduction, enhanced clinical relevance and inclusion of pertinent information. A particularly error-prone weak link exists at the seam between a paper-based data collection phase and electronic data entry. For example, the lack of a demographic section in the paper CAP checklists makes transfer of this entry of this data into an electronic system vulnerable to a patient mismatch error. To address these issues, [Qu 2007] created a template- based tumor reporting data entry system that was accessed on the web. Data was collected using web forms completed by pathologists, and automatically copied and pasted into text-editor buffers in the laboratory information system. Information in the web browser was purged following entry. The system allowed for free text comments. The authors stated that this system reduced typographic efforts and errors, simplified the reporting process, reduced the error-prone intermediate steps of going from paper copy to electronic format and counteracted the drawbacks of static checklists.
Incompatibility exists between present-day laboratory information software and the structured reporting data model, as noted by [Qu 2007]. Most commercial systems store data in a central (typically relational) database. The data schema for the central database tends to be fixed at the time the system is installed, and systems are not designed for frequent schema changes. Yet structured reports, if persisted in a database, typically require new data tables added to the existing database schema to accommodate the uniquely defined data elements that comprise the structured data model. Furthermore, since structured report standards evolve continually as medical content standards evolve frequent modifications to the underlying representations are required. This rapid evolution of the model is at odds with the design and maintenance routine of most commercial systems. In response to this incompatibility, [Qu 2007] offered a hybrid alternative model: a reporting system that can collect report data and disperse it into specific fields in an LIS. The authors warn that including the information into an existing LIS as a single field will make future retrieval difficult, but that even with this suboptimal solution the additional cost of implementation can be significant.
There is good evidence that structured pathology reporting facilitates reuse of data for biomedical research. Investigators have noted that lack of uniformity in pathology cancer reporting adversely affects research use of cancer diagnostic data. Inconsistencies result from non-uniform selection of reportable elements, diversity of terms used for common items, and variability in individual and institutional document styles. Standardized checklists have been advocated as a means to minimize these inconsistencies and improve granular and interoperable data collection in electronic data systems.
With the advent of molecular and translational medicine and desire to connect research to patient care, [Mohanty 2007] stated that synoptic reporting systems can be used to convey information to researchers in a consistent, concise way. Using the CAP cancer checklists for hematological and lymphoid neoplasms, supplemented by the WHO classification and additional custom data fields, the authors created a digital synoptic reporting module integrated into a commercial LIS. The system used the question-and-answer paradigm and stored results as discrete data fields. Terms were linked to SNOMED CT to support query activity. Validation logic was built into the system to ensure proper answer choices for all required fields. Synoptic reporting either replaced the traditional free text or supplemented free text in the final report. The authors concluded that this form of synoptic reporting resulted in more clear and consistent reports and reduced the need to re-review slides for missing information. By using discrete values, assessment of quality of care studies were expected to improve. Cancer surveillance was also expected to benefit from synoptic reporting by allowing the extraction of common data elements to populate the cancer registry environments.
[Harvey 2005] studied the changes in pathology reporting and histopathological prognostic indicators. During a ten year period, the quality of the pathology reports markedly improved in parallel with the adoption of a synoptic reporting process. The use of synoptic reports increased the effectiveness of data abstraction by a research scientist. Data in this study was extracted from case records by a research scientist and by a medically qualified author, recording whether each report used checklist format. Data reporting metrics improved during this ten year period in proportion to the number of reports using a synoptic format. In 1989, synoptic reporting was not used in the study area, but by 1999, 94.1% of breast
cancer reports were in this format. Overall, the authors attribute the marked improvement in completeness of reporting prognostic factors in breast cancer to: increased requests by clinicians for more information; introduction of synoptic reporting; standardized national approach to reporting and introduction of mammo- graphic screening. The authors found limitations in the ability of non-pathologists to reliably extract data from text reports without pathologist assistance. When synoptic reporting was used, a research scientist was able to perform the task from most reports; in complex cases pathologist assistance was still needed. The authors concluded that pathologist involvement in data extraction remains necessary to reduce the potential for significant errors despite large improvements attributable to synoptic reporting.
[Tobias 2006] reported a case study using CAP cancer checklists within the Cancer Biomedical Informatics Grid (caBIG), an NCI-sponsored data network for cancer research. The authors stated a common data standard that permits interchange among clinical and research systems is urgently needed to advance tissue-based research. Citing the nearly ubiquitous use of the CAP cancer checklist in pathology practice, the researchers created caBIG-compliant data automated tissue annotation for cancer research using the checklists as a basis. The goal was to preserve the meaning of the CAP paper-based checklists in a way that supported semantic interoperability across diverse CaBIG systems. They reviewed CaBIG infrastructure and conceptualization of interoperability, and they carefully distinguished syntactic interoperability (ability to exchange information) from semantic interoperability (ability to understand and use information). Using caBIG methodology, they created UML models and semantic metadata for three CAP cancer checklists (invasive breast carcinoma, invasive prostate carcinoma and cutaneous melanoma), rendering the intent of the CAP cancer checklists as faithfully as possible. They encountered challenges to accurate rendering, including the need for new complex data types and relationships. In the end, three models were developed and form the foundation for future work to develop all CAP checklists as part of one common information model.
[Wingo 2005] reviewed the history of cancer registries in the United States and emphasized the value of interoperable cancer diagnostic data reporting at a granular level for the Public Health. (The relevant organizations are reviewed in Appendix 5.) The authors concluded that future cancer incidence surveillance should be accomplished within an integrated system that maximizes the use of data management information tools to maintain and enhance efficiency and quality and that allows flexibility in responding to new questions. Timeliness could be improved through the adoption of electronic messaging, standardized vocabularies, and interoperable systems.” They recommended improvements for cancer data collection at all phases of disease from pre-cancer to death. Other sources [Qu 2007] concur that a tumor reporting system should be standard and uniform while remaining institution-specific, and that the development of simple and accessible systems will improve cancer surveillance by cancer registries.
No published evidence was found to support the conclusion that structured reporting makes care cheaper. This is in contrast to other domains, for example, in aircraft maintenance^4 , where the end user cost is reduced since the turn-around time for repair and maintenance is faster (therefore, on an hourly billing basis, cheaper). In general, in any service-based industry that depends on skilled knowledge-based work (such as maintenance), if the workflow can be optimized then the end user costs can be reduced.
Social infrastructure : A system of identifiers, to be useful, must have a community of committed users. This community can be small if the identifiers serve a specialized use case. But if the identifiers are intended to serve in a broad variety of use case contexts, then the user community must be correspondingly large.
Self-documenting : Identifiers stand in a relation to the thing(s) they denote. It may be helpful if identifiers are “self-documenting”, in the sense that a defined operation on the identifier can reliably exhibit the identifier’s object or a depiction of the object. This is a property, for example, of web addresses. When entered into a web browser, a web address causes the object of the identifier (a webpage document) to be exhibited on the screen.
Alias ability : Often, the same object may have multiple identifiers. In fact, freedom to coin a name “on-the- fly” for an object without first verifying that no one else has also applied a name to the same object is a practical requirement. But it is also necessary in many cases to be able to “resolve” identifiers, that is, to collect together all the identifiers that refer to a single object and combine them in some sense into a single identifier. Typically, this is done by selecting one identifier of a group of synonymous identifiers to take priority, and linking the others to the priority identifier as aliases.
Persistence : Identifiers, ideally, should never change or go out of existence. In particular, identifiers should not change their object. Two exceptions are of note: 1) Because objects themselves may change (e.g. a single person may become a married person, a living person may die) the definition of persistence itself may be problematic, and 2) In some cases, a single identifier may in the future point to multiple sub-concepts.
Unlimited supply : The supply of identifiers must be (for practical purposes) infinite.
Existing healthcare information frameworks provide virtually no support for core identification of particular things (medical specimens, pieces of medical equipment, parts of a particular individual’s body, geographical locations, individual actions and events, etc.); patchwork support for core identification of concepts (vocabularies); and some support for common core identification of healthcare providers.
The quest for common core identifiers for patients has been rendered difficult by complex issues of privacy and security [Appavu 1997], by resistance to government as the naming authority, and by the failure to identify any alternate naming authority. The publication of the National Provider Identifier (NPI) Final Rule in 2004 [Pickens 2005; Centers for Medicare & Medicaid Services, HHS 2004] resulted in a national system of identifiers for physicians and providers, but not one for patients. The equally critical need for common identifiers for entities other than patients, physicians, and healthcare organizations [Windley 2005] in a national health information architecture has received surprisingly scant coverage in the published literature.
One effort in the direction of a common identifier framework directed expressly at healthcare applications is the Referent Tracking System (RFS) of Ceusters & Smith [Ceusters 2005; Ceusters 2006; Ceusters 2006; Ceusters 2007]. This experimental central repository system provides a framework for tracking identity, defined according to ontological criteria, of entities including physical objects (medical equipment, specimens, etc.) through time. It is thus far more elaborate than simply a scheme for generating and publishing reusable identifiers. Whether practical healthcare contexts might require a system quite this elaborate is yet unknown.
Other than experimental systems such as RFS, what identifier systems are functioning or proposed in the healthcare domain? We consider here sharable patient and provider identification formats, but also systems intended to provide common core identifiers for physical and conceptual objects.
Health Level 7 version 3 (HL7v3) uses a subclass of the ASN.1 [Anonymous 2009c] Object Identifier (OID) type as its preferred identifier for coding schemes and identifier namespaces. HL7v3-compliant systems may also use OIDs for identification of individual information items (objects, concepts). OIDs can be rendered in a web context in a standardized way as URIs [Mealling 2001]. In contrast to other identifier systems in common use in healthcare, OIDs use a system of registries open to users outside the immediate HL7 community, even though this registry system is rudimentary. Therefore OIDs are genuine common core identifiers , in the sense defined above. Health Level 7 version 2 (HL7v2) uses a native identifier format based on two- or three-character alphanumerics. The identifiers are assigned centrally by
HL7, and hence while they are core identifiers , they are not common core identifiers.
Integrating the Healthcare Enterprise (IHE) does not propose its own system of identifiers, but instead relies on identifiers provided by other systems that are selected as part of its application profiles.
Medical vocabulary systems each use their own unique identifier systems and syntax, for which the vocabulary authority is the vocabulary publisher.
Systems of unique identifiers rely upon two alternative mechanisms for avoiding name clashes: namespaces and randomization.
The namespace mechanism associates names with a graph structure that partitions names into non- overlapping families. In the common case that the partitioning structure is a tree, the namespaces are distributed in a hierarchy. This hierarchical structure assures absence of name-clashes. As the tree can be extended infinitely, the supply of names is assured.
There are several namespace-based identifier schemes in existence, including Digital Object Identifiers (DOIs) [International DOI Foundation 2009; Paskin 2005], forms of ASN.1 identifiers including Object Identifiers (OIDs) [Anonymous 2009c] and a variety of others [Anonymus 2007b]. The primary example of this identifier type is the Uniform Resource Identifier (URI) standard [Berners-Lee 2005], a family which includes among several dozen other subschemes, including the familiar web location identifiers of the form “http://www.example.com”. In what follows, several different URI schemes will be mentioned, but details of the various available schemes within the URI family will not be exhaustively treated.^5 Analternate mechanism relies on maximizing randomness of names to guarantee uniqueness. The prototypical example is the UUID scheme [ITU 2004].
The technical landscape of identifiers has been thoroughly reviewed in [Anonymous 2007b]. This group systematically examined candidate identifiers for suitability as a common core identifier. All the standards that satisfy the technical requirements for common core identifiers and that are currently the subject of formal standardization belonged to the URI family. The XRI (Extensible Resource Identifier) scheme, developed in the XRI Technical Committee at OASIS (http://www.oasis-open.org), was the group’s recommendation for the global common core identifier form. Since the report was issued, discussions between OASIS and W3C have resulted in convergence between the XRI model and URI schemes in common use on the World Wide Web, in particular the http: scheme [Barnhill 2008].
Members of the World Wide Web Consortium (W3C) Technical Architecture Group have consistently argued that novel URI schemes (i.e. schemes other than http:) should rarely be used to name information resources on the Web, and that special registries for such identifiers (i.e. registries independent of the existing DNS infrastructure) should probably not be provided [Jacobs 2004; Thompson 2006]. [Sauermann 2008] discusses additional requirements for the use of URIs as semantic identifiers, including content negotiation mechanisms to distinguish requests about an identified resource that seek computer- processable metadata from those that seek human-readable documentation.
[Tonkin 2008] reviewed a number of identifier schemes (including several also reviewed by [Anonymous 2007b]), with particular attention to the issue of managed persistence. She found, as expected, that persistence is an issue for URI-family identifiers, but she also noted several persistent URL family variants in successful use in the library community, including PURL, OpenURL, the Handle System, and the ARK system. Among the persistent identifier systems reviewed, DOI (a non-URI-family specification) was noted to have the widest dissemination in the library community. However, a specification exists for transmission of DOI information as a URI, and resolution of a DOI to a URI [Anonymous 2006b].
The OID is of special interest because it is the standard identifier for HL7 and DICOM. It was not reviewed in The Open Group report. However, since a well-defined mechanism exists for representing OIDs as URIs [Mealling 2001], the choice between these two representations may depend on the quality of the infrastructure available to support these two types of identifiers. Currently, there are fewer than ten registrars for OIDs worldwide. The largest OID repository (http://www.oid-info.com) has roughly 95, entries with new entries being added at an average rate of about 12 per day. By comparison, the http: scheme in the URI system has hundreds of registrars worldwide, which are estimated [Anonymous 2009d] to have currently registered about 109,000,000 domain names, and to be adding domain names at a rate
Section 4: Standard Terminologies
There is wide agreement that one key to interoperable data in the medical domain will be the availability of standardized terminologies for sharing data that facilitate the use of strictly defined data elements over the free text approach. In the current landscape, patient data is stored by the provider, for example in a hospital or physician’s office filing system. In the majority of cases today’s systems rely on paper-based document storage, but the obvious hope is for increasing use of electronic storage.
The U.S. medical terminology landscape today is dominated by a moderate number of large, highly evolved terminologies that are centrally curated. Most terminologies are adapted for some particular business-process context. For example, for outpatient billing, the Current Procedural Terminology (CPT) system, curated by the American Medical Association, dominates, and is required for this use by CMS and insurers. For inpatient billing, the International Classification of Diseases (ICD-9) curated by the World Health Organization is used. Many other examples could be cited.^6
The literature did offer several studies that attempted to reproduce coding results from previously coded clinical concepts. Studies were also reviewed that attempted mappings between terminologies. The quality and integrity of coded medical data was reviewed in studies where the focus was on intra- and inter- observations using SNOMED CT as the coding vocabulary. While study numbers are small, there is evidence that variability in coding may be an issue for data integrity and reusability. More studies are needed that address issues of variation in coding results. We did not find any systematic studies of the intra-rater or inter-rater reliability of semantic annotation. The literature has little information on consistency and reliability of SNOMED CT coding across institutions. Few studies addressed the differences in coding accuracy and reproducibility for patient records versus coding for research use.
[Andrews 2007] compared the consistency of SNOMED CT encoding of clinical research concepts by three professional coding services. A random sample sent for coding consisted of question-and-answer pairs from Case Report forms. Coders were asked to select SNOMED CT concepts (including both pre- coordinated concepts and post-coordinated expressions) that would capture the meaning of each data item. All three agreed on the same core concept 33% of the time; two of three coders selected the same core concept 44% of the time; and, there was no agreement among all three services 23% of the time. The authors sought to determine the cause of this surprising lack of agreement. For example, was code choice inherently underdetermined? If so, coders should have expressed uncertainty about their selections. But this was not the case: each company evaluated their choice as an “exact match” for the vast majority of items they coded. Nor did companies agree about which items were “hard” to code. When asked to rate their own level of certainty for each item, all three reported same certainty level only 25% of the time; two of three companies reported same level of certainty in 55% of cases; and in 20% of cases there was complete disagreement about certainty. Considering the high variation in actual coding, this high level of certainty and the lack of agreement on which items are difficult to code are alarming. The authors conclude there is a need for efforts to make SNOMED CT more user-friendly. Yet the study did not change the authors’ opinion of SNOMED CT as a viable and appropriate data standard for clinical research.
[Chiang 2006] investigated whether a controlled terminology can adequately support EHR systems by measuring coding agreement among three physicians using two SNOMED CT browsers. Both inter- and intra-coder variability was measured using ophthalmology case presentations. Pre-coordinated and post- coordinated concepts were acceptable. The study found that inter-coder agreement was imperfect and unequal and, intra-coder agreement was imperfect. Results obtained from exact code matching were different from those obtained by manual review to determine semantic equivalence. The authors raised concern about the reliability of coded medical information in real-world situations and also for retrospective clinical research. The specific browser used affected the way the reports were encoded, and it was suggested that improved browsing tools may improve reliability.
[Rothschild 2005] studied inter-rater agreement in physician-coded problem lists by having ten physicians review each of five cases. They concluded that inter-rater agreement in non-standardized problem lists is moderate, and that much variability may be attributable to differences in clinicians’ style and the inherent fuzziness of medical diagnosis.
Finally, with regards to validation of the logical consistency of an ontology, [Wang 2008] developed a method for detecting “complex concepts” within SNOMED CT, which map upward to more than one hierarchy. Complex concepts were found to have an error rate (as defined in the paper) of 55%, while a control sample had an error rate of 29%. The study was limited to the Specimen hierarchy and did not examine the remainder of SNOMED CT. Issues raised in this article about the Specimen hierarchy have been addressed by the IHTSDO.
To make free text machine-processable, rather than merely retrievable by loose criteria for strictly human perusal, several strategies are possible.
Full-text indexing of text fields in medical databases is the best-tested strategy [Hanauer 2006; Erinjeri 2008; Erdal 2006; Ding 2007; Wilcox 2003]. Modern text-indexing systems are extremely fast and thorough, and allow for very complex search criteria to be specified. Full text search indexes are not commonly provided in commercial medical records systems today, and advocates argue that they are underutilized. One obvious limitation is that purely lexical indexing fails to take account of the fact that natural language makes extensive use of synonyms and alternative spellings for words with similar or identical meaning. But this limitation can be remedied by supplementing the lexical index by synonym lists, and indeed such synonym lists are among the simplest types of “terminologies”. The addition of “smart” retrieval algorithms to full-text indexing has proven a remarkably scalable and useful strategy for dealing with masses of free text (as for example in the domain of web search engines) [Moskovitch 2007].
A second strategy is to use natural language processing (NLP) techniques to parse free text into logical units based on algorithms that take grammatical, as well as lexical, criteria into account [Jagannathan 2009; Wang 2009; Chen 2006; Friedman 2004]. Some experts consider controlled natural languages as a necessary bridge to support human computer interactions. They point out that natural languages evolved to express and support human ways of thinking while computer languages enable IT professionals to think about the data and operations inside the computer system. Forcing clinical subject matter experts (SMEs) to think about their own subject in computer terms is counterproductive because:
The advantages of NLP are based on their ability to infer relationships among topics not merely based upon proximity of words but on linguistic structure. This sensitivity to structure allows, in theory, more accurate computable depictions of the intended meaning of the text. NLP techniques recognize that meaning consists not merely in juxtaposition of entities, but equally in the syntactic relationships among entities.^8 This strategy has already been implemented in the field of cancer diagnostics. The National Cancer Institute’s caTIES automatically extracts coded information from “free text surgical pathology reports” to support cancer research. However, no accuracy results have been reported. More recently [Coden 2009] has described an updatable knowledge representation model for cancer research (MedTAS/P) that uses open source NLP tools to parse free text pathology reports. Of note this study developed a methodology to validate the system against a set of colon cancer pathology reports. Results were highly satisfactory for histology and lymph nodes, with lower scores for metastatic tumors (possibly due to the limited number of samples in the training and test sets).
A third strategy, epitomized by the CAP structured cancer reporting templates that are the subject of this white paper, is to restrict the freedom of the author by presenting a predetermined form , rather than a blank page, as a slate for content entry. The limitations of free text are overcome by making the text less free. When appropriate, these forms require the pathologist to select from a predetermined set of coded responses. The codes may be derived from or mapped to other logical structures (such as ontologies) to
