Running head: CLINICAL OUTCOME MEASUREMENT

Clinical Outcome Measurement: A Call to Action Timothy A. Kelly Graduate School of Psychology Fuller Theological Seminary

Journal of Psychology and Christianity, 2003, Vol. 22, No. 3, 254-258

Clinical Outcome Measurement: A Call to Action

“America’s mental health service delivery system is in shambles, . . . [and] needs dramatic reform” (p. ii, 1). So proclaimed the Interim Report of the President’s New Freedom Commission on Mental Health (2002). This is a remarkable statement for a Presidential commission to make, and it applies to both the public and private sectors of mental health care. The problem is not that there is a lack of talented and dedicated providers, for there are many skilled clinicians who have a heart for those they serve. And the problem is not primarily a lack of resources (although additional funds could be put to good use). After all, America currently spends over $69 billion on direct treatment costs each year (U.S. DHHS, 1999) – an average of well over $1 billion per state. Even if more is needed to meet demands, nobody argues that current funding is as well spent as it should be.

The problem is that the mental health service delivery system, though it has come a long way, is moribund. Though there are notable exceptions, the system overall is fragmented, tradition-bound, and resistant to reform efforts (Kelly, 2000). The status quo stifles innovation, and as a result “consumers” (clients) of mental health services too often receive ineffective care. In a word, the mental health system is broken and in need of sweeping reforms.

But how is reform to be pursued, even if many agree that it is needed? What is the starting point? Increasing numbers of clinicians, researchers and consumer advocates are turning to the concept of “evidence-based practice” as a key component for improving and reforming mental health services (Barkham et al., 1998, 2001; Evans et al., 2000; Kelly, 1997, 2000, 2002; Nathan & Gorman, 2002; Roth & Fonagy, 1996; Tan 2001, 2002). This means turning to science – notably clinical outcome data – to guide mental health care and policy. Since that which is measured tends to get better, routinely measuring actual clinical outcomes will inevitably improve the quality of care and advance needed mental health reforms. Outcome information helps the practitioner to ensure that treatment is maximally effective, and at the same time provides aggregated data for decision-makers managing program development and support. Most of all it helps those on whose behalf mental health services exist – men, women, and children with serious mental illness. It provides consumers with a significant voice in their own care through self-report surveys, and helps ensure the best outcome to their treatment based on their feedback.

Most practitioners have not yet addressed these matters since state legislatures have not yet required the use of clinical outcome measures. But that is likely to change in the not too distant future as the move towards evidence-based mental health care gathers momentum. For this reason it behooves the forward-looking clinician to get ahead of the curve by voluntarily implementing outcome measures now. By doing so, the clinician is taking a proactive approach to assuring quality of care that consumers and third party payers alike deeply appreciate. The consumer appreciates the fact that the clinician cares enough to track their progress, and tends to see that as an indicator of excellent clinical care. The third party payer appreciates the fact that the clinician has firm data upon which to base treatment recommendations, and is more likely to authorize further care when the request is data-based. Most importantly, outcome data provide the practitioner with an objective read on the client’s real-time clinical status that can inform treatment decisions and maximize positive outcome.

This article is written as a guide for clinicians who would like to consider taking the initiative to implement clinical outcome measures in their current practice. Two measurement options are presented, one that can be purchased for a reasonable fee and one that is available without cost. Then, a measure of spirituality is presented as an add-on to the clinical measures. But first – a little background on outcome measures is called for.

Background

There are a variety of reliable and valid clinical outcome measures on the market and in the literature, and many have been used frequently enough that they have gained widespread name recognition. Among the best known are the Derogatis Symptom Checklist (SCL-90-R; Derogatis, 1977) and the Beck Depression Inventory (BDI; Beck et al., 1961). The SCL-90-R has 90 questions that generate nine primary symptom dimensions such as Obsessive-Compulsive, Depression, Anxiety, and Hostility, as well as a Global Severity Index. The BDI has 21 items that together determine the extent to which the client is suffering from depression. These measures are widely used in both clinical and research settings. However, both have significant drawbacks. The SCL-90R is far too long for many clients – who may end up resenting the time required to finish the survey. Also, it does not cover the client’s level of functioning – an important aspect of improvement. A shorter version is available (Brief Symptom Inventory; Derogatis & Spencer, 1983), but with 53 items it is still too lengthy – and still does not cover functionality. The BDI is shorter, but as the name indicates it measures only one kind of pathology – depression.

In response to these and related concerns, recently-developed outcome measures tend to be more comprehensive than the BDI, yet shorter than the SCL-90-R. They typically are in the range of 30-40 items and cover three areas of clinical improvement that together are fairly comprehensive: symptomatology, functionality and well-being. Symptomatology involves measuring a range of typical symptoms reported by the client at intake, such as sad mood, loss of appetite, elevated anxiety, racing thoughts, ritual handwashing, aggressive outbursts, etc. Functionality involves measuring the extent to which the client is performing well at home, at work, at school, etc. Well-being requires asking the client to report on their subjective sense of wellness.

Psychometrically sound and easy to use, these new measures could potentially transform the status quo mental health system into evidence-based high-quality care if used regularly to inform clinical and programmatic decisions. The result would be that a far larger portion of people with serious mental illness would receive treatment that is demonstrably effective for them. In the long run, consumers receiving such care would be more likely to succeed in their own community with a good job, a real home, and fulfilling relationships. With these goals in mind, let us consider two state-of-the-art outcome measures – one proprietary and one freely available. Either would do an excellent job in providing reliable and valid outcome data for a clinical practice.

Option 1: The Treatment Outcome Package (TOP 37)

The Treatment Outcome Package is available from Behavioral Health Labs (BHL), an organization begun in 1992 by clinicians interested in developing reliable and valid measures that can readily be used to improve clinical outcomes. The TOP 37 is a 37-item instrument for measuring adolescent and adult clinical status that has strong psychometric properties (Kraus & Seligman, 2003). It is designed to be appropriate for anything from solo practice to large networks of providers, and takes only a few minutes for the client to complete. The items primarily cover four areas of concern: depression, anxiety (including panic attacks), suicidality, and violence. An accompanying “Clinician Rating Tool” is provided so that the clinician can rate global assessment of functioning, level of functioning in various settings, well-being, and other areas of concern such as manic symptoms, psychotic symptoms and drug/alcohol abuse.

BHL not only provides the instruments, but also the data management required by their use. Each survey is sent in to BHL by fax or email, and turned around in minutes so that the treating clinician can have current clinical outcome data before the session is over. A nice feature is a tracking graph showing client improvement over time on target areas that are case specific. BHL states that their instruments and database meet HIPAA, JCAHO, and other accreditation and regulatory standards. They claim to have the largest customer base and database of clinical outcome data, with over 250,000 surveys processed to date.

The TOP 37 is a proprietary instrument, and can be used only by contracting with BHL. The actual fee is based on program needs and the extent of data management required. BHL claims that since they offer this service to large numbers of users they are able to keep costs down, and that in fact their fees come to about 1/5 of what it would typically cost to pay for equivalent data management on site.

The best way to explore the TOP 37 and BHL services is via their website: www.Bhealthlabs.com. The site allows for an immediate downloading of the TOP 37, the Clinician Rating Tool, and samples of outcome data results. It also explains pricing policy.

Option 2: Clinical Outcomes in Routine Evaluation (CORE-OM)

The CORE Outcome Measure (CORE-OM) was developed in England by Britain’s Department of Health and has been in use there since 1998 (CORE System Group, 1998). The CORE-OM has 34 items designed to measure common symptoms, subjective well-being, life/social functioning, and risk to self and others. The instrument is designed to generate a “global level of distress” that is calculated as the mean score of all 34 items. This mean score, as well as individual items, can be tracked over the course of therapy as measures of clinical improvement. Extensive field research has demonstrated that the CORE-OM is a reliable and valid measure, useful both for clinical assessment and case management (Barkham et al., 2001).

The CORE-OM is accompanied by two other instruments designed to help with clinical management. The “Assessment Form,” to be used at intake, covers previous treatment, medication, presenting difficulties, social support, and the usual array of demographics. The “End of Therapy Form,” to be used at termination, generates a retrospective treatment profile covering length, type, and frequency of therapy. It also documents the overall effectiveness of care, such as specific therapy benefits and successful termination and transition.

The CORE instruments are protected by copyright, but the copyright owners (CORE IMS Ltd) encourage unlimited, free usage. Their stated goal is to maximize implementation of outcome measurement in order to improve quality of care. Clinicians interested in using the CORE are invited to do so with only one proviso – the instrument content may not be changed in any way. Users are encouraged to purchase the nominally-priced CORE System User Manual, which includes normative data and scoring guidelines. If the user so desires, CORE IMS Ltd is available for additional paid services such as outcome data entry and analysis.

The best way to investigate the CORE-OM and available support services is via their website: www.coreims.co.uk. It is not necessary to purchase any services in order to use this instrument, but larger practices might find that it is cost-effective for CORE IMS Ltd to manage the data.

The Spirituality Factor

The only drawback with these otherwise-excellent clinical outcome measures is that they make no attempt to cover issues of spirituality or faith. Yet the literature is clear that some forms of religiosity and spirituality are positively related to both health and mental health (Hill & Pargament, 2003; Miller & Thoresen, 2003). For those clinicians wishing to track not only symptoms/functioning/well-being but also spirituality, an additional instrument will be required. Many such instruments are available in the

literature (see Hill & Hood, 1999), and at least one has enjoyed fairly widespread usage –

the Spiritual Well-Being Scale (SWB; Ellison, 1983). However, at twenty items the SWB is somewhat lengthy to use as an add-on for outcome assessment.

In contrast, Everett Worthington recently developed a spirituality measure that is easier to use (10 items), psychometrically sound, and conceptually focused – the Religious Commitment Inventory (RCI-10; Worthington et al., 2003). The RCI-10 consists of 10 items designed to measure both intrapersonal commitment (e.g., “My religious beliefs lie behind my whole approach to life.”) and interpersonal commitment (e.g., “I enjoy spending time with others of my religious affiliation”). The intrapersonal items cover what many would see as inner, personal spirituality. The interpersonal items cover actions and social behaviors that would more typically be associated with religiosity. Thus, the RCI-10 provides a psychometrically sound measure of both religiosity and spirituality as commonly defined.

Worthington states that a highly religious person evaluates the world through “religious schemas,” and thus integrates issues of religion and spirituality into all of life (including counseling). He defines a highly religious client as one who is greater than one standard deviation above the mean on a normed measure such as the RCI-10. It follows that clinicians working with religious clients could benefit by measuring religious commitment in order to better understand and track their client’s spirituality. The RCI-10 could be used to determine to what extent a client is religiously committed, and thus to what extent issues of spirituality may or may not be relevant to therapy. It could also measure to what extent, if any, the client’s spirituality may shift over the course of therapy. For some clients, a more moderate form of spirituality may constitute one component of a positive clinical outcome. For others, a new appreciation of healthy spirituality might be entailed. Either way, the RCI-10 would provide helpful information for the clinician wanting to maximize effectiveness with spiritually-oriented clients.

The RCI-10, including normative data, is presented in the January 2003 Journal of Counseling Psychology (Vol. 50, No. 1, pp. 84-96). It would not be difficult to generate copies of the RCI-10 and use them as an add-on to the TOP-37 or CORE-OM for use with faith-oriented clients.

Conclusion

Some therapists may feel that outcome measurement is theoretically easy to endorse, but realistically hard to implement. Of course, there is some truth to that. Yet this author has found that implementation need not be the nightmare many might fear. The key is to use measures that are neither burdensome for the client who fills them out nor the clinician who manages the results. Either the TOP 37 or the CORE-OM can be completed in about 5-7 minutes, and the RCI-10 adds only another few minutes to that. Thus an outcome-oriented clinical practice is simply requesting that their clients dedicate no more than 10 minutes before session to a survey that is clearly for their benefit. This is typically done at intake, at regular intervals set by length of therapy (e.g., every 3-4 sessions), and at termination. The results are either computed by the instrument provider for a fee, or calculated by hand. Those data are then available for fine-tuning the course of therapy, as well as for making the case for authorizing further care as needed to third party payers.

This author found, contrary to expectations, that clients at a standard outpatient private practice responded positively when asked to use outcome measures. They approved of the procedure, and seemed to value therapy all the more for the effort being put forward on their behalf. Even more importantly, we found that in many instances the resultant data were extremely useful for case management. The reason was that clients often revealed issues on paper that either did not come up in therapy, or that they were hesitant to verbalize face-to-face. Thus the feedback loop of outcome data provided a valuable supplement to the sessions themselves, and tended to improve the resultant quality of care. For instance, one client neglected in session to mention a recent crying spell (perhaps not wanting to disappoint the therapist), but dutifully included it on her survey. Consequently the therapist was able to address an important matter that would have otherwise been missed. Additionally, insurers seemed much more agreeable to authorizing further care when the request was accompanied by standardized outcome data.

At a time of decreasing reimbursement and increasing paperwork for clinical care, it is not an easy thing to add outcome measures to an already-burdensome workload. Certainly not all clinicians are able and/or willing to do so. But for those who are, the potential benefits are great. Outcome data provide a scientific and detailed answer to a critical question: How effective is treatment? As already noted, a clinician’s willingness to address this key issue means a great deal both to the consumer and to the third-party payer. Perhaps even more importantly, it means a great deal to the clinician in pursuit of excellence. For outcome data enables clinicians to do their best on behalf of those who come in great need.

America’s mental health service delivery system may be in a shambles, but there is a way out through reforms driven by evidence-based care. By proactively embracing outcome measures, forward-looking clinicians can ensure that their clients indeed receive the effective care that they need and deserve. And effective care, simply put, enables consumers to succeed – to live quality lives in their own community with good jobs, real homes, and fulfilling relationships.

References

Barkham, M., Evans, C., Margison, F., McGrath, G., Mellor-Clark, J., Milne, D., et al. (1998). The rationale for developing and implementing core batteries in service settings and psychotherapy outcome research. Journal of Mental Health, 7, 35-47.

Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., et al. (2001). Service profile and benchmarking using the CORE-OM: Toward practice-based evidence in the psychological therapies. Journal of Consulting and Clinical Psychology, 69, 184-196.

Beck, A. T., Ward, C. H., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561-571.

CORE System Group. (1998). CORE System (information management) handbook. Leeds, UK: Author.

Degogatis, L. R. (1977). SCL-90-R: Administration, scoring and procedures manual I. Baltimore: Clinical Psychometric Research.

Derogatis, L. R., & Spencer, P. M. (1983). The Brief Symptom Inventory: Administration, scoring and procedures manual -I. Baltimore, MD: Clinical Psychometric Research.

Ellison, C. W. (1983). Spiritual well-being: Conceptualization and measurement. Journal of Psychology and Theology, 11, 330-340.

Evans, C., Mellor-Clark, J., Margison, F., Barkham, M., McGrath, G., Connell, J., et al. (2000). Clinical outcomes in routine evaluation: The CORE-OM. Journal of Mental Health, 9, 247-255.

Hill, P., & Pargament, K. (2003). Advances in the conceptualization and measurement of religion and spirituality. American Psychologist, 58, 64-74.

Kelly, T. A. (1997). A wake up call: The experience of a mental health commissioner in times of change. Professional Psychology: Research and Practice, 28, 317-322.

Kelly, T. A. (2000, January 7). Principled mental health system reform. The Heritage Foundation Backgrounder, 1341, 1-12.

Kelly, T. A. (2002, March 7). A policymaker’s guide to mental illness. The Heritage Foundation Backgrounder, 1522, 1-16.

Kraus, D., & Seligman, D. (2003). Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package. Manuscript submitted for publication.

Miller, W., & Thoresen, C. (2003). Spirituality, religion, and health. American Psychologist, 58, 24-35.

Nathan, P. E., & Gorman, J. M. (Eds.). (2002). A guide to treatments that work (2^nd ed.). New York: Oxford University Press.

President’s New Freedom Commission on Mental Health (2002). Interim report to the President. An interim report to the President as required by Executive Order 13263 of April 29, 2002.

Roth, A., & Fonagy, P. (1996). What works for whom? A critical review of psychotherapy research. New York: Guilford.

Tan, S.-Y. (2001). Empirically supported treatments. Journal of Psychology and Christianity, 20, 296-301.

Tan, S.-Y. (2002). Empirically informed principles of treatment selection: Beyond empirically supported treatments. Journal of Psychology and Christianity, 21, 54

56.

U.S. Department of Health and Human Services. (1999). Mental health: A report of the Surgeon General. Pittsburgh, PA: Superintendent of Documents.

Worthington, E. L., Wade, N. G., Hight, T. L., Ripley, J. S., McCullough, M. E., Berry, J. W., Schmitt, M. M., Berry, J. T., Bursley, K. H., & O’Connor, L. (2003). The religious commitment inventory – 10: Development, refinement, and validation of a brief scale for research and counseling. Journal of Counseling Psychology, 50, 84-96.

Please address all correspondence to Timothy A. Kelly, PhD, Graduate School of Psychology, Fuller Theological Seminary, 180 N. Oakland Avenue, Pasadena, CA 91101.