Printer-friendly Version
Performance Measurement Case Study:
Search Developmental Center
 
 
Organization Overview

Dedicated to providing quality service alternatives which foster personal growth, self respect and dignity for person with disabilities and their families.
 

Explaining Value to Stakeholders


Search Developmental Center tells its story to funders, peer organizations, family members and guardians of its program participants. The organization communicates its value to stakeholders through an annual report, agency newsletters, and website.

 

Telling our story to our employees, Board and program participants: We communicate with out employee, Board and program participants using a variety of methods which have been, over time, embedded in our operational structure. These include:

 

-Quarterly Management Reports culminating in an Annual Management Report

-Annual Accessibility Improvement Plan and six month Progress Report

-Three Year Strategic Plan and annual Progress report

-Periodic Quality Assurance Survey Reports

-Annual Quality Improvement Plan

 

Operationally, in conjunction with the various reports, several employee and particiant-focused planning events and committee meetings are held throughout the year that serve the purpose of sharing information about the organization’s value with employees, participants and the Board, as well as planning for quality improvements, making management decisions, and long range positioning. Those include:

  • The annual quality improvement planning meeting- at this meeting program participants and employees are given information about the organization’s progress on the previous year’s quality improvement goals, the organization’s demographics, participant and stakeholder satisfaction with the services provided and numerous performance indicators. Employees and particiapants also use the information to formulate quality improvement goals for the upcoming year, as well as the accessibility improvement plan.
  • Consumer council meetings- Peer advisory groups, make up of the individuals receiving Search’s services, meet at all group homes and adult learning centers monthly. The councils serve many purposes, chiefly as opportunities for our participants to exercise choice and control over their services and lifestyles. They are also valuable venues for sharing information about the value of our organization’s services.
  • Board meetings and the annual Board retreat- This is where the Strategic Plan is created and monitored, setting Search’s policy-level direction over a three year period. These meetings include focused conversations, using the AMR/QMR data, regarding Search’s program areas of highest value, as well as those program areas that require improvement.
  • Various organizational committee meetings- Committee meetings, where all levels of staff are represented, are used to carry out the organization’s business operations and communicate information about the organization’s performance and value. Committees also provide useful oversight over the organization’s performance in key areas. Those include: community inclusion, organizational employment, community employment, healthcare and intake (enrollment). Committees include: human rights, behavioral health, quality assurance, vocational, intake, medical services, corporate compliance, employee education, program and support services and the executive services committee. The executive services committee members include key personnel such as the Chief Executive Officer, Chief Operations Officer, and Division Manages fro all programs and is the ‘clearinghouse’ for the work product of all the other committees.

Knowledge of Positive Impact/Achieving Goals


Search currently tracks 54 indicators in 9 categories. Broadly, they measure program performance and program support performance.

Performance results are published and distributed in the Quarterly Management Report, Annual Management Report and are distilled into a variety of other formats throughout the fiscal year. This information is used to develop the Annual Quality Improvement Plan, Three Year Strategic Plan, and the Annual Accessibility Improvement Plan. The information is also used by management and administration to make ongoing decisions regarding service delivery and business operations.

 

How the Organization Developed its Current Process to Measure and Report Effectiveness


 At first, we relied heavily upon the standards published by CARF, the Rehabilitation Accreditation Commission. We are mandated by the State of Illinois to be accredited, and in an effort to meet this requirement we used the CARF standards on outcomes management in designing our own system. It is a process that took several years to become 100% integrated into our ogranization’s operational structure. Interestingly, the difficult part was not establishing the process to measure and report information on our performace. The hard part was what to do with all this information, how did this new capability fit in with our pre-existing management structure. Simply put, we had to decide who was going to get the information and how it was going to be used.

 

In determining our ‘metrics,’ we focused upon our three core services, Supported Living, Adult Learning, Community Employment, and a set of foundational business operations. Again, this was not difficult to do. But it did take some work to accomplish, mainly developing the infrastructure to do what we needed to do to measure our preferred quality indicators.

 

The final process was to determine specifics about the indicators we had chosen. In other words, if we are tracking how often people go out, how often is enough to meet the kind of quality expectation our participants expect. To do this we took into account input from our participants and employees, as well as what would be realistic. You don’t want to choose something you could never realistically accomplish within the time frame of your measurement. You also don’t want to set a target that is so easy it presents no challenge. For the supported living program we came up with a measure of the % of participants having 8 community outings or more as the minimum measure of success. An organization has to be willing to adjust its measures and metrics too. If things are going along and the organization sees that it is not meeting the measure it has set, they need to determine whether the problem is a matter of poor performance or that we started with an unrealistic target. 


Why the Organization Chose to Expend Time and Resources for Metrics

As mentioned above, our initial performance-measurement efforts were driven by CARF, our accrediting body. However, over time we became heavily invested in the process as we saw how the information we were collecting could help us improve the organization’s performance and service delivery. In fact, outcomes are such a big part of our organizational life that even if the State no longer mandated accreditation from CARF, which could be a distinct possibility over the next 1-2 years, we would still seek accreditation. 

As far as the time and money, our organization has made substantial investments in infrastructure to carry out our management information system. The most significant investment has to be our purchase of the Harmony Information System- an electronic records system we use to gather most of our information about participant outcomes. Through Harmony we can track: (1) participant progress toward achieving personal goals, (2) incident reports; (3) medication changes; (4) the number and types of medical services participants receive; (5) client hours worked, wages; (6) the number and kinds of community recreation and leisure activities participants go on; (7) the number of referrals we receive, new enrollments, attendance, etc. Our initial investment in this system was significant, but the value we received and the potential for further enhancements has more than justified this cost. In addition to the system costs, significant portions of staff time, including the majority of Search’s Communications Manager position as well as smaller portions of Search’s entire management team, are spend maintaining, collecting, and using this information.

Lessons Learned


It is not opportunity that creates outcomes; it is what one does with opportunity. In order to communicate the value of our services effectively, we had to define what we meant by the term, value. Sometimes we’ve confused the metrics used to measure outcomes with the value of the services provided.

Facts, figures, charts and graphs may be interesting or even impressive, but numbers and percentages are not a particularly compelling way to communicate an organization’s value. Value is best communicated using compelling stories about real people striving and achieving great things for themselves.

 

They are dramatic accounts of the change in people’s lives. The desire for change, the effort it takes to accomplish change, and the positive impact it can have on a life are all things most of us can relate to.

  •  Our employees crave to see the value in their work
  • Outcomes are best expressed through compelling stories of personal accomplishment.
  • Collecting information about outcomes is not the hard part, sharing the information and using it for improvements is the hard part.
  • For an outcomes management system to work it has to be 100% integrated into the operations of the organization at every level possible.
  • A focus on outcomes makes an organization more transparent and this is a good thing. An organization has to be prepared to take the good with the bad. A “not so positive” outcome can become an opportunity for growth and improvement.

How the Organization’s Metrics can be Generalized


Our community inclusion indicators are used for all three of our core services. While any of our indicators could be used by a similar organization, community inclusion speaks to the highest-level goal (perhaps even to mission) of agencies like Search- to deliver an integrated and inclusive experience to people with disabilities.

In general, this information could be tracked without an expensive data base like Harmony, though it would certainly take more staff time to do so. Before we had Harmony, we tracked community outings and medical appointments manually. With Harmony however, with a little tweaking, we have the flexibility to use similar electronic means to track new indicators. For example, our healthcare and community inclusion indicators use essentially the same system: activities related to medical services or recreation are entered similarly, the only thing different is the coding of the data. Medical codes: annual physical, MRI, CT scan, etc… Recreation codes: horseback riding, picnic, movie, etc… Using the same basic format we could begin tracking, for example, every time a participant travels independently in the community (without staff supervision). The codes might be: CTA Bys, CTA Train, RTA, PACE, etc.

Best Metrics Story


Probably the best value we’ve gotten from having access to performance information has to be the impact it has had on community inclusion. Based on performance information, our organization has put a heavy emphasis on community inclusion, creating a whole new division, budget line items, etc. We created a Community Recreation Coordinator position and two assistant recreation staff positions whose sole focus is to get people out into their communities. We also developed a new program called Search for Adventure, to provide participants with outdoor vacations, which has been very successful. Search for Adventure provides large-scale outdoor adventures and vacation experiences to our program members. Recent adventures have included an eight-person trip to the historic Appalachian Trail, an eight person trip to the Grand Canyon, and a three person trip to the Powderhom Dude Ranch in Colorado.


This case study was pulled from The Center for What Works' fully-animated eLearning CD-Rom, Measure What Matters. It is available as part of The Center's Performance Measurement Toolkit. We invite you to visit our online Store to learn more.


© 2010 The Center for What Works. All rights reserved. Contact Us