Data Drives Performance: The Performance Improvement Calculator

One of the focus areas of this year’s CoC competition is a movement from analyzing the performance of individual programs to looking at the entire crisis response system in a community. Moving from siloed programs to a comprehensive system includes the work of affordable and supportive housing providers, supportive services, rapid rehousing programs, and mainstream programs for housing, health, and employment. To move the needle on the 2015 goals and to establish a clear set of strategies for our 2020 goals for families, youth, and children, communities are asked to analyze their current performance measures and holistically plan for the years ahead.

A tool for communities to access for this analysis and systems planning was released not long ago, the Performance Improvement Calculator (pictured below). Created by Focus Strategies and the National Alliance to End Homelessness, the Performance Improvement Calculator makes it possible to use data to see where there is room for improvement in a community's homeless crisis response system. USICH spoke with Katharine Gale and Megan Kurteff Schatz of Focus Strategies about the tool, its alignment with HEARTH Act performance measures, and the value of this tool for communities at this critical time during the annual CoC applications.

How does the HEARTH Act connect with the Performance Improvement Calculator?

Focus Strategies: There are two key pieces that really foreground the HEARTH Act:

  1. A recognition of local community programs working as a system, and focusing on how all programs work together to house people and to keep them housed
  2. An inclusion of selection criteria and performance measurement in what grantees are asked to report back on. From Focus Strategies’ perspective, the HEARTH Act hit the nail on the head in terms of what performance measures they are requiring communities to capture. 

The HEARTH Act puts three things into sharp focus for measurement: the rate of exit to housing, how quickly a program helped that individual or family into housing (average length of stay in shelter), and if the intervention sticks (rate of returns to homelessness). Focus Strategies was started a year and a half ago to work with communities to use the data they are already collecting through Homeless Management Information System(HMIS) and their Annual Performance Reports and to use them in a different way. Taking these data points (like average length of stay) and using it along with budget data (cost per intervention) allows communities to model which interventions are most cost-effective. How many nights individuals and families remain in homeless programs is directly related to the cost of serving those households (each night in a homeless program has an associated cost) and to the number of people that can be served with a given number of beds. Integrating these two items is critical in benchmarking and model efforts of communities.

It’s helpful to think about all the different types of interventions across a community’s homeless services system as “levers” that all work in particular ways to serve those experiencing homelessness.This tool allows a community to see the levers they have within their system to create change – taking the data they have on those measures already and how a change in an outcome or a reallocation of resources can change how many people their system can serve and the cost effectiveness of their work.

How does the tool work?

Focus Strategies: Using this tool requires calculations using 5 different data points, which also coincide with some of what HEARTH requires as well:

  • How fast people exit the system – average length of stay in emergency shelter
  • How many people are exiting to particular housing interventions
  • The rate of return to homelessness
  • Capacity of the system (how many beds you have)
  • How much it costs per type of intervention and the total amount currently spent for that intervention system-wide (which can be found in budget documents for the Continuum of Care)*

* If your community does not have data for each of these data points (which is essential to help determine the cost-effectiveness of each intervention per successful exit to housing) the Performance Improvement Calculator can help estimate it.

After you enter data for each of these data points, you have a full report on the outcomes you’re currently achieving. For example, you will be able to see the average length of stay calculated for each type of emergency or transitional bed and how much that intervention costs. This is the first step for a community: modeling the outcomes you’re currently achieving with the resources and inputs you currently have.

The next step is to model what would happen if you improved program outcomes or reallocated resources to help improve the outcomes across one of the four main data points. Users can now enter shifts in outcomes by creating new targets for a specific outcome, and then see how this affects the entire system.

Let’s say your community is exiting 17 percent of the people who enter the system to permanent housing per year. If you change how some of your shelters operate or programs improve, you estimate that you can exit 30 percent to permanent housing. Putting this new outcome in the system, the tool calculates for you how it changes your cost per person in the system. If you work your way through the tabs, you can see the difference changing these outcomes make to the system overall – a change like this (from 17 percent to 30percent of exits to permanent housing) greatly reduced the cost per permanent housing (because there are more exits to permanent housing) and the cost/service also goes down because you are serving more people with the same amount of resources. With no new money, you got better outcomes and your cost effectiveness went up.

You can change each of the four data points (how many beds per intervention type, length of stay, exits to permanent housing, returns to homelessness) separately as well.You can see the effects of what happens if you don’t necessarily change the amount of exits to permanent housing, but you speed up the time it takes for people to get permanent housing.

One thing being talked about in this year’s NOFA application season is the reallocation away from transitional housing and into other interventions. Is there a way to model the effects of this in the tool?

Focus Strategies: Yes. If your community does not have exact data on exits to permanent housing from transitional housing and rapid rehousing, the Performance Improvement Calculator has estimations of typical outcomes that we are confident in. We found that generally the exit rate to permanent housing for transitional housing is 55 percent and the exit rate for rapid rehousing is 85 percent.

Let’s say if you have those rates, a 55 percent rate of exit from transitional housing to 85 percent from rapid rehousing programs, while reallocating $1M out of transitional housing (effectively doing a better job with fewer people in transitional housing), the summary will show you the rate of performance.

This tool can show people that reinvestment and reallocation of resources does not necessarily mean devastation of another intervention – in this example and in others, you can change how you reallocate things and get a net increase for the entire system.It’s not just about changing investments though, it’s also about modifying currentpractices and program administrative rules (if needed) so that it is possible to achieve more placements in permanent housing and shorter stay.

How do you envision community stakeholders or CoCs using this tool?

Focus Strategies: This is a modeling tool, and a simplified version of what happens in the real world. A community will have to look at their existing infrastructure to determine if they can reallocate resources in particular ways or if they have the budget flexibility for some items. But we believe this tool is valuable because it is able to strike a balance between what is digestible and what is relevant to communities in their particular context. The tool can be used to focus on each outcome separately or together. You can use it by yourself, report to a community, or project it to use the tool in meetings together.

A key element of this tool, and using data to help make decisions, is that it helps communities and stakeholders get away from anecdotal evidence on how a system works. While this tool is not perfect, it helps to move the conversation towards what resources communities have that can be used strategically to improve the outcomes that HEARTH emphasizes. Using a finite system of resources (e.g. shelter beds, PSH units), money, and people experiencing homelessness in a community, you can focus one or more of the “levers” available to change. At minimum, this work helps communities in starting to identify and then track key pieces of data from HMIS and their budgets. It’s our goal, though, that once a community has been able to work with their current data, model where they’re going, and agreed on program changes, they can continue to use this as a benchmarking tool.

Go to the Performance Improvement Calculator