Measure Your Centre’s Impact

NBS

| October 21, 2018

Logic models let organizations — including B-school sustainability centres — understand whether they are meeting goals.

Every business school sustainability centre has admirable goals: Develop student skills. Affect business practice. Create strong research.

How do centres know if they are reaching their goals? At the 2018 Sustainability Centres Workshop, centre leaders explored how to assess their progress and impact by using a logic model, also known as a theory of change. Logic models detail the steps toward a goal. Nick Greer, Vice President for Interconnection at non-profit Thread, led the workshop session on logic models.

Centres struggle to measure impact

Sustainability centre leaders shared their challenges around measuring impact:

Keeping metrics aligned with changing organizational goals. Metrics are most valuable when analyzed over time, but organizational shifts in priorities and direction can make a metric less relevant.
Quantifying the unquantifiable. Sustainability centres may work towards long-term change by changing mindsets, but capturing that impact numerically is difficult. Often, universities value immediate outputs — e.g., number of publications — rather than ultimate transformation.
Having impacts resonate with different stakeholders. Sustainability centres address many stakeholders: faculty, students, advisory board, Dean. What impact metrics will speak to all of them? How do data become a compelling story? (And, what if stakeholders aren’t invested in sustainability efforts?)

Many organizations face these challenges, said Greer. “We hear this a lot — how do we tell the story of our data, how do we convince others that what we’re doing is really important.”

Thread’s logic model provides an example

Greer shared his organization’s journey to measuring impacts. Thread works with disadvantaged students, connecting them with dedicated volunteers and resources. The organization seeks to build relationships that support student success.

Thread uses a logic model to understand and measure its impact. Logic models map the resources and activities an organization needs to reach desired goals. A generic logic model looks like this:

For Thread, the desired outcome — the intended impact — is student success, which they define as students’ meeting their own goals. Thread assesses student achievement in part through a quiz that produces an “Are You Okay?” score — e.g. “Do you have a plan for your life? Can you enact that plan?”

Outputs, a more short-term measure, lead to the ultimate outcome. Thread’s output metric is TouchPoints, or meaningful interactions between people in the program.

Greer encourages organizations to keep their metrics simple. “Often, having a single metric is the best way to really see impact,” he said. That metric can then be analyzed in different ways, to address the interests of different stakeholders.

Thread inputs include volunteers and the students at the beginning of the process. Activities include ice-skating or community engagement; these result in TouchPoints between students and volunteers.

Greer used a cooking metaphor to clarify how the logic model elements relate. In cooking, inputs are the raw ingredients and activities are the steps in cooking a meal. Outputs are prepared dishes and desired outcomes are satisfied guests.

Developing Thread’s model took time. Thread’s founders always viewed relationships as important to student success. As the organization collected data on model implementation, identifying the metric driving outcomes took time. Gradually, Thread staff began to see a correlation between TouchPoints occurring and outcomes achieved.

Tools can help you build a logic model

Greer offered a series of tools to help people with different expertise in logic models.

For beginners, “Logic Models: WTF?!?” Use this worksheet to scope your organization’s purpose.
For those with some experience, “Is This Right?” Use this worksheet to build out your logic model.
For advanced logic modelers, “I’ve Got This…” Use this worksheet to assess your existing logic model.

All tools are available in this package.

Outcomes are the most important element

Greer urges organizations to begin logic model development by focusing on the outcome. What is your organization trying to achieve? What is at the core of what you do? “If you don’t have a mission that is really well understood, you’ve got to start there,” he explained. “You need to begin with a really grounded understanding of who you are.”

The outcome identified should be so fundamental to your organization that it is unlikely to change even through a re-organization or re-prioritization. “If you know what’s needed in the world, you should be heading toward that at all costs,” said Greer.

A workshop participant noted that for sustainability centres, impacts can be indirect: “We are enabling people to go out and be change agents.” Does that approach require a nested logic model, so that the centre’s initial impact might be on a student, who then affects practice? Thread is wrestling with similar questions, Greer said: “I don’t know the answer yet.”