{"id":2741,"date":"2026-08-26T09:03:59","date_gmt":"2026-08-26T08:03:59","guid":{"rendered":"https:\/\/www.newforesight.com\/?p=2741"},"modified":"2026-08-26T09:05:03","modified_gmt":"2026-08-26T08:05:03","slug":"why-impact-measurement-systems-need-to-be-built-for-decisions-not-just-reporting","status":"publish","type":"post","link":"https:\/\/www.newforesight.com\/nl\/2026\/08\/26\/why-impact-measurement-systems-need-to-be-built-for-decisions-not-just-reporting\/","title":{"rendered":"Why impact measurement systems need to be built for decisions, not just reporting"},"content":{"rendered":"
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t


Author: Tendai Jonhasi, MEL Lead, <\/span>NewForesight<\/span>.<\/span><\/em>
E-mail: tendai.jonhasi@newforesight.com<\/em><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

|<\/strong><\/span> What is the problem?<\/span><\/span>\u00a0<\/span><\/strong><\/em><\/h3>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Most organisations can describe what they delivered. Far fewer can say what it changed.<\/span><\/b><\/p>

Across skills programmes, enterprise support, community investment and corporate sustainability spend alike, the same pattern shows up: attendance figures, completion rates, number of suppliers onboarded, number of beneficiaries reached. These numbers are accurate, and they are collected for good reasons: governance, compliance, donor reporting all require them.<\/span>\u00a0<\/span><\/p>

The problem is not that this data is wrong. The problem is that it stops one question short of the one that actually matters: did this change anything for the people or the system it was meant to serve?<\/span>\u00a0<\/span><\/p>

A leadership team reviewing a portfolio of ten, fifty, or a hundred programmes eventually asks a different set of questions than the ones the reporting system was built to answer. Which intervention is producing the greatest change? Which one should be expanded next year? Which one needs a redesign? Which one should be defunded? At that point, many organisations discover they have extensive information about activity and almost no evidence about outcome. They can describe what happened but they cannot confidently decide what to do next.<\/span>\u00a0<\/span><\/p>

The instinct, when this happens, is to collect more. Most organisations respond to exactly this discomfort by adding indicators, extending surveys, reaching more people. The decision, however, does not become any clearer for it.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

|<\/strong><\/span> Why does more data not solve this?<\/span><\/span>\u00a0<\/span><\/strong><\/em><\/h3>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Because coverage and credibility are different problems, and most measurement systems are built to solve the first while quietly assuming they’ve solved the second.<\/span>\u00a0<\/span><\/p>

A system optimised to prove reach (how many people were trained or how many businesses were supported) is a different system than one built to test whether an outcome occurred and whether the programme contributed to it. Trying to make one system do both jobs usually means it does neither well. Output data gets collected at scale, but scale is exactly why quality is hard to hold. Outcome data, when it’s collected at all, tends to arrive late, inconsistently defined, and disconnected from the decision it was meant to inform.<\/span>\u00a0<\/span><\/p>

The result is a measurement system that can produce a report but not an answer.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

|<\/strong><\/span> What should organisations be asking instead?<\/span><\/span>\u00a0<\/span><\/strong><\/em><\/h3>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

If a measurement system starts from the report, the first question is: <\/span>what indicators are we required to provide?<\/span><\/i>\u00a0<\/span><\/p>

If it starts from the decision, the first questions are different: <\/span>what are we trying to decide? What do we need to learn to decide it well? What evidence would give us enough confidence to act?<\/span><\/i>\u00a0<\/span><\/p>

Those are not the same exercise, and they don’t produce the same system. A report-led system adds indicators. A decision-led system asks, of every indicator already in place, whether it is actually informing a choice someone is going to make – and if it isn’t, whether it’s worth the cost of collecting it at all.<\/span>\u00a0<\/span><\/p>

Three shifts follow from that reframing, and they hold regardless of sector, portfolio or programme.<\/span><\/p>

Scale should follow the decision, not precede it.<\/span><\/b> Surveying every participant feels thorough, and for compliance purposes it may be necessary. But for understanding outcomes, a large sample is not automatically a more credible one, it’s often a harder one to control for quality. A smaller, deliberately representative sample, built to answer a specific question at the level a decision will actually be made, tends to produce evidence leadership can act on with more confidence than a dataset that is merely large.<\/span>\u00a0<\/span><\/p>

A questionnaire is not the measurement system.<\/span><\/b> The same question, asked by different people in different ways, produces answers that look comparable and aren’t. “Did the participant find work?” means something different depending on whether one day of casual work counts, whether follow-up happened at the same interval for everyone, and what happened to the people who couldn’t be reached. Turning a set of questions into a system that produces trustworthy, comparable evidence requires consistent definitions, trained data collectors, fieldwork-stage quality checks, agreed rules for handling missing data, and clear ownership at every stage from collection to sign-off. Without those governance controls in place, two programmes can report the same indicator while having measured entirely different things.<\/span>\u00a0<\/span><\/p>

Knowing what changed is not the same as knowing what worked.<\/span><\/b> An organisation can build a technically sound impact measurement system and still make an unjustified leap from correlation to causation. Incomes rise, employment improves, businesses grow; the programme may have contributed, or prices may have moved, conditions may have shifted, or people who took up the most support may have been more likely to succeed anyway, regardless of the programme. The way through this is not to stop at “what changed,” but to measure exposure alongside outcome: who received what support, how often, for how long, and in what combination so the organisation can see which pathways are associated with stronger results, and build toward evidence that can eventually test the causal question properly.<\/span>\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

|<\/strong><\/span> What does this mean in practice?<\/span><\/span>\u00a0<\/span><\/strong><\/em><\/h3>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Start with the decision, not the indicator list.<\/span><\/b> Before adding another metric, commissioning another survey, or building another dashboard, an organisation should be able to name the specific decision the evidence is meant to inform. If it can’t, the evidence has no clear destination.<\/span>\u00a0<\/span><\/p>

Review what already exists before building something new.<\/span><\/b> Most organisations aren’t starting from zero; there’s usually a strategy, a theory of change, existing indicators and reporting processes already in place. Some of what’s there will hold up under a decision-first lens. Some will need to change. Some will need to be dropped, because it consumes time and budget without informing anything.<\/span>\u00a0<\/span><\/p>

Assign ownership across the full chain, not just collection.<\/span><\/b> Who collects each piece of evidence, how often, who checks its quality, who receives the findings, and – critically – who is expected to act on them. A measurement system without a named decision-maker at the end of it will keep producing reports nobody uses to change course.<\/span>\u00a0<\/span><\/p>

Pilot before scaling.<\/span><\/b> Testing a redesigned system on one visible programme first shows whether the questions are understood, whether the data can realistically be collected, and whether the people meant to use the results actually find them useful – before the same design is rolled out across an entire portfolio.<\/span>\u00a0<\/span><\/p>

Treat the dashboard as a means, not an end.<\/span><\/b> You’ll still want the analysis, evaluation, and reporting tools; that is not going away, but the objective should also be a conversation in which you can genuinely use evidence to compare investments, redesign a programme, or decide where the next unit of spend should go. And whether you have an internal team capable of running and improving that system without permanent reliance on outside support.<\/span>\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

|<\/span><\/strong> The choice most organisations<\/span> are faced with<\/span><\/span>\u00a0<\/span><\/strong><\/em><\/h3>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Every organisation running programmes at scale is already sitting on activity data. The question is whether that data is connected to a decision anyone is actually trying to make. A farmer who adopted every regenerative practice on the checklist and still hasn’t seen the soil health or yield resilience the programme promised isn’t evidence the transition failed. It’s evidence nobody measured past the checklist..<\/span>\u00a0<\/span><\/p>

Six months of redesigned measurement will not produce definitive proof of long-term impact. It is, however, possible in that time to change the quality of the decisions an organisation makes with the evidence it already has the means to collect, if that evidence is built around the decision from the start, rather than bolted onto a system designed for a different purpose.<\/span>\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Disclaimer: This article reflects general observations on measurement system design and does not constitute programme-specific advice. The right approach for any organisation will depend on its portfolio, its data <\/span>infrastructure<\/span> and the decisions it is trying to make.<\/span><\/span>\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"

Author: Tendai Jonhasi, MEL Lead, NewForesight.E-mail: tendai.jonhasi@newforesight.com | What is the problem?\u00a0 Most organisations can describe what they delivered. Far fewer can say what it changed. Across skills programmes, enterprise support, community investment and corporate sustainability spend alike, the same pattern shows up: attendance figures, completion rates, number of suppliers onboarded, number of beneficiaries reached. […]<\/p>\n","protected":false},"author":10,"featured_media":1068,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2741","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/posts\/2741","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/comments?post=2741"}],"version-history":[{"count":8,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/posts\/2741\/revisions"}],"predecessor-version":[{"id":2750,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/posts\/2741\/revisions\/2750"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/media\/1068"}],"wp:attachment":[{"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/media?parent=2741"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/categories?post=2741"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newforesight.com\/nl\/wp-json\/wp\/v2\/tags?post=2741"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}