Retrofit Case Studies: Caution

Retrofit case studies are stories of real projects, with real homes, real measures and real results, and they can be genuinely useful for understanding what a measure looks like, how disruption was handled and what a finished project feels like to live in. They become misleading when they are read as promises, because a case study is selected, usually because it succeeded, and it describes a specific home, a specific household, a specific climate, a specific budget and a specific set of installers, none of which will match another home exactly. A case study that shows a large saving, a dramatic comfort improvement or a smooth project should therefore be read as an example of what is possible in those conditions, not as a forecast for a different home in different conditions.

Selection bias is the central caution. Case studies published by product makers, installers or funding schemes are chosen because they show the product, the installer or the scheme in a good light, and projects that failed, overran, disappointed or were abandoned are rarely published as case studies, even though they may be more instructive about risks.

Baseline, weather and household context are the details that decide whether a case study result transfers.

Costs and timescales in case studies deserve particular scrutiny. A case study that reports a measure cost without enabling costs, such as scaffolding, making good, temporary heating or professional fees, understates the project cost, while a case study that reports a project duration without surveys, consents, lead times and inspection gates understates the project time. Both omissions flatter the project and set a new owner up for disappointment when the full cost and time emerge during their own project.

Before relying on a case study, list what it claims, and check, for each claim, what baseline it used, what weather period it covered, what household changes occurred, what full costs were included, what measurement method was used and whether the project was independently verified or self-reported by the party selling the measure. Note the home type, construction, climate and installer, and ask how closely those match your own home, since a case study for a modern, detached, south-facing home in a mild climate transfers poorly to an old, terraced, north-facing home in a cold climate, however impressive its headline figure.

Also check whether the case study reports problems, defects or lessons learned, since a case study that reports only successes, with no problems at all, is less trustworthy than one that honestly reports a defect, a delay or a cost overrun and explains how it was handled. Problems, honestly reported, are evidence of a real project, while a flawless story is often evidence of selective reporting.

In the EU, retrofit case studies follow national scheme and research reporting practices, with member states differing on whether case studies are independently evaluated, on measurement methods and on how costs and savings are reported for supported schemes. A case study from one country, with its own measurement and reporting rules, should not be compared directly with a case study from another country without understanding those method differences.

In the UK, retrofit case studies from research bodies, from PAS 2035 evaluations and from funding schemes follow different reporting standards, with BRE and other research bodies providing more rigorous evaluation than marketing case studies from product makers. Those sources should be weighted accordingly, with independently evaluated cases given more weight than self-reported marketing cases, however polished the latter may look.

A frequent mistake is choosing a measure, a contractor or a budget based on a single case study headline, without checking its baseline, costs or context, then being disappointed when a different home, household and climate produce a different result. Another is treating a case study photograph of a finished room as proof of hidden quality, when insulation continuity, ducting, commissioning and moisture detailing, the parts that actually determine performance, are invisible in a finished photograph.

A third mistake is ignoring negative case studies, or the absence of negative cases, and concluding that a measure has no risks, when every retrofit measure has failure modes that a balanced evidence base would show. Risk awareness, from a range of cases including difficult ones, leads to better planning than optimism from a single success story.

Keep the case studies you rely on, with notes on their baseline, costs, context and verification, alongside your own home assessment and quotes, so a later decision can be audited against the evidence actually used, rather than against a remembered headline. Those notes turn a case study from a sales story into a piece of evidence, with known limits.

Ask a case study author, installer or scheme: what baseline, weather and household context apply, what full costs are included, how were results measured and verified, and what problems or lessons were learned. Evidence-based answers show a case study that can be trusted, within its limits, while vague headline answers show one that cannot.