Measuring a community: start with the management decision, then outcomes, access, quality and attribution

Community measurement should show whether eligible members complete the intended activity, what it takes to deliver and which decision follows. It should not treat every visible action as evidence of impact. Posts, visits and reactions may reflect value, confusion, conflict or a campaign that temporarily altered the audience.

This guide is for organisations measuring an English community in 2027. It distinguishes England-specific evidence from UK or wider material and avoids claiming a universal industry benchmark. Evaluation sources were reviewed on 5 September 2026.

What to take away

Begin with the management question

Choose the decision before selecting metrics. Typical questions include:

One report can support several decisions, but each measure needs a clear user. A chart that nobody can act on adds maintenance and analytical flexibility without improving the service.

Write the current explanation and a plausible alternative. If repeated participation rises, the change may reflect better value, a prize, easier notifications or the arrival of a highly active cohort. Decide what evidence could distinguish them.

Define the member outcome

State what should be different for whom. A customer-support group may aim for an appropriate answer that lets a customer proceed. A professional network may support a relevant introduction followed by a useful exchange. A learning community may focus on applying a skill, not attending a session.

Trace the outcome backwards to observable steps. For peer support:

Measure failure at each stage. A strong resolution rate among posted questions can coexist with severe barriers before posting.

Government guidance on measuring service success applies to public services. Its recommendation to combine performance measures with user research and consider end-to-end journeys is useful beyond that setting, but a commercial community must choose indicators for its own purpose.

Create a metric dictionary

For every figure, record:

Add these fields too:

The definition of "active member" should say whether viewing, logging in, contributing or attending counts. The definition of "resolved" should explain who decides and what evidence they see.

Keep rates with underlying counts. A perfect result from two journeys carries different uncertainty from a slightly lower rate across hundreds. Avoid decimals unsupported by the data quality.

Version definitions. When tracking, eligibility or policy changes, annotate the series and avoid presenting the break as organic behaviour.

Measure access before engagement

Record invitations, delivery failures, joining attempts, completion and rejected access. Also record account recovery and use of alternative channels. Research why eligible people chose not to enter.

Platform analytics begin after a person arrives. They do not represent all prospective members. A survey sent only to current participants cannot explain every reason former or excluded people are absent.

Test the joining, searching, reporting and leaving journeys with relevant devices, assistive technologies and support needs. Use qualitative evidence to interpret failure and quantitative evidence to estimate frequency only within a defensible sample.

Limit demographic and behavioural collection to an established purpose. The Information Commissioner's Office explains data minimisation. More profile fields do not automatically create a more representative report.

Separate activity, workload, quality and outcome

Activity

Members, visits, searches, contributions and reactions describe use. Messages and event attendance describe use. Define unique people, events, bots, staff and period.

Use counts for capacity and journey investigation, not as automatic success; Google Analytics 4 is free for standard use but has data thresholding and consent-mode limits, while Matomo is open source, can be self-hosted, and has paid cloud plans.

Workload

Measure queue size, handling time, expert referrals, reports and appeals. Measure administrator changes, event preparation and off-platform work.

Reconcile email and support systems so the community does not appear cheap because another team absorbs its effort. Zendesk prices per agent and can connect community tickets to support workflows, while Salesforce Experience Cloud is quote-based and suits organisations already using Sales Cloud or Service Cloud.

Service quality

Sample answers, facilitation and moderation. Check relevance, accuracy, evidence, explanation and consistency. Check context and correction. The quality method should state who reviews, how cases are selected and how disagreement is resolved. Qualtrics sells enterprise survey research with advanced sampling and analysis. Typeform offers a free tier and paid plans for lighter forms.

Outcome

Observe whether the intended member task was achieved and whether it persists where relevant. A person receiving an answer is different from later applying it successfully. Do not extend the causal chain beyond what the evidence measures.

Safety and inclusion

Review incidents, repeat behaviour, appeals, exposure and access failures. Also review unequal outcomes. A low number of reports may indicate few problems or an unknown, inaccessible route. Include journey tests and member interviews.

Build a decision-led dashboard

Use separate views for operators, accountable owners and specialists. Operators need outstanding work and current service health. Owners need outcomes, capacity, cost and material risk. Restricted case information should reach only authorised reviewers.

Put the member outcome and important adverse evidence ahead of vanity activity. Show definition, period, count, comparison and annotation near every measure. Avoid a single red, amber or green status that combines safety, growth and quality.

Government guidance on using performance data to improve services describes the difference between a measurement and a metric for a decision. Although its specific requirements are for government, the analytical distinction is helpful for community reporting.

Tool choices matter. Power BI Pro is licensed per user and Microsoft publishes a monthly price. Looker Studio is free but queries may incur BigQuery charges. R and Python are free and support reproducible analysis through packages such as tidyverse and statsmodels. Dovetail is paid and helps store qualitative evidence.

Test comprehension. Give an intended reader a changed metric and ask what they think occurred and what action they would take. If the display invites an unsupported conclusion, change the note, comparison or metric.

Choose an attribution method

Impact claims need a credible account of what would have happened without the community or change. A before-and-after chart is vulnerable to seasonality, product changes, marketing and customer mix.

Experimental designs can support strong causal inference when valid random assignment, implementation and measurement are feasible. Quasi-experimental approaches use comparison groups or trends under stated assumptions. Matched cohorts adjust for observed differences but cannot remove every unobserved selection effect.

Theory-based methods can investigate contribution when a comparable control is unavailable. The Magenta Book is central-government evaluation guidance and outlines experimental, quasi-experimental and theory-based approaches. Its analytical methods annex describes contribution analysis and process tracing, including their limitations.

A commercial organisation should adapt the methods with suitably skilled evaluation support. Method names do not validate a design. Predefine the outcome, intervention, comparison, confounders, period and analysis.

Where causal inference is not defensible, say that results are associated with participation or that evidence supports a plausible contribution. Do not convert voluntary membership into an attributed retention percentage through confidence alone.

Calculate value carefully

Connect financial benefit through observable steps. A support saving requires an eligible need, appropriate community resolution and evidence that direct work was avoided or reduced. A page view cannot establish deflection.

Use this formula:

eligible volume x measured change x attributable share x unit value

Show low, central and high cases. For example, 1,000 eligible needs a quarter x 10 per cent measured reduction x 70 per cent attributable share x £25 unit cost = £1,750 a quarter. That is £7,000 a year.

Include complete costs over the same period. Platform licence may be £6,000 a year. Moderation may be £12,000 a year. Implementation may be £3,000 once. Evaluation and assurance may be £2,000 a year. Exit may be £1,000 once.

Add hidden costs from other teams. Specialist time, data engineering, legal review and training often sit outside the community budget. If total annual cost is £24,000 and central benefit is £7,000, the case needs non-financial value or a lower-cost design.

Keep member benefits that lack a credible monetary conversion as non-financial evidence. Trust, access and learning should not receive invented prices merely to fit ROI.

Research benchmarks without inventing an England average

Begin with an internal baseline using stable definitions. It provides a relevant comparison for operational change. External references require matched purpose, audience, access and maturity. Also match channel, geography and period.

Public product reports may use customer subsets and proprietary definitions. Record publisher, commercial interest, sample selection, missing cases and method. A vendor figure can provide context within its stated population; it cannot become a universal England target.

The Office for National Statistics publishes guidance on quality in official statistics. A community benchmark is not thereby an official statistic, but relevance, accuracy and reliability, coherence and accessibility are useful dimensions for scrutinising it.

Do not rank services from voluntary submissions with incompatible definitions. Where normalisation fails, publish reference cases and differences rather than false quartiles. Discourse can be self-hosted or bought as a hosted service, with paid plans. Circle and Mighty Networks sell subscription community platforms with different member and course features.

Report uncertainty and adverse evidence

Every conclusion needs its population, period, source and limit. Explain missing groups, tracking gaps, changes and alternative causes. Preserve findings that weaken the preferred explanation.

Protect member privacy. Small segments can expose identity even after names are removed. Control case-level access, set retention and avoid emailing static extracts broadly.

Ask an independent analyst or reviewer to reproduce important calculations and challenge attribution. Higher-stakes financial or public claims need appropriate methodological review.

Do not use dashboards to manage individual member performance unless the purpose, fairness and governance support that use. A metric designed for service learning can become harmful when converted into a public ranking or eligibility rule.

Establish the reporting cycle

Operational queues and safety alerts need timely review. Service quality may require weekly sampling. Member outcomes and value may need longer periods to become meaningful. Align frequency with the decision rather than with the tool's refresh rate.

At each review:

Retire unused measures. Add a new field only after defining the decision it supports, the data-protection purpose and who will maintain it.

Good measurement makes uncertainty easier to see. It connects a real member experience to operating evidence and a choice, while leaving a clear boundary around what the data cannot prove.

Before you act

Common questions

Why should a community choose the decision before choosing metrics?

Because each measure needs a clear user, and a chart nobody can act on adds maintenance and analytical flexibility without improving the service. One report can support several decisions, but the decision should come first so the measurement stays useful.

Why can a strong resolution rate still hide a problem?

A strong resolution rate among posted questions can coexist with severe barriers before posting. People who never find the community route, or who abandon joining, never appear in the posted-question figures. Measuring failure at each stage of the journey exposes those gaps.

What should a value calculation for community support include?

Use eligible volume multiplied by measured change, attributable share and unit value. Show low, central and high cases. Include complete costs over the same period, such as people, platform, implementation and specialists. A page view cannot establish deflection.