A method for community management benchmark research

Community management benchmark research starts from a basic limitation: there is no single authoritative benchmark for community engagement in England. Products define members and activity differently; customer, professional and peer-support communities serve unlike purposes. A credible benchmark compares a precisely defined service with an appropriate cohort and publishes what cannot be normalised. This does not mean surveys or vendor reports do not exist; it means their definitions and coverage need to be checked before using them as comparators.

What to take away

Start with an internal baseline

Define the member outcome and measure it before a material change. Record the audience, eligibility, period, collection method and service conditions. Baseline direct-support use, event participation or existing group behaviour where those are the real alternatives.

An internal series often supports a better operating decision than an industry average. It uses the organisation's own definitions and can reveal whether a change persists, provided tracking remains stable.

Fix each metric

Write numerator, denominator, time window, exclusions, source and interpretation. "Monthly active member" may mean a login for one provider and a contribution for another. Do not convert values that use incompatible events.

Show counts with rates. A small specialist group can move sharply after one case, while a large public forum may conceal important segments inside a stable average.

The Government Digital Service publishes guidance on usability benchmarking. Its methods concern public services, but the focus on defined tasks, completion, time, confidence and repeatable testing offers a useful model for an organisation benchmarking member journeys.

Choose a comparison cohort

Match purpose, access, audience maturity, geography, channel, operating hours and measurement period. An invitation-only professional network is not comparable with a public product forum merely because both report active members.

For example, take a Discourse-based customer forum. Define the numerator as eligible members who post at least one answer during the period, and the denominator as all members eligible to use the forum during that same period. Compare it only with customer forums using the same access rules, answer definition and period; this illustrates the method, not typical performance or a published Discourse benchmark.

England-specific claims need evidence from English services or a transparent derivation. A UK-wide provider survey should remain labelled UK-wide. A global customer sample may provide product context but not an England benchmark.

Assess source quality

Prefer raw definitions and methods over a chart copied into a presentation. Record who supplied the data, coverage, missing cases, collection date, sample selection and commercial interest. Vendor benchmark reports can be useful but remain provider-produced evidence.

The Office for National Statistics explains quality in official statistics. Community datasets are not official statistics, yet the dimensions of relevance, accuracy, coherence and accessibility provide questions for evaluating any claimed benchmark.

Build a range, not a league table

Use quartiles or distributions only when underlying definitions are genuinely compatible and the sample supports them. Otherwise, present a set of dated reference cases with differences and limitations. Do not rank providers or communities from voluntary submissions.

Compare like periods and annotate product launches, campaigns, rule changes and outages. Preserve an adverse or quiet period rather than selecting only a successful month.

Maintain a benchmark evidence register

Give each external figure an identifier, source URL, publisher, access date, population, geography, period, metric definition, sample method and commercial interest. Save a permitted copy or extract where the page may change, while respecting copyright and licence terms.

Ask a second analyst to reproduce any normalisation. Currency conversion, weighted averages and estimated denominators can create apparent comparability that the source did not supply. Keep the original value beside the transformation.

Retire references when definitions or cohorts no longer match. Do not preserve a favourable old benchmark simply because replacing it would make current performance look weaker.

Where a source publishes only a chart without underlying method, label it illustrative and keep it out of target setting. Missing methodology is a limitation, not permission to infer one.

Use the benchmark for a decision

Set a question such as whether operator capacity is unusual, whether joining has become harder or whether answer quality improved after a change. Pair external context with member research and internal sampling.

Refresh sources and definitions at each comparison. A benchmark is valuable when it helps explain a gap and choose an investigation. It is dangerous when an unexplained industry number becomes a performance target that encourages empty activity.

Before you act

Common questions

Why is there no single authoritative benchmark for community engagement in England?

Products define members and activity differently, and customer, professional and peer-support communities serve unlike purposes. A credible benchmark therefore compares a precisely defined service with an appropriate cohort and publishes what cannot be normalised, rather than ranking everyone on one scale.

What should a benchmark evidence register contain?

Give each external figure an identifier, source URL, publisher and access date. Add population, geography, period, metric definition, sample method and commercial interest. Save a permitted copy or extract where the page may change, while respecting copyright and licence terms. Keep the original value beside any transformation.

How should a benchmark be used in a decision?

Treat it as context to help explain a gap and pick what to investigate, alongside your own member research and sampling. It becomes harmful if an unexplained industry figure turns into a target.