Operations
Part of Community management operations: workflows and review
Community management launch review: trace member journeys
A community management launch review traces member journeys and operating evidence to guide the decision to improve, restrict, expand or stop a service.
A community management launch review should decide whether to continue, correct, restrict, expand or stop. It is not a celebration report. This desk-review framework uses evidence an English service team can collect from its own pilot; it does not claim first-hand testing of any community.
What to take away
- A launch review should decide whether to continue, correct, restrict, expand or stop, not celebrate.
- Trace complete member journeys from invitation to outcome to understand motive and off-platform work.
- Reconcile queues with interviews and handoffs because work can disappear from the apparent workload.
- Check controls in practice, including permissions, retention and moderation appeals, as the service develops.
- Compare cost and outcome using invoices and recorded staff time, connecting benefits through a causal route.
Revisit the original decision
Restate the intended member group, task, activity, operating boundary and accepted risks. Note any changes made after recruitment began. A successful launch cannot be judged against objectives rewritten to match its most flattering numbers.
List who joined, who did not and which recruitment routes were used. Do not describe the cohort as representative of England unless the sample design supports that claim.
Trace complete member journeys
Sample people who achieved the intended outcome, received no answer, watched without contributing, reported a problem and left. Follow the sequence from invitation through outcome. Platform events alone cannot explain motive or off-platform work.
Member journey sampling routes
- Invitation sent
- Achieved intended outcome
- Received no answer
- Watched without contributing
- Reported a problem
- Left the community
Interview participants about recent experiences without leading them towards praise. Include access and support needs. Government guidance on analysing user research is written for public-service teams, but its practice of involving the team and grounding findings in observations can inform a commercial review.
Worked example. Trace one invitation from a named recruitment route. The invited person accepts, opens the sign-up form, abandons it before verifying an email address, and never posts.
The platform log shows one visit and no content, which might suggest low interest. The interview can show a different barrier, such as a verification message sent to a spam folder. Record the route, first action, barrier and outcome for each case.
Examine delivery evidence
Review acknowledgement and resolution, answer quality, corrections, reports, appeals, expert handoffs, administrator work and outages. Compare actual effort with the staffing model. Identify tasks absorbed informally by product, support or volunteer colleagues.
Delivery evidence to review
- Acknowledgement and resolution times
- Answer quality and corrections
- Reports and appeals
- Expert handoffs
- Administrator workload
- Outages and informal tasks
Read a sample of cases rather than relying only on averages. A short response time may conceal irrelevant replies, while a low report count may reflect an inaccessible process.
Reconcile queues with member interviews and specialist handoffs. Work can leave the platform through email, meetings or support systems and disappear from the apparent workload. Ask operators to identify tasks they performed but could not record. Then improve the evidence route rather than treating memory as a permanent reporting system.
For workload reconciliation, set an illustrative threshold such as five hours a week of unrecorded work across the team. Above it, treat the evidence route as the first gap to fix. The right figure depends on team size, case volume and how much of the work is already logged.
Check controls in practice
Confirm that member visibility matched the explanation, permissions remained appropriate and retention or deletion behaved as designed. Review supplier incidents and integration errors. Under the UK General Data Protection Regulation (UK GDPR) and Data Protection Act 2018, check that safeguards remain appropriate as the service develops; the Information Commissioner's Office's data protection by design guidance supports this. Also check whether the organisation must pay the ICO data protection fee under the Data Protection (Charges and Information) Regulations 2018.
Controls to confirm in practice
- Member visibility matched explanation
- Permissions remained appropriate
- Retention and deletion behaved as designed
- Supplier incidents and integration errors
- Moderation decisions consistent and appealable
Assess moderation decisions for consistency, context and appeal. If the service is a user-to-user service within scope of the Online Safety Act 2023, check its duties and Ofcom guidance; where content raises a suspected crime or immediate risk, follow the documented escalation route for the service rather than asking the review team to make a legal judgement.
Compare cost and outcome
Update setup and recurring costs using invoices and recorded staff time. Connect benefits through a documented causal route. Posts, logins and page views can describe activity but do not prove retention, support savings or member learning.
Present non-financial effects separately where no defensible monetary value exists. Include harm, exclusion or burden alongside positive outcomes.
Review supplier and technical performance
Check actual support cases, availability, administrative changes, data flows and export behaviour against procurement evidence. Distinguish a supplier failure from an unsuitable buyer configuration or unclear ownership. Each cause leads to a different remedy.
Diagnosing a supplier failure
Was the failure supplier-side?
Supplier remedy or replacement
Fix buyer configuration or ownership
Run an administrator succession exercise and confirm critical contacts outside the service. If a key integration failed during launch, follow the affected member record through repair and deletion across each system.
Recheck the exit test after configuration and real content structure have changed. Carry each technical finding back to the member journey it interrupted.
Make the next choice
For each gap, assign consequence, evidence, action, owner and date. Expansion requires a fresh workload and risk scenario; a larger audience can change moderation, support and access.
Keep one row per gap. Use these columns:
| Gap | Consequence | Evidence | Action | Owner | Date |
|---|---|---|---|---|---|
| [the gap] | [effect if unresolved] | [case sample or log entry] | [change to make] | [named role] | [review date] |
The accountable owner should approve one decision with conditions and a review point. Archive the evidence, definitions and rejected explanations. A rigorous launch review protects the community from momentum: it allows a promising service to improve and an unsupported idea to stop before sunk cost becomes its main justification.
Before you act
- Revisit the original decision and note changes after recruitment began.
- Sample people who achieved, received no answer, watched, reported or left.
- Review acknowledgement, resolution, answer quality, corrections, reports and appeals.
- Confirm member visibility, permissions and retention behaved as designed.
- Update setup and recurring costs using invoices and recorded staff time.
- Assign consequence, evidence, action, owner and date for each gap.
Common questions
Why should a launch review trace member journeys instead of reading the report?
Platform events alone cannot explain motive or off-platform work. Sampling people who achieved the outcome, received no answer, watched, reported a problem or left shows the sequence from invitation through outcome. This evidence helps decide whether to continue, correct, restrict, expand or stop.
How can a review reconcile queues with member interviews and specialist handoffs?
For each queue, compare the recorded volume with the member interviews and the specialist handoffs. Ask operators to identify tasks they performed but could not record. Then improve the evidence route rather than treating memory as a permanent reporting system.
What should happen after identifying gaps in the launch review?
For each gap, assign consequence, evidence, action, owner and date. Expansion requires a fresh workload and risk scenario because a larger audience can change moderation, support and access. The accountable owner should approve one decision with conditions and a review point.
