Metrics for benchmarking community health


(Dean Samways) #1

As Engagement Manager at Zapnito, I’m responsible for making sure our clients who buy into our platform are using it to its full potential, recognise success and are able to achieve it.

Zapnito’s client’s are somewhat unique in so much as we are targetting players in the knowledge industry who want to share, collaborate and seek engagement with long-form pieces of content. Think medical and scientific journals entries and other similar editorial.

With that in mind, I’ve been tasked with designed an analytics set, mainly for Zapnito’s use, that demonstrates usage and engagement of each of the communities that have been created using Zapnito. To that end, my initial list of metrics to report on goes something like this:

  • Daily active users
  • Monthly active users (thanks for these two @HAWK)
  • Total visits
  • New visits
  • Conversions (from visitor to signed up member of the community)
  • Average visit duration (defined by GoSquared’s engagement metric)
  • Pages per visit
  • Bounce rate
  • Number of active members
  • Number of new engagements (comments and upvotes separated out)
  • Average number of engagements (comments and upvotes separated out) per active user
  • Number of new articles
  • Average number of new articles per active user

[Context: The list is partly made up of what I understand to be good engagement metrics from my social media management experience, the key engagement indicators according to GoSquared, and the headline requirements of one of our key clients]

So, my question I guess is, would reporting on these metrics adequately reflect the health of a community? Perhaps there’s something vital I’m missing or I’m including something that really won’t offer me much value at all.

I’d be interested in hearing your thoughts @HAWK. Feel free to bring others who may have an opinion on this into this thread also.


[27 Feb] What are you working on this week?
(Sarah Hawk) #2

Note: I’ve split this into a new topic @dean_samways so it doesn’t get lost in the other thread.

I’d also go with DAU/MAU (not individually) which gives you an indication of the community’s ‘stickiness’ – more info here.

It’s also important to separate out posts/engagement by the community team.


(Dean Samways) #3

Ah I see!

Adding DAU/MAU to the list.

By this I assume you mean content created by the members of staff, not visitors?

What is your view on everything else in the list? In your opinion, is it likely to give me valuable data?


(Sarah Hawk) #4

Yes – esp in the early stages. Quite a high proportion of posts could be from staff, which implies engagement that doesn’t necessarily correlate with health.

I think you’ve covered things off with that list. Here is what I measure in my own communities.

The only thing I feel I should add (for people reading this in the future) is that engagement metrics are great for demonstrating some aspects of community health, but they aren’t necessarily the most important ones to measure. e.g. If the primary goal of your community is product retention, then the number of posts is probably of little relevance.


(Dean Samways) #5

This is awesome. Thank you @HAWK.


(Dean Samways) #6

Hi @HAWK,

So here’s what I’ve settled on so far:

Engagement KPIs Zapnito should be tracking include:

  • Daily active users (registered, logged in users visiting site)
  • Monthly active users
    Stickiness (DAU / MAU)
  • New Users or Conversions (new registrations)
    Conversion Rate (New Users / User Visits x 100)
  • New Articles
    No. of new articles per active user (Articles / Monthly active users)
  • New Posts (no. of responses to threads (including initial post))
    No. of posts per active user (Posts / Monthly active users)
    Article Engagement (Posts - Articles)
  • First response time (time before first response to a new topic)
  • Actively engaged time per visit (defined by GoSquared’s engagement metrics)
  • Pageviews per visit (defined by GoSquared’s engagement metrics)
  • Bounce rate (defined by GoSquared’s engagement metrics)
  • Posts by community team
  • Engagements by community team

Secondary (vanity) KPIs:

  • Total visits
  • New visits

NOTES:

  1. Bullets indicate key metrics I feel I must look to track (unless listed under ‘Secondary KPIs’) and will require work to pull and flow data in into an analytics set as a result
  2. Italics represent metrics that require a simple formula in calculating other already-sourced data
  3. Secondary or vanity KPIs are those which I consider to be less of a priority for the purposes of measuring engagement but understand that some clients may want to track their network’s popularity as a standard online destination for visitors

These are going to remain under review until I can speak to our primary customer tomorrow (Friday 2 March) when I will seek their feedback.

If the wider community has any views on my intended analytics set please do feel free to chime in.

To give you some context as to why I believe this data in particular is significant, this is what Zapnito is all about - enabling knowledge industry organisations to share, promote and monetise, collective expertise, i.e., publishing long-form intellectual pieces of material for the consumption, sharing and visible deliberation of the community.


(Robert McIntosh) #7

Just a further thought, as I am going through a similar exercise, … what about tracking your gamification stats (assuming you have one)?

In some ways these are only proxies for other stats, but you could take a subset of your user levels or achievement milestones and measure the numbers who have achieved them as a proportion of total members, also looking at the recent changes.

In discourse it might be to see what percentage of members have moved from one Trust Level to another and achieved certain additional rights. Similarly, what % have achieved certain key badges (such as answering a set number of queries, or posted certain total number of posts)

Do you, or @HAWK, track any of this, and is it worth it?


(Sarah Hawk) #8

I don’t because it’s not aligned with the values of my audience, but I can totally see that it might be useful in other types of communities where gamification was strong. I like the idea.


(Robert McIntosh) #9

Agreed. I am not keen on overemphasising gamification, but I can see that it could create useful metrics.

For example, if there is a specific “New Member Questions” category, and participants gain badges for responding to these queries and helping to bring on board new members, it would be useful to know what % of members have gained that badge. It could be that there is a lot of community activity in general discussions, but new members are being ignored, so it would be a useful ‘health indicator’


(Dean Samways) #10

Thanks for this @thirstforwine.

As the Engagement Manager for a platform provider that doesn’t necessarily have gamification levers inline with what you’re talking about, I’m not sure how useful this would be for me. However, that’s not to say this isn’t something that I might suggest we add to our pipeline in the future.

Definitely given me something to think about.