Member Spotlight: Bas van Leeuwen talks analytics

metrics
spotlight

(Sarah Hawk) #1

In the first of our series of Community Spotlights, I chatted with @Bas_van_Leeuwen. Bas is a Community Manager and founder of Community-Analytics.com and we talked about… analytics!

Hawk: Can you give us a brief bio and history of your community career?
Bas: I started my professional career as a project manager at a software company called Hippo. After a couple of years I became their community manager, which was my intro into this wonderful world.

Now I’m a freelance community manager and co-founder of Community-Analytics.com.

Hawk: What was the driver to launch Community-Analytics.com?
Bas: At Hippo I became frustrated at the lack of measurements that were available for Community Managers – especially for the (gasp) mailing list that I was managing. It was hard to find proper metrics. In the end I jury-rigged a Google Doc that automatically imported my mail. It was messy but it worked.

Later I spoke to an old colleague of mine and we decided we wanted to start a company. Fixing the lack of community analytics seemed like a good initial problem to tackle. :slightly_smiling:

Hawk: Are you wearing pants?
Bas: Yes

Hawk: Community Managers seem to feel the need to measure everything. Is that smart?
Bas: Yes it is smart to measure everything. However, in business (and life) there must always be room for intuition and intellect.

I would approach measuring in one of three ways:

  1. Measure the basics, if there is something peculiar, investigate further.

  2. Ask yourself a question based on your experience, try to find data to answer that question.

  • Do some exploratory analysis (using something like Tableau) and see if something sticks out.

Hawk: It’s not just about ROI, right? What do you measure?
Bas: ROI needs to be measured, yes, but measuring your ROI doesn’t help your community.

I’d try to focus on measuring those things that signal either:

  • something that works great, in which case you’d try to copy it
  • a pain point, in which case you should try to remedy it
  • a change in behaviour, which you should try to understand

No change in behaviour should be ‘just because’, noticing/influencing these changes – the little things – are what makes a good Community Manager great.

Hawk: What would be your No. 1 tip for people that want to get a handle on their stats?
Bas: Pick one metric (visitors, posts, registrations, whatever), measure it daily, try to get a feel for what happens and try to figure out why. Once you get a feel for it, pick another one.

Hawk: And now for a question to make me feel good. What would you say is the most valuable part of FeverBee Experts?
Bas: I really value the open culture, and the way that people aren’t afraid to admit to their mistakes to the betterment of others.


Monthly Member Spotlight
Our next member spotlight: Ambassador Programs
(Bas van Leeuwen) #2

Thanks @hawk!

Happy to answer any additional questions that you all might have :slightly_smiling:


(Nick Emmett) #3

Nice read @HAWK and @Bas_van_Leeuwen - great start to the spotlights series.
I suspect there are likely a fair few people reading this who are fairly new to the concept of measuring, both for ROI and general understanding. You mention that:

… what would you class as being the basics? What are the things that people getting started in Community Management and/or measuring/analysing their data should be looking at for a base level?

cheers


(Bas van Leeuwen) #4

Excellent question @Nick_Emmett , bit difficult to answer to be honest. I shied away from the specifics because they are very dependent on the specific community, but I’ll give it a go :slightly_smiling:

When we started Community Analytics we’ve thought long and hard about what we should display and we arrived at several base metrics (all geared to engagement), I would supplement these with the Google Analytics basics so you end up with something like:

  • Active members (nr of people that have made a contribution in the past 28 days)
  • New threads started (measured per week)
  • New registrations (measured per week)
  • New participants (how many people made their first contribution)
  • Visitors (unique sessions in GA)

Your ROI needs custom metrics, either goals or other things.

One advanced, but quite simple, trick that you can use is to tell GA if a visitor is a logged-in member or an anonymous visitor, that leads to interesting insights as well.


(Sarah Hawk) #5

What do you measure @Nick_Emmett ?

I measure:

a) New visitors to feverbee.com (GA)
b) New members (Discourse)
c) Newcomer to registered member conversion rate (b/a)
d) ``# new members who made a contribution (Discourse)
e) % of newcomers who made a contribution (d/b)
f) Active members (Discourse)
g) Total members (Discourse)
h) % of community that is active (g/f)
i) Total posts (less Hawk) (Discourse)
j) Total posts for month (Discourse)
k) Posts per active member (j/f)
i) Cohort analysis (Bas’s tool)


Deep engagement: our list of ideas and best practices
(Nick Emmett) #6

My measurements are currently:

  1. Active members
  2. Total contributions broken down into:
    2.1) New Posts
    2.2) New Comments
  3. New members per week
  4. Contributions per active member
  5. new ideas created on Ideas board

  6. growth rate
  7. Page views
  8. Visitors/Sessions
  9. Pages per session

  10. Avg. Duration of session

I use a combination of Salesforce reports and Google Analytics to pull these and most of my time based measures I do in as many combinations as I can, predominantly per week, per month and Last 30 Days (L30D).


(Sarah Hawk) #7

What quantifies Growth Rate?


(Nick Emmett) #8

I only started tracking that following this post last year. New Registrations/New Visitors


(Richard Millington) #9

just curious with everyone that’s measuring stuff, how do those measurements inform your actions?


(Bas van Leeuwen) #10

Well, that’s the next phase in the Deming Cycle isn’t it? :slight_smile:

First you need to know know what’s normal, what’s supposed to happen.
Only then can you see when things deviate and find underlying causes.
Based on those you can take action (to reduce or strengthen the measured effect).

What action you should take is really, really difficult to generalise. But will be something like
Hey, I noticed Y, I think it’s caused by X; let’s see if this is the case by tweaking X
But yeah, generic at best :-/


(Nick Emmett) #11

agreed. It informs you how things are trending and where you need to take action, which pages are losing people, which topics are pulling in and which ones need some deeper engagement. What do you need to switch up to change the trend back in the right direction. Another good article from this site.


(Ad Huikeshoven) #12

For Wikipedia and other projects there is a ton of statistics available on the stats portal https://stats.wikimedia.org/. So, excellent question @Nick_Emmett. For the Dutch Wikipedia for example there is a summary report card https://stats.wikimedia.org/EN/SummaryNL.htm which specifies i.a.

  • Page Views per Month
  • Article Count
  • New Articles per Day
  • Edits per Month
  • Active Editors
  • Very Active Editors
  • New Editors
  • Speakers
  • Editors per Million Speakers
    and a couple of graphs. There is also a nice Wikimedia report card http://reportcard.wmflabs.org/ with
  • pageviews desktop
  • pageviews mobile
  • new editors
  • active editors
    in graphs (years on the x axis) with colorful lines for different languages.

Tell me what we are missing. What is really interesting and relevant, but is not measured?


(Bas van Leeuwen) #13

What does that mean? I guess I don’t quite understand what you classify as a speaker.

Since you probably can answer a bit more specifically, I’d be eager to hear you answer @richard_millington’s question: “what do you do with this information”?


(Ad Huikeshoven) #14

The number of editors of a language version of Wikipedia is measured. The number of speakers of that language - those people with that language as native tongue is treated as an external fact. The ratio is a measure of popularity or participation rate of Wikipedia per language.
My answer to @richard_millington’s question: “what do you do with this information”? not so very much. Wikimedia chapters who receive so called APG grants in the so called FDC process have to report “global metrics” like new editors recruited, articles added and bytes added, not all of them, but what can be attributed to programs and activities of the chapter.
My guess for what is interesting and relevant but doesn’t get measured is Sense of Community Index. Does anyone regularly measure sense of community? Is that a metrics that is helpful for you?


(Sarah Hawk) #15

I suspect that is the answer from many people. Either that, or “I put it into reports for management”.

I’d love to hear from someone that uses their data to inform decisions.


(Bas van Leeuwen) #16

But that is INSANE; how can you do your job properly without knowing if your actions have any effect at all?

A doctor doesn’t take your temperature because it’s a required field on the chart… It’s used to make informed decisions and to validate the desired outcomes.


(Nick Emmett) #17

Agreed, however, it’s possibly systemic of several factors, including people that genuinely don’t know what to do about affecting the data, what changes to make and how to make them. Also, there’s probably a ton of people who have the Community Manager role as a secondary role to something else and quite possibly the other role often supercedes (I imagine).

There’s a lot of information out there, people need to know where to look for help in understanding the basics. Maybe that’s another thread - or another spotlight from someone that has that NAILED - what to do about your data trends.

I also understand the concept around around building communities and resources aimed at a higher level of CMs but is there a danger there that the entry level people are getting oushed down the chain rather than helped to get on the first couple of rungs by understanding “the basics”. One of my favourite resources on this site is this Ultimate List of Resources… I still come back to this regularly to refresh.


(Sarah Hawk) #18

I’ve been building that list out and refreshing it a bit lately. There is a lot of work to be done, but you’ll find it all here at The Knowledge when it’s finished.


(Richard Millington) #19

think i blogged about this some time back.

I’m far less interested in ‘what to measure’ and far more interested about the ‘decision tree’.

i.e. if this {x} is getting less traffic, should you cancel it or drive more traffic to it?

I think there is a HUGE amount of discussion out there on what to measure and absolutely none, literally not a single discussion I can find (aside this) about what we do with this information.

It’s crazy.


(Sarah Hawk) #20

Do you know of anyone that does this well? Does anyone know anyone that does?

(I agree with you and like @Nick_Emmett’s idea of interviewing someone that has this nailed).