Metrics on a closed community?


(Duncan Field) #1


We’re running a closed community that is (almost entirely) limited to members of our organization. This is still a pretty big pool, with a reach of 20 000+.

We’re beginning to coordinate with a few other similar organizations, who also want the group to remain ‘closed’. I know growth is the sexy metric, but has anyone had experience in reporting activity in a closed group? I realize in practicality every community has a finite membership, but the situation seems more defined in this case.

We’re already looking at engagement rather than membership, but has anyone had experience dealing with boards/bosses in this context? I’m trying to pre-empt any issues here.

(rhogroupee) #2

Whenever someone asks about metrics, I always try to steer them back toward the purpose of the community. That will reveal what the key metrics should be. Looking back at one of your introductory posts, you said “The purpose of the community is to connect charities who are solving similar problems (something they’ve been asking for for years, and connecting groups of people that have been existing separately under a single banner.”

With that purpose in mind, I’d look at measuring “problems solved,” which might be surfaced by periodic interviews with the key charity participants, to see if they feel this is happening. You could also look at member to member connections as a way to measure whether the separate groups are coming together as you desire.

Your reporting might be better off including some stories, rather than just numbers, when you present to the bosses.

Hope that gives some possibilities!

(Sarah Hawk) #3

I 100% agree with what @rhogroupee says. Whether or not the community is closed shouldn’t technically affect what you measure, if you’re measuring the right things.

Establish what the primary goal of the community is and then tie it to a key value metric. That is what you should be measuring. More info on doing that here.

Measuring engagement as a fail safe (to ensure that you are maintaining community health while you focus on your strategic goal) is important too, but it shouldn’t be your primary focus.

Does that make sense?

(Duncan Field) #4

@rhogroupee @HAWK Those are both amazing responses, thank you.

I’m going to take these ideas and run with it

(Matthew Dodd) #5

We are in the process of launching a similar closed community. We had similar discussions about demonstrating success by surfacing the success stories. @duncanfield I’d be really keen to hear how you managed to surface those stories - and also whether that satisfied your boss/stakeholders?

We are trying to see if we can find a way of flagging where a discussion has resulted in a best practice document being produced or a new school or employment policy being introduced. We were also considering adding in an automated script to remind a member to update the community on success after a period of time. After all, it might be several months after they have engaged the community to solve their problem and a result coming in. Telling everyone what happened won’t be top of mind, but if prompted they might be inclined to update.

(Duncan Field) #6

This is great

A good way of tracking those documents is encouraging regular use of a community wiki - one that is specific to that specific function-group. This is easier for us because by design our larger community is split into areas specific to certain functions (finance, hr, etc), meaning that it’s easy to capture useful documents. You would be able to track how many documents are uploaded, or with more work, how many people collaborated, how many requests for help were successfully answered (or a good attempt was made). That would be a good plug-in, a button that says whether a request or question has been answered to the asker’s satisfaction.

Do you have a good set of volunteers/moderators? A good idea is to get them to save/screengrab examples of positive interactions like this so that you can relate some actual stories on top of existing numbers (messages, average messages/posts per user, number of user generated discussions versus seeded, participation in regular weekly/monthly/etc community events).

I have found that most C- level supervisors want numbers/charts. I have found it to be useful to provide a smaller amount of criteria, shown visually, paired with notable examples of the behavior being tracked. This will help you demonstrate the value of the things you are tracking, and help you avoid having to bump simple traffic to increase that particular number.

I think it’s a common piece of knowledge here that there are far more useful numbers than simple visits, but you have to demonstrate the value of alternative metrics. For example, we are focusing on the engagement of existing members, which is a fraction of our total possible membership. That is working for now, but we are also tasked with demonstrating actual growth, or interactions that will lead to it, to keep everyone happy.

This involves a lot of compromise.

Is that helpful? Great question!