Three Very Different Ways To Analyse An Online Community

(Richard Millington) #1

Originally published at:

Most people work from a simple assumption (e.g. “higher levels of activity per member, an increase in retention rates”). This means you measure activity per member and design your engagement activity to maximise activity per member.

The downside is you haven’t proved if the relationship is true (i.e. does it correlate well?), nor the influence levels of activity have on an individual’s retention rate (does it account for 100% or 5% of retention rate increase?), nor whether the relationship is linear (does it drop off after a certain level?).

You could blindly be pursuing more activity when data might show…

Read more.

(Ernesto Izquierdo) #2

Hi Rich,

I find that your post is also in line with the benchmark results from activity and social cohesion made the team of Laurence Lock Lee. His results show that it can be actually dangerous to only focus on activity if what you are trying to get is valuable collaboration.

"Using data from more than 135,000 people across more than 20 organizations, we have uncovered the following relevant key insights:

1. There is no association between activity levels and response rate. We expected that the more activity a network has, the more likely it is that posts are replied to. Getting a response to a post is obviously essential for collaboration — otherwise you are talking to yourself. So it was a big surprise for us to discover that there was no association between activity levels and a response.
2. Social Cohesion (measured as the intensity of reciprocated interactions) is the single measure that most differentiates the organizations in our benchmarking sample. Measuring social cohesion is equivalent to measuring your blood pressure — it is an essential health indicator. There is no association between activity levels and social cohesion. We found examples of highly active networks with poor social cohesion. That’s like having lots of people at a party, but the dance-floor is empty.
3. After the initial launch hype, the average adoption rate is around 24 percent with the highest being 63 percent. C-Suite executives are looking for a fully inclusive collaboration solution. Adoption rates are still a major challenge."

The lessons are clear, we should create our engagement strategies based on what we want to achieve, not just generating more activity on the community or network.

(Sarah Hawk) #3

Any idea how to measure social cohesion?

(Ernesto Izquierdo) #4

I’ve got some hints from that article, but I think that we have two options:

  • try to figure out with our teams (re-invent the wheel)
  • Or ask Laurence to get the specific way they do it. I would imagine that he might charge for that service, but since we don’t have Yammer his solution won’t work for us. And it will might still be easier for us than trying to reinvent the process.

Here are the hints I talked about written in his interview by Virpi Oinonen:

“Our ‘Social Cohesion’ measure is a simple proxy for ‘reciprocity’. It simply measures the degree to which your interactions are reciprocated by others in your network e.g. you reply to my posts and I reply to yours. Aggregating individual results to the whole enterprise means that we have the ability to assess social cohesion at all levels of an organisation. In network science, reciprocity is a fundamental and key indicator of trust and therefore performance.”

(Richard Millington) #5

I’m not sure social cohesion (by that metric) is the best metric here.

Let’s imagine I ask a question in an internal community and someone with
the right expertise answers the question.

By the social cohesion measure, I should then answer any question they then

The problem is I might not know the answer. In fact, statistically, the
odds of me having the specific answer to a question from someone that
answered my question must be extraordinarily low. It feels like this is
measuring the wrong thing. It doesn’t matter if the same person answers my
question, only that my question receives a good, quick, answer.

I think better measures here aren’t social cohesion by that definition, but
I’d suspect more along the lines of speed, quantity, and quality of
answers. An internal community has to be seen as more valuable than any
other communication method.

  • Rich

(Ernesto Izquierdo) #6

Thanks for the replies. For the Q&A section, we are focusing indeed in quite specific metrics:

Questions asked, solved.
Existing questions (and their answers) appear as you type.

Concerning efficiency:

  • % of new questions that get an answer within 2h (we are aiming at 50%)
  • % of new questions that get an asnwer within 24h ( we are aiming at 70%)
  • % of questions solved (we are aiming at 90%)

Concerning connections:

  • % of questions solved by people from other locations (our objective to connect people from across regions)
  • % of questions solved by people with higher or lower positions (our objective to connect people from across regions)

The social Q&A tool we are trying is called Starmind and it brings the statistics in its service, the goals we fix are the ones that they obtain with other networks with over 5K users.

I know that Philips gets 50% of new questions answered within 1h, but they have 200K employees.

I would think that the social cohesion could be more interesting with regards to blog posts, micro-blogging, or comments on documents.

Social Tools, Social Business, And Cleaning Out The Garage