Social collaboration metrics – what does engagement really mean?

(Ernesto Izquierdo) #1

We are discussing what to measure in our future social collaboration network. I received this post by Virpi Oinonen interviewing Laurence Lock Lee and found it super useful.

“You really need to be clear about what your executive sponsors are looking for. The common response is ‘increased employee engagement’. For me ‘Engagement’ means open and transparent conversations. Therefore I would advocate that the breadth and depth of conversations on Yammer should be a priority over simpler activity measures.”

What should we measure?
"Social cohesion, it simply measures the degree to which your interactions are reciprocated by others in your network e.g. you reply to my posts and I reply to yours.
By focusing on amplifying social cohesion, you will generate a more sustainable collaboration performance.

The choice is clear.

If you define engagement as engagement with the technology, then activity measures will suffice.

But if you define engagement as people engaging with each other in productive collaboration, then social cohesion and related relationship centered measures are essential."

Another really useful post cited in that article is this one: The smoke and mirrors of Enterprise Social Networking metrics

Both are worth reading.

[12 September] What are you working on this week?
So, what are you working on?
Giving the best first impression (through email) after members sign up
(Sarah Hawk) #2

Interesting topic Ernesto. I’m keen to hear what metrics you end up deciding on.

By using analytics designed for media consumption and not collaboration, efforts to drive adoption are misplaced.

It sounds like collaboration tools are going through the same ‘what do we measure to prove ROI?’ crisis that we had with community platforms over the last couple of years.

There is no association between activity levels and response rate.

This is particularly interesting to me. Again it sounds like a fairly typical community crisis – the wrong behaviour is being modeled.

Both @Julie_Schoeneck and @jamestyer have experience with Yammer based communities. I’d be interested in hearing how they approach defining metrics.

(Ernesto Izquierdo) #3

Hola Sarah, We have at least five areas to keep an eye on:

  1. The global community
  2. The global Q&A community
  3. Specific communities: communities of practice, learning communities, projects, forums in Team Spaces and Event spaces
  4. Personal profiles
  5. Blogs

We are getting inspiration also from the experience by Denis Agusi at Philips.

Our high level objectives are to:

  1. connect people
  2. enhance peer to peer support
  3. improve our ways of working together
  4. enable us to accompany major change projects and our ongoing transformational processes

From that we aim to measure:

  1. How people are connecting on the network across geographical regions and hierarchies (replies, @mentions)
  2. How fast we are finding solutions to questions, and the quality of the answers (with the smart Q&A tool)
    3.A How fast are we finding the right internal experts.
    3.B We still need to figure out how to measure the or reciprocity index mentioned in an earlier post, we don’t have Yammer or the tool to do that for us.
  3. Visualize trending topics internally which should give us ideas to support our internal influencers (Social network analysis could help in that)

For the specific metrics, we still have to define them and make a system to follow up on these for some of those areas.
For the social Q&A we’ve got it covered, the tool we’d like to implement brings already interesting stats.

If other members here have ideas or practical tips, I’m all ears :slight_smile:

(James Tyer) #4

Haven’t been in here in so long!

How did your launch go? My main advice would be that while it may be turned on for everyone, forget about that generally. Forget 30 days of Yammer and other Microsoft rubbish for a while too.

Find real business value cases, which might mean 10 people. Get them using the tool deeply. These success stories get you your stories to sell to others. They can put your efforts into their own business metrics.

At Kellogg’s I spend my first few months speaking to close to 2000 employees (in groups most of the time, but generally invited by someone interested). I didn’t suggest solutions, just asked about their jobs. Through these conversations I found my first user cases.

I’d be happy to talk more if useful.

(Ernesto Izquierdo) #5

Hey James, we have not launched yet. But I take your advice indeed. Our main focus will be to support those colleagues who have requested communities with quite specific needs. I’ll let you know how that runs.

(James Tyer) #6

Hey Ernesto - how did it go? Funny that you started with a link to Laurence Lock Lee…I now work for him at SWOOP Analytics (it’s a tool I tried very hard to get budget for at Kellogg’s).

(Ernesto Izquierdo) #7

Hey James, we have launched 5 communities of practice on SharePoint so far. Two have matured to a “community” level, beyond being just a space where one person spreads information. We’re about to launch 3 more in the upcoming weeks. We have no real tool for measuring behaviors, everything is done manually. We should be able to get a new tool next year. But it’s still work in progress. I’ve read that SWOOP is now working with Facebook Workplace, that must be an interesting opportunity.