Measuring reading engagement with the Discourse Data Explorer plugin

(David McClure) #1

I’ve recently been playing around with the Data Explorer plugin in Discourse to see what I can learn about how engaged users are. Many of our users consistently come to the site and read posts, but do not (yet?) interact in any other way.

That means they don’t like anything or otherwise show themselves.

I had been occasionally checking the user list page in the Admin section, but there are also a number of users that bubble up in that list when their last_seen_at time gets bumped even though they aren’t reading posts.

So those users were just noise getting in the way of my understanding of how many folks are coming back consistently to read.

This afternoon, I created some queries using the Data Explorer plugin to help fill this gap, and I’m pretty happy so far:

Here are some of the things I’ve been able to get out of this plugin:

  • Posts Read (Daily)
    Total number of new posts read by all users per day
  • Posts Read Percentiles
    Number of posts read for users in each percentile
  • Active Readers (Past Month)
    Users with the most visits in the past month that include reading activity

What other metrics do you try to keep an eye on?

Introduce yourself (or at least just say hi)
How to Write a Community Health Report
Introduce yourself (or at least just say hi)
(Alessio Fattorini) #2

I’ll test them soon, sounds pretty cool
Just a few suggestions:

  • how many of the new comers (registered in the last month) has read at least x topics or posts? In relation to newcomers amount?
  • how many of the new comers has post at least once?
  • how many of the newcomers has visit Discourse at least x days?

These numbers might be helpful to understand my retention strategy againsy newcomers is working good or not, it’s improved/got worse over the last period

(Sarah Hawk) #3

I use and to monitor

  • new users
  • anonymous visits

as well as the standard topic/post/like counts etc

Like you, I’ve always felt a bit in the dark about what people that weren’t actively hitting posting stats were doing, so I’m going to give your read querites a go. That’s awesome – thanks very much.

Edit: Woah, Active Readers Past month makes fascinating reading!

(Alessio Fattorini) #4

Cool! Just asked a beta test

(Richard Millington) #5

A question for everyone here.

How do we make this actionable?

Whenever data gets discussed, I always sense it’s a vanity discussion. I’m really interested in plugging these metrics into our model and using it to change future actions.

Any ideas?

(David McClure) #6

At the moment, this is somewhat an exploration. It is called the data explorer plugin after all :wink:

But in seriousness, I was interested to better understand what kind of engagement our site had from the perspective of readership, and those metrics are not really surfaced too well out of the box.

The next step would be to come up with some hypotheses and experiments to improve readership.

Some ideas:

  • Let’s try changing the default digest frequency from 1 week to 2 days.
  • Let’s try adding avatars to the digest - maybe people will be more likely to visit if they see who’s posting.
  • Let’s try making sure there is fresh content that appears to personas A and B.

After running the experiment, the measurements before and after can be compared (unless you can segment the users and run the experiment with a control group at the same time).

If you see improvements in your metrics, keep the change! If not, try to come up with some theories about why it didn’t have the intended effect. Then decide what to try next…

(Alessio Fattorini) #7

Can you also see which topics they have read recently? It would be awesome to reach out newcomers with more effective personal emails.

(Sarah Hawk) #8

That would be hugely powerful. The current queries don’t do it but I imagine it’s possible. Definitely outside the scope of my skills though! @mcwumbly ?

(David McClure) #9

@ale_fattorini and @HAWK, here you go:

(Alessio Fattorini) #10

I’m astonished, I can’t thank you enough :slight_smile:
You made my day, really.

(Alessio Fattorini) #11

Just asking, can I select a range of users? And not just one? Sorting all results by users id

(David McClure) #12

Can you describe in a little more detail what you’d like to be able to input and what the report would look like?

(Sarah Hawk) #13

Brilliant and much appreciated. :slight_smile:

(Alessio Fattorini) #14

Of course, I’d like to input an id user range, for example the last 30 registrations 790-820
and I need to see all topics read by these 30 members sorted by user id and time
Is it possible? Just to not replicate 30 times the same query with different parameters
Am I more clear now?

(Sarah Hawk) #15

That would be useful but it might be hard to deliver via the data explorer interface.

(David McClure) #16

Here you go:

-- [params]
-- integer :user_min = 1
-- integer :user_max = 5
-- integer :since_days_ago = 7

with topic_timing as (
  select user_id, topic_id, sum(msecs) / 1000 as seconds
  from post_timings
  where user_id >= :user_min
  and user_id <= :user_max
  group by user_id, topic_id
SELECT tv.topic_id,
from topic_views tv
left join topic_timing tt
on tv.topic_id = tt.topic_id
and tv.user_id = tt.user_id
where tv.user_id >= :user_min
and tv.user_id <= :user_max
and viewed_at + :since_days_ago > CURRENT_TIMESTAMP
order by user_id asc, seconds desc

(Sarah Hawk) #17

Brilliant. This stuff is gold. I suddenly have a new level of visibility and it’s empowering.
I’ve just awarded you a bespoke badge.

(Alessio Fattorini) #18

Yeah, how can I pay you back?

Definitely, you deserve it!

I was sure that it was possible, it’s matter of SQL query!

(David McClure) #19

Just keep an eye out for good stuff from your own experience or other’s and please share!

I’d especially like to hear more stories of how data is actually being used to drive or validate decisions :wink:

(Sarah Hawk) #20

At the moment I’m using it to try and pinpoint the pain points of new users that sign up and read but don’t engage. If I can see that someone joined and read 5 or 6 posts about boosting engagement (for instance), I can personally contact them asking them if they need support with an engagement issue and offer appropriate resources. I’ll have to come back to you with evidence of efficacy once I’ve been doing it for a while. In short, being able to really specifically personalise the contact seems like a powerful tool.