Hello,
I’m getting closer to landing on a platform and beginning the process of setting up space for internal testing. Our staff would go online and start some discussions on the platform itself so that we can begin to test our settings, features, etc.
Has anyone gone through this step, and could you provide some tips?
I was also wondering if anyone has a testing plan, as in the things you need to test for before rolling out to a larger group.
Hi @duncanfield , we are just ramping up for internal launch on our community. I can share some of the things we are doing and planning, but not the results yet 
Would that help?
Hey Duncan,
Exciting times. This is a subject close to my heart!
Given you’re not using a bespoke platform what you really need to focus on testing are the ‘golden paths’ – the functionality that people will use most regularly.
A good approach is to list those functions (e.g. registering a new account, making a post, replying to a post, changing your username, sending a PM etc) and get each person to perform them, recording any issues/concerns/required changes as they go.
The most important thing is to carefully document each step of the process, so that if something needs to be fixed, your dev team can replicate the issue.
I’d put together a shared Google doc that everyone can access. Then I’d get your team to compile a list of those golden paths, and get each person to perform every one of them – people all do things slightly differently.
The trick to successful testing is empathy. Try and think like your users.
People get bored pretty quickly when they test, and they tend to gloss over things. For that reason you need as many people as possible on the testing team. One person only does things one way, and that’s not thorough.
If you have other questions, fire away.
Nikoletta,
That would greatly appreciated - I’d love to see whatever you have to share!
hi Duncan, I think I might have misunderstood your question. Looking at @HAWKs answer, I have attached our QA playbook. That will help you see what to cover when you are testing. QA test playbook.xlsx (77.1 KB)
Personally I get QA done with my platform admins and moderators using admin and regular user credentials. I even asked one of my writers to run through all the elements of the community to test it. for the next iteration i will use iBeta, a company who specializes in QA testing for all the devices and operating systems. I did not have all the ones available that we should have tested. iBeta is super affordable, so I will use them going forward.
Under internal launch I understand something completely different. We are launching the community to the market in a phased approached. Phase I is an internal launch to a select group of our employee base.
We are now planning on sending an email invite for roughly 300 employees (who have previously signed up to be beta testers for our company’s new projects). We will do an A/B test with these employees. Group A will receive an email invite (April 4) with very brief introduction and the URL for the community and ask them to sign up and browse and engage with the content. Group B will receive the same email a week later (April 11) plus a 2 min video that has a quick walk through and top community features introduction before they are set loose on the community. We want to measure how much difference the “primer” video will make in the engagement, interactions and depth of involvement of our testers. We will follow up with a survey two weeks later(April 18). I can share the survey questions if interested. And we will follow the survey up with a focus group (1hr conversation face to face) with the most active users from the community.
We will take all the learnings from the internal launch and refine UX, UI, Content and engagement programs before we then launch to the following audiences (also in a phased approach, but we don’t have the timing figured out yet):
This is all fantastic - thank you for your generous help.
One thing that stood out to me was the possibility of other effects on your A/B test. I can imagine that just the experience of walking into a community two weeks in will also have an effect on people’s behavior, as there will already be some established conversations/norms.
This information is going to be very helpful, as we’re also planning for an internal test phase.
This is SO important. I think every CM should have a short list of tasks which they retest monthly as a general user (e.g. registering a new account) to ensure that things haven’t broken which you’re not aware of because as an admin it works (or is of no relevance).
Services like this are awesome for testing vulnerabilities or bugs in the software, but they won’t replace in-house testing in so far as things like messaging and processes go. e.g. your registration process may work perfectly from a technical perspective, but you might find that the messaging in your welcome email isn’t persuasive, or that the primary CTA goes through to the wrong area of the community. A third party won’t pick up on those things, which can be make or break for a new community.
Oh no, I only use an external agency to do but testing. UX testing is all in house