Migrating a forum

(Sarah Hawk) #1

Most of you probably know that we're about to undertake a fairly comprehensive platform migration, of both our main Feverbee site, and this forum.

The plan at this stage (which is very much the planning stage) is to move off a platform which is built on Typepad, Drupal, Salesforce and Braintree, to an integrated solution which will include WordPress, Discourse, Salesforce and Stripe. Before moving into my role here at Feverbee, I migrated the SitePoint forums (~280k active members) off vBulletin onto Discourse. It was a huge undertaking and one that I learned a lot from. This project has the propensity to be even bigger and more complicated.

I imagine that most of you will go through a migration process at least once in your career. Would you be interested if I documented the process as we go through it? At least the forum aspect of it. 

(Steven Cyrkin) #2

Very interested!

(Colleen Young) #3

Yes, please. I will be going through this soon. At least I hope soon.

(Lincoln Russell) #4

For what it's worth, I've probably migrated over a hundred forums to Vanilla by now (in addition to coding the actual migration tools from over a dozen platforms). I'd be happy to answer questions about the process, or share some of the things I look for when planning a migration.

Obviously each platform has its own challenges and foibles (and thus each combination of source/destination does as well), but I think there's plenty to look at generally. My role is typically limited to migrating the forum proper, but I do plenty of integration work before and after.

(Sarah Hawk) #5

Excellent! What do you think the most common mistake that people make when planning a migration is, Lincoln? Do you see the same ones recurring?

(Lincoln Russell) #6

HAWK - Yes, we've made a pretty comprehensive guide to avoid most of the trouble spots. The 2 big things that immediately come to mind are the social aspects:

1. Not onboarding your moderators and other stakeholders early in the migration process. We require a test migration step for vetting data transfer before the final move. During that time, it's critical to get stakeholders in the "dummy" site and using it, to get them to buy-in to the transition and have time to get acclimated. I'd go so far as to invite a few superusers, if you have some you trust, so they can talk up the coming transition and what they like about it.

On a technical level, having those extra eyes on the actually-migrated-data makes it far more likely you'll spot any problems before you switch. If only the owner/manager/authority looks at it, only their pet areas will get evaluated.

2. Not briefing your entire community on the coming transition in the weeks beforehand. No one likes a surprise when it comes to their community. Highlight some of the positive changes they're going to see. If there are tradeoffs, talk about those too and why you're making them. A statement of values here goes a long way to making the transition happier for everyone.

Also, don't panic about community complaints the first couple weeks after the migration. Let it settle in. It's really difficult to separate "moved my cheese" issues from actual UX issues in those first two weeks, so focus on actual bugs in the first month afterward and let everything else simmer.

(Lincoln Russell) #7

On a technical level, these are my Big Things:

  • If you're migrating yourself, test and test and test some more. Do the migration process over and over, and click every button afterward. You should be able to migrate the site blindfolded by the time the final migration happens.
  • Make a checklist of data to verify after the migration. Are categories organized as expected? Did you lose any user/role associations? Every forum is different here and needs special, detailed attention.
  • If you're handing off the migration, be super clear in communicating exactly what data is required vs nice-to-have, and be sure you're actually sending all the data. There is a huge amount of variation in where different types of data & files are stored, so simply "exporting your forum database" may not be enough. Make no assumptions here and articulate everything in technical terms.
  • Carefully establish 301 redirects for all major types of content (users, categories, discussions, comments) and any secondary ones you can. If you don't have a Webmaster Tools account already, get one, and monitor it very closely for the first 2 weeks after the migration for dealing with 404s and other unexpected problems preemptively. Many sites have legacy sections long forgotten that still need a proper redirect to avoid penalization by Google.

(Lincoln Russell) #8

You can drill even further into it if you're rolling your own migration and get into minutiae like "avoid renumbering unique IDs" and "optimize your SQL scripts and do as much transformation in the database engine as possible" and on and on. And, the devil is inevitably in the details of your particular situation.

HAWK - it sounds like you're in for a good challenge, but my feeling is that, with that SitePoint migration under your belt already, you've got a huge leg up already. The more you do it, the more you start seeing the common patterns in how systems store data and the "easy" way of making difficult conversions. It gets orders of magnitudes easier with each one, even if the complexity rises. Much respect for making the big jump to a modern system even if it's not my favorite one. ;)

(Sarah Hawk) #9

That is a tonne of really helpful information, Russell ‚Äď thanks for taking the time to put it all down.

be super clear in communicating exactly what data is required vs nice-to-have, and be sure you're actually sending all the data.

Yes! Managing expectations across the board is key in this process, but in my experience, one of the trickiest aspects is communicating technical aspects to non-technical people.

Also, don't panic about community complaints the first couple weeks after the migration.

Yup... another big one. Complainers will always complain, and standing on your head to rectify everything before the dust settles is ultimately a huge waste of time.

Much respect for making the big jump to a modern system even if it's not my favorite one. ;)

Heh, thanks. ;) (It grows on you... )

(Rachel Kirton) #10

This would be really helpful for me, too - we are in the process of migrating our discussion forum for 208,000 UK doctors to a new platform at the end of March so we are going through something similar right now!

Some really helpful comments on this thread. One of Russell's comments chimed in particular - that moderators need to be on board. We've acted on some of our moderators' concerns (for instance, they were worried that people would abuse a new anonymous posting function, so we have given them the ability to remove the ability for certain individuals to post anonymously). It seemed to make them a bit happier.

We did a major upgrade of our doctor email service last year and learned a lot from that - the service had been the same for 10 years pretty much and so it was a major change. We didn't do enough to guide people through the transition and as a result we burned some bridges. With the forum upgrade we are planning to do some videos to give people a flavour of the changes before putting it live, so more robust user testing, share screenshots and FAQs beforehand and start making people aware of the changes earlier to try and smooth the way a bit.

Best of luck and look forward to hearing more about it!

(Gear Buzz) #11

Yes interested in this topic

Predicament facing a lot of VBulletin, PHBB and IBP forum owners is - they are open minded about moving to move platform to Xenforo - but there aren't enough skilled devs out there to

a) replicate all the forum owners platform customisations on Xenforo

b) migrate it. 

They do exist but they are the type that will yawn in your face as you tell them the work you need done and then inform you they aren't interested or are too busy.

I am sure corporations using Lithium and Jive etc have the bankroll to get developer interest!



(Damien Trevatt) #12

Keep the updates coming. We're upgrading at the moment from tired old VB. 

Spending loads of time on the SEO aspect as well. Expecting the Google bounce but then looking forward to continuing the rise with the scope a new platform opens up to us. 

(Lincoln Russell) #13

Jules brings up another good point: replicating customizations. A migration is a good opportunity to reevaluate your current feature set and consider which ones you still need. Each platform has strengths and weaknesses, so it's important to stay focused on the big picture of why you're migrating and not get lost in the minutiae of X feature. If you need to say, "This is a great feature, but it's not critical, and we can address this at some point in the future *after* the migration" it will save you a ton of headaches. Migrations are huge, stressful events so if you can leave some things on the cutting room floor for later, it's well worth it.

(Sarah Hawk) #14

We're upgrading at the moment from tired old VB. 

What are you switching too, Damien?

(Damien Trevatt) #15

We're XenForo converts HAWK. 

Strong points on customizations - As is the feeling of many VBull customers (personal view) the platform has lost its way and has become bloated. The Experience we've had with XF so far has been like night and day. 

(Sarah Hawk) #16

I share your view of vB. It's become cumbersome. I'm interested in hearing about your experience with Xenforo. I looked at it as an alternative to Discourse for SitePoint and decided against it.

(Damien Trevatt) #17

XenForo (so far) has been a much better platform to work on both in terms of built in services and add-on's that aren't resource heavy. We've yet to load test but looking good so far. 

I met with Jeff from discourse probably 2+ years ago (forumCon San Francisco) when we first started this journey, back when they were looking for 3 volume sites to test discourse on. He wasn't sure they could manage the backend requirements we had at the time - little did I know that no one could. "Cleaning" our Vbull install / getting rid of the stuff that wasn't needed or used meant we could stay on our hybrid heavy customized platform for a bit longer (and saved us thousands in development before the transfer). But now I feel it's definitely time to move. 

Next few weeks we're on boarding more of our "power" users to the test platform to manage expectations and get feedback. But from a purely in house perspective we're impressed.

Looking forward to your documentation of the process HAWK. 

Best of luck. 

(Steve Bridger) #18

A belated 'thank you' to Lincoln for the insight he shared above :)

(Sarah Hawk) #19

@Damien How is your testing and feedback loop going?

(Damien Trevatt) #20

@HAWK too well. Positive comments only so far.