Data Migration can be a challenging and stressful procedure, even for knowledgeable experts. The more complex the system, and the longer you have been gathering data, the harder it can be to accurately understand, process, and transfer all your important business data.
Over the years, possibly you changed some of your business processes or added various new applications. In any case, the way you handle your business data can change over time. Also, that various people simply are more thorough at contributing data than others. Because of all of these factors, data migration frequently gets the reputation of being quite a headache.
However, when working with any new execution, your data migration stage is crucial. Any kind of small interruption can be annoying and cause delays, while significant problems can potentially cause the implementation to fail totally. So with all of this in mind, and in the interests of everybody migrating data out there, we need to introduce a couple of tips to help guarantee an effective and successful data migration.
1. Start by deciding on what type of data you want to migrate
Especially if you are moving to a new CRM or another system, do yourself a favor and start fresh. Don’t just transfer everything. You wouldn’t throw all your old junk and trash from your old car into a new one, so don’t do it with your data either.
At the beginning of this project, you should decide exactly what it is you really want to keep, and how far back you need to go. If you have been in business for 20 years, is it really going to be that helpful to know open and close dates for accounts in 1998? Was your system even digital then? Don’t be afraid to clip out fields and data that you just don’t use anymore.
Also, because you are already taking the time to look at your business process, you can also take this opportunity to identify what fields or data you don’t currently have but would like to begin adding and tracking.
2. Make realistic time estimates
Data migration is complex and time-consuming. Don’t assume it will be quick and easy, no matter how “simple” you think your database is. There will almost always be a number of hidden challenges and slowdowns along the way that you weren’t able to foresee when you got started.
What takes so long? Well for starters, some amount of time is needed for every aspect of every field, including understanding the field, mapping the source field to the target field, configuring or building transformations, performing tests, measuring data quality for the field, and so on.
In particular, understanding the source data – which is likely the most important part– cannot be automated by tools, but requires analysts to take time going through the list of fields one by one.
Long story short, making an accurate estimate can be challenging, but no matter what you come up with, make sure you are generous with your time so you have realistic expectations going into the project.
3. Take a look at your data quality
Having poor, or ‘unclean’ data can be one of the biggest issues you can encounter during data migration. For example, incorrect formatting on phone numbers or dates can throw off the whole process via mapping errors. And while it may not make the migration fail, let’s be honest, things like N/A in a name or company field don’t really do much good for anyone.
Yes, it can be a lot of work to look into all of the different data fields and fix any outstanding issues, but if you don’t, the whole data migration may fail. So, it should go as a given that you should get started cleaning your data ASAP and give yourself plenty of time to do it right.
4. Involve the business team
So far we have been talking a lot about working with fields, but an essential ingredient here is to loop in your business people. You know, those people who actually use the fields in their everyday jobs. They are the ones who know the system’s ins and outs, and can best advise on what data to keep, what to get rid of, and what can be done to make improvements.
A recommendation here is to run a test batch and let your business team look at the data through the new UI so they can really see and get a feel for how it works. Not only can your team see it in action, but you get to ensure that all fields are populated and all expected relationships are working. If all is good, then go forward with the rest of the data migration.
5. Get ready for long data loads
As great as Salesforce is, it does have a few drawbacks. One of them is slower speeds when it comes to moving data back and forth from the cloud.
Local databases with good hardware can move at lightning speeds, however, a Salesforce Cloud transfer has many factors that can slow down the process, such as the network speed, Salesforce API bottlenecks, and custom Salesforce code which can also limit loads and processing.
A common strategy is to batch your data loads to ensure successful transfers, smaller load times, and to avoid clogging up the entire processing bandwidth. Just imagine how upsetting it would be to be 90% of the way through a 2-week data load and to be hit by a power outage.
6. Document everything
Ironically, our final data migration tip is to create… more data! However, this data is essential. You should ideally document everything you do in the process of data migration so that if there are any changes to the project team or you need to move your data in the future, you will have a guide available.
What information should be documented?
- How did you decide which data to migrate / not migrate? And why?
- Lessons learned, including how long it took to prepare your data for the migration. This way, if you move your data in the future you have a better idea of how much time to allow.
- What exactly have you had to do with the data to ensure it is migrated successfully?
- Note the data restrictions for the fields in both the existing system and the new system.
- Project timelines including milestones and key dates you achieved them.
- An issue log which includes any problems you came across.
- The relationships between the records (primary keys and foreign keys).
- The project team and their roles.
Hopefully, these tips have not only given you a better understanding of the data migration process but the next time you are moving all of your records, you can have a better, smoother, and more successful project.