Data Migration Software: Which Solution Fits Your Project Best

Businesses can no longer count on using the same technologies for decades. Hardware and software become obsolete sooner than ever before. So data migration is an unavoidable challenge each company faces once in a while.

Transferring data from one computer environment to another is a time-consuming, multi-step process involving such activities as planning, data profiling, and testing, to name a few. You can read more about it in our previous article Data Migration: Process, Types, and Golden Rules to Follow. Here, we’ll focus on tools that can save you the lion’s share of tedious tasks — namely, key types of data migration software, selection criteria, and some popular options available in the market.

Types of data migration tools

Three types of data migration tools.

Self-scripted tools

Automation scripts can be written by data engineers or ETL developers in charge of your migration project. This makes sense when you move a relatively small amount of data and deal with simple requirements. Besides, self-scripted tools become a necessity if a source or target location is not supported by any other software.

Among cons of the do-it-yourself approach is the need for coding skills, extra time your engineers have to spend on scripting, and scalability issues.

On-premise tools

On-premises software is meant for migrations within an enterprise network or when mergers and acquisitions take place and moving to the cloud is not on the agenda. Such tools are a natural choice if compliance or security requirements restrict companies from using cloud-based technologies.

The bright side of the on-premise tools is the ability of the company’s IT department to take full control over every single data element and migration step. On the dark side, your engineers must keep everything up and running on their own. Besides, this type of software has limited scalability compared to cloud solutions.

Cloud-based tools

These tools automate data migration to the cloud or from one cloud location to another. They are usually very flexible and handle different types of data. Besides, cloud-based solutions easily scale up and down depending on your current needs. They typically offer a pay-as-you-go pricing model that wards off wasting money on unused resources.

The greatest concern about cloud solutions is whether it can guarantee protection of data in the move.

Now, that you’ve got a general picture, it’s time to explore key selection criteria in more detail.

How to choose the right data migration solution

Functionality

  • planning including job scheduling and workflow orchestration,
  • pre-migration data profiling,
  • migration execution — or Extract, Transform, Load (ETL) processes,
  • testing, and
  • post-migration audit (validation).
Phases of the data migration process.

If a tool lacks functionality, you have to complete it with another solution or task your data engineers with creating scripts to fill the gaps.

Data sources and destinations

Ready-to-use tools, both on-premises and cloud-based, cover a wide range of systems and formats. Nevertheless, you should carefully check whether your files to be migrated and the environments are on the list of supported options.

Performance and scalability

Ease of use

Customer support

  • by email,
  • via a forum, or
  • via a live chat.

With e-mail or a forum, you ask a question and just wait for someone to answer it which can take hours or even days. Live chats are better options to get a quick response. But this is to be checked beforehand. You can read reviews by other customers to understand if the quality and speed of support meet your requirements.

Security

Pricing

Free solutions are typically developed by large corporations as a supplement for their main product. They are of a high quality but with limited functionality intended for their core system.

Open-source solutions are either free or significantly cheaper than commercial alternatives. But you get what you pay for. Such as, they can be poorly maintained and updated, have limited functionality, and require coding skills to work with them. Some open-source products are really good — however, you’ll get access to only a few features while the rest is fee-based. Among most popular open-source tools used for migrations are Apache NiFi, CloverDX, and Pentaho Data Integration.

Commercial products offer a wide range of features along with user-friendly interface, training, and technical support, which is most suitable for organizations lacking expertise in data migration. They also tend to cover numerous data sources and targets.

Best data migration software overview

  • data replication — or continuous duplication or making copies of data to store the content in different locations;
  • data integration — or gathering data from different sources to provide users with a single view of information.

Similar to migration, data integration involves Extract, Transform, and Load procedures. So, products supporting data transfer are often called integration or ETL tools.

With this in mind, let’s examine popular end-to-end, system-agnostic solutions — in light of the mentioned-above selection criteria.

Data migration software comparison.

Astera Centerprise: focus on migrating complex and legacy data

Functionality. The software covers the entire migration process, from profiling the source data to validating the content loaded to the target location. It provides a variety of tools to schedule, orchestrate, preview, and monitor the migration flow. Particular attention is paid to the mapping of complex hierarchical data.

Data sources, and destinations. Astera offers pre-built connectors with a wide range of sources and destinations — for example, Oracle, SQL Server, MySQL, PostgreSQL, SAP HANA, Salesforce and more. It features over 200 built-in tools to enable processing any type of data, including complex formats like XML, JSON, EDI, COBOL as well as with unstructured files (texts or PDFs).

Astera Centerprise use cases, components, and supported data sources/destinations.

Centerprise is great for database migrations and processing complex datasets. However, when it comes to metadata, the solution leaves room for improvement as it requires some manual post-processing. Besides that, many clients wish Astera had more pre-built connections with popular cloud storage services and apps.

Performance and scalability. The parallel-processing ETL engine ensures high performance even when you move large datasets. To boost the speed of migration, there is a Job Optimizer feature that allows you to modify the flow at run-time (say, to remove unnecessary operations) and thus reduce execution time. As with any other on-premise software, Astera Centerprise scalability depends on your hardware.

Ease of use. The drag-and-drop interface enables no-coding data migrations, so the process can be handled by data analysts or even business users with no technical background. Simple to set up, configure, and get started with, Centerprise is often referred to as “the most intuitive data migration tool.”

Security. The special Security Manager feature enables you to assign user roles and create different access levels to specific resources and operations.

Customer support. Astera is known for superior support services provided via email or conference calls.

Pricing. The product is rather pricey for small companies, but you can try it no matter your business size — just download a free trial version from the provider’s website.

Talend: a fast shift from anywhere to AWS and other cloud locations

Functionality. Free solutions are enough to prepare data for migration, orchestrate tasks, and run a simple ETL process. Commercial products cover the entire migration cycle and boast a large number of features including automated quality checks as data moves between locations and visual mapping for complex data structures.

Data sources and destinations. As an AWS Migration Competency Partner, Talend guarantees smooth data transportation from anywhere to Amazon cloud facilities. But it also deals with numerous other on-premise and cloud sources and destinations.

Its free suite supports Oracle, Teradata, Microsoft SQL Server, Salesforce, AWS, Microsoft Azure, Google Cloud platform, and more. Commercial products offer a far longer list of available connectors. It works particularly well with databases and CRM software.

Performance and scalability. Talend is praised for its ability to transfer millions of records at a time without any issues. Even its free ETL tool shows the impressive speed of migration and scalability.

Ease of use. The software features an intuitive drag-and-drop interface. However, to perform complex migrations you will need tech specialists with expertise in Java.

Security. The company has a dedicated team of security experts to ensure data protection at each level, from preventing physical damage of data centers to cloud workload monitoring. Talend products are compliant with main industry standards including HIPAA, GDPR, and CSA STAR (the Cloud Security Alliance Security Trust Assurance and Risk program).

Customer support. Clients can get free support from the Talend Community and find the documentation they need in the Talend Help Center. Professional tech support is fee-based, and the response time depends on the support level (Silver/Gold, Platinum, or Mission Critical) you have purchased.

Pricing. Talend has an open-source version with 900+ free components and connectors that you can download from the vendor’s website. Commercial products come with a 14-day free trial and the price depends on the package you choose and the number of users. Its data integration suite costs $1,170 per user/monthly or $12,000 per user/annually.

Informatica PowerCenter: enterprise-grade migration from multiple sources

Functionality. PowerCenter provides end-to-end functionality for data migration projects. It offers parallel management and monitoring mechanisms, load balancing, repeatable audit and validation of data, and other enhancements.

But if you see room for improvement, your data engineers can create rule-based workflows to optimize the overall process.

Data sources and destinations. The platform easily connects with all leading database management systems (DBMSs), cloud services, application systems, etc. It creates advanced data transformations for XML, JSON, unstructured documents, industry formats (like HL7 or EDI), legacy formats (COBOL). IoT machine data, and more.

The platform is unrivaled for migrating data from different sources But if you deal with a single data source, you can opt for one of the numerous cheaper options.

Performance and scalability. Due to the parallel ETL processing, the solution demonstrates great performance even when dealing with large datasets. It’s scalable enough to cover the needs of enterprise-grade projects.

Ease of use. Though Informatica provides a graphical interface to create data migrations, simplicity is not the strong point of the solution. PowerCenter requires more learning than many other ETL tools. According to product reviews, set up and integration with existing services can be a hassle, while training may take a few weeks.

Security. The vendor provides documentation to manage security within the Informatica domain. The measures encompass authentication of users and services, protection of data transmitted and stored, the security of connections for external components, access control to data and services. You may also employ an additional tool to ensure compliance with regulations like GDPR, CCPA, and CCAR.

Customer support. Customer support is provided via email, phone, or Informatica network. The regional support team is available during local business hours.

Pricing. The high pricing on connectors and overall usage makes this solution unsuitable for mid-size and small businesses. To make sure that the cost is worth it, enterprises can take advantage of a 30-day free trial.

StarfishETL: safe switching between CRMs and ERPs

Functionality. The system simplifies the most popular migrations — based on customer feedback. It performs commonly requested mappings, ensures data quality, and allows you to schedule and control workflows. But you can go beyond pre-built features and run more complex migration projects using self-scripted tools. StarfishETL supports Python, C#, JavaScript, and several other scripting languages to expand your possibilities.

Data sources and destinations. The solution focuses on CRM systems ensuring quick move from one to another. However, it also offers pre-built connectors with popular databases (MongoDB, SQLite, PostgreSQL, MySQL, Oracle, Amazon Redshift, and others) and supports Excel and XML files.

Performance and scalability. The platform runs on AWS and takes advantage of Amazon Elastic Cloud capabilities that make the migration tool suitable for companies of different sizes. It is powerful enough to quickly move information from legacy CRM and ERP systems to the cloud for midsize businesses. Complaints arise when dealing with huge amounts of data. Also, many clients wish the platform could run more than one scheduled job at a time.

Ease of use. The tool aims at computer-savvy users who have scripting experience and a basic understanding of data types, databases, and web services. For those without а technical background, it won’t be quite simple to make the most of the system and its capabilities.

Security. To ensure safety, the system applies strict data encryption creating two-fold security protocols between their internal servers and AWS. StarfishETL also protects information from system outages by backups.

Customer support. You can find technical documentation, ask questions to the community, or leave a ticket describing your problem on the Support Portal page. Another option is to submit a consultation request to the dedicated support team. The response time depends on the severity of your problem and the level of your subscription. It ranges from one business hour to one week.

Pricing. The fee depends on the number of connectors and transactions and starts from $495 per migration from a single source to a single target. Some additional options are really expensive for small businesses.

There is still a lot of work for data engineers

But even with the best software in place, you will still need data engineers to define a migration strategy, run the migration process, validate results, and fix any issues. A tool is as good as the hands using it — this statement is fair enough for migration of any complexity.

Originally published at AltexSoft tech blog “Data Migration Software: Which Solution Fits Your Project Best

Being a Technology & Solution Consulting company, AltexSoft co-builds technology products to help companies accelerate growth.