15. Migration & Transfer

Explore

AWS Migration Tools

DataSync

AWS DataSync is an online data movement and discovery service that simplifies data migration and helps you quickly, easily, and securely transfer your file or object data to, from, and between AWS storage services.

⁠

On-premises storage transfers

DataSync works with the following on-premises storage systems:

⁠

Network File System (NFS)⁠

⁠

Server Message Block (SMB)⁠

⁠

Hadoop Distributed File Systems (HDFS)⁠

⁠

Object storage⁠

⁠

AWS storage transfers

DataSync works with the following AWS storage services:

⁠

Amazon S3⁠

⁠

Amazon EFS⁠

⁠

Amazon FSx for Windows File Server⁠

⁠

Amazon FSx for Lustre⁠

⁠

Amazon FSx for OpenZFS⁠

⁠

Amazon FSx for NetApp ONTAP⁠

⁠

Other cloud storage transfers

DataSync works with the following other cloud storage services:

⁠

Google Cloud Storage⁠

⁠

Microsoft Azure Blob Storage⁠

⁠

Microsoft Azure Files⁠

⁠

Wasabi Cloud Storage⁠

⁠

DigitalOcean Spaces⁠

⁠

Oracle Cloud Infrastructure Object Storage⁠

⁠

Cloudflare R2 Storage⁠

⁠

Backblaze B2 Cloud Storage⁠

⁠

NAVER Cloud Object Storage⁠

⁠

Alibaba Cloud Object Storage Service⁠

⁠

IBM Cloud Object Storage⁠

⁠

Seagate Lyve Cloud⁠

⁠

Edge storage transfers

DataSync works with the following edge storage services and devices:

⁠

Amazon S3 compatible storage on AWS Snowball Edge⁠

⁠

AWS Snowcone⁠

⁠

Here's an overview of its key features:

Data Migration:

Facilitates the migration of active datasets to AWS Storage services, allowing organizations to seamlessly transition their data to the cloud.

Supports various use cases including migrating data for analysis and processing, archiving data to free up on-premises storage capacity, and replicating data for business continuity.

Built-in Security:

Provides built-in security capabilities to ensure the confidentiality and integrity of transferred data.

Encrypts data in-transit to safeguard it from unauthorized access or interception.

Verifies data integrity both in-transit and at-rest, enhancing the overall security posture of data transfers.

Optimized Bandwidth Usage:

Optimizes the utilization of network bandwidth during data transfers, maximizing transfer speeds while minimizing network congestion.

Dynamically adjusts transfer rates based on available bandwidth to ensure efficient utilization of network resources.

Resilience and Fault Tolerance:

Automatically recovers from network connectivity failures or interruptions, ensuring the reliability and resilience of data transfer operations.

Minimizes the impact of network disruptions on data transfer processes, thereby improving overall reliability.

Control and Monitoring:

Offers comprehensive control and monitoring capabilities to manage data transfer operations effectively.

Allows users to schedule data transfers according to predefined schedules, enabling efficient resource allocation and workload management.

Provides granular visibility into the transfer process through Amazon CloudWatch metrics, logs, and events, enabling real-time monitoring and performance optimization.

Supported Data Sources and Destinations:

Supports a wide range of data sources and destinations, including Network File System (NFS) shares, Server Message Block (SMB) shares, Hadoop Distributed File Systems (HDFS), self-managed object storage, AWS Snowcone, Amazon S3 buckets, Amazon EFS file systems, and Amazon FSx for Windows File Server file systems.

Offers flexibility and versatility in transferring data between different storage platforms and AWS services, accommodating diverse data storage environments and architectures.

⁠

AWS DataSync simplifies and streamlines the process of transferring data between on-premises storage systems and AWS Storage services, providing robust security, optimized performance, and comprehensive control and monitoring capabilities. With support for various data sources and destinations, DataSync offers flexibility and scalability to meet the evolving data transfer needs of organizations.

Use cases

These are some of the main use cases for DataSync:

Discover data – Get visibility into your on-premises storage performance and utilization. AWS DataSync Discovery can also provide recommendations for migrating your data to AWS storage services.

Migrate data – Move active datasets rapidly over the network into AWS storage services. DataSync includes automatic encryption and data integrity validation to help make sure that your data arrives securely, intact, and ready to use.

Archive cold data – Move cold data stored in on-premises storage directly to durable and secure long-term storage classes such as S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive. Doing so can free up on-premises storage capacity and shut down legacy systems.

Replicate data – Copy data into any Amazon S3 storage class, choosing the most cost-effective storage class for your needs. You can also send data to Amazon EFS, FSx for Windows File Server, FSx for Lustre, or FSx for OpenZFS for a standby file system.

Move data for timely in-cloud processing – Move data in or out of AWS for processing. This approach can speed up critical hybrid cloud workflows across many industries. These include machine learning in the life-sciences industry, video production in media and entertainment, big-data analytics in financial services, and seismic research in the oil and gas industry.

Benefits

By using DataSync, you can get the following benefits:

Simplify migration planning – With automated data collection and recommendations, DataSync Discovery can minimize the time, effort, and costs associated with planning your data migrations to AWS. You can use recommendations to inform your budget planning and re-run discovery jobs to validate your assumptions as you approach your migration.

Automate data movement – DataSync makes it easier to move data over the network between storage systems and services. DataSync automates both the management of data-transfer processes and the infrastructure required for high performance and secure data transfer.

Transfer data securely – DataSync provides end-to-end security, including encryption and integrity validation, to help ensure that your data arrives securely, intact, and ready to use. DataSync accesses your AWS storage through built-in AWS security mechanisms, such as AWS Identity and Access Management (IAM) roles. It also supports virtual private cloud (VPC) endpoints, giving you the option to transfer data without traversing the public internet and further increasing the security of data copied online.

Move data faster – DataSync uses a purpose-built network protocol and a parallel, multi-threaded architecture to accelerate your transfers. This approach speeds up migrations, recurring data-processing workflows for analytics and machine learning, and data-protection processes.

Reduce operational costs – Move data cost-effectively with the flat, per-gigabyte pricing of DataSync. Avoid having to write and maintain custom scripts or use costly commercial transfer tools.

⁠

Loading docs.aws.amazon.com⁠

⁠

On-premises storage transfers

AWS storage transfers

Other cloud storage transfers

Edge storage transfers

Use cases

Benefits

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.