Audience
Any user of the Migration App.
Context
Planning a migration is a daunting task. This article covers the step-by-step process based on the identified best practices.
Prerequisites
- Migration App can only migrate data from on-premises sources that can be accessed via a UNC path or mapped letter drive.
- Run a speed test by going to https://<yourdomain>.egnyte.com/speed-test/. This checks for the available bandwidth from the on-premises server to the cloud. It doesn't guarantee throughput speeds, but will provide a baseline. For more information on speed, see the Helpdesk article How Long Will it Take to Upload, Backup, or Download my Files?
- Ensure that the desired Users and Groups are created in the Egnyte Cloud for the domain. For more information, refer to the Helpdesk article for Importing Users and Groups.
- Log into the Egnyte domain with an Admin account. The user must ensure that they can access the entire data set on the local network share with admin level privileges.
- Review the Migration App Product Guide and the Migration App FAQs to gain a basic understanding of the tool.
- Communicate with the Users to help them understand the migration process. They can continue using the data on the Source for standard day-to-day file system activity. But it's critical that no large data sets be moved, renamed, or deleted during data migration. This type of activity can seriously endanger the success of the migration and the integrity of the final data set. It is also imperative that no users be allowed access to Egnyte until the entire migration and cutover are complete.
Planning Your Migration Project
Data migration requires some initial planning and analysis. It’s important that the user understand and plan the strategy around some of the fundamental limitations on Egnyte that could impact your migration project.
- Egnyte does not permit more than 50K immediate children in any folder. If possible, the user should address this before initiating your migration, but Migration App will alert and interrupt the migration if this limit is exceeded.
- Some characters in file and folder names are unsupported by Egnyte, but Migration App offers two ways to mitigate this. Users can elect to “sanitize” such characters on the source before migrating, or on-the-fly during migration. See Unsupported Characters and File Types.
- As a general rule, individual migration jobs should not exceed 10TB or 15 million objects, so plan the project accordingly. So, any migration job that exceeds this will be interrupted. In some exceptional cases where it is not possible to break jobs down into smaller subsets, these limits can be raised. However, the overall complexity of the job is increased significantly, as is the probability of errors and problems.
- The Egnyte Cloud Platform currently cannot allow individual files of more than 100GB.
- If there are many thousands (or millions!) of small files to migrate, the job will take longer than if there are a smaller number of large files, even if the total amount of data is the same.
- If the user has limited bandwidth, they may plan the migrations over evenings and weekends to minimize negative impact to users on the production environment prior to cutover.
- Migration App does not migrate file types that are incompatible with Egnyte, such as TMP files, active database files, and package files. See Unsupported Characters and File Types.
Taking a Phased Approach
We recommend that customers take a phased approach when planning their migrations using the Migration App.
Phase 1:
Create Migration Jobs. When a job is created, Migration App automatically scans the Source and provides intelligence as to the size and nature of the job. This can advise whether to proceed or instead restructure the migration project before beginning any actual migration.
This scan helps identify if any unsupported characters need to be sanitized, as well as any files that will not be migrated due to incompatibility with the Egnyte Cloud platform. After the scan is complete, the user will have some basic information made available in the Migration Details page user interface. Additionally, it is recommended that the user download the Migration Report and review the information provided in the Scan register of the file.
Phase 2:
Seed the data from the Source. This is the initial Data Migration to copy data to Egnyte and will take the most time. It will also give the user visibility into whether they need to address any unsupported filenames or pathnames that are too long. Don’t worry about users making normal everyday changes to the source data; they can continue working with their content and you'll be able to true it up in a later phase. Be sure to communicate basic guidelines to the users, as described above in Before You Begin, point 6.
There will be several Data Migration Options after selecting Migrate Data from the Migration Actions dropdown button. To understand how these options are applied, refer to the following Helpdesk articles.
Important Notes about Case-Sensitivity:
- Egnyte recommends instructing users not to change the case of file or folder names on the source once the migration has begun. The Egnyte platform is not case-sensitive, so it sees 'LEGAL' and 'Legal' as the same file name. But the utility at the heart of the Migration App is case-sensitive, so it sees 'LEGAL' and 'Legal' as two different file names. This can potentially lead to data loss during Phase 3 (True-Ups.)
- If Canonical Support is enabled on user directories, it is strongly recommended to disable it, as it is not compatible with the Egnyte Cloud File System (CFS).
Phase 3:
Run True-Ups. After some time has passed since running the Migrate Data command, it’s time to True-Up or sync the data. To do this, return to the Migration Dashboard, navigate to the Migration Job in question, and select True-Up Data from the migration action menu. This generally takes much less time than the first Data Migration, as it will mirror the Source and only migrate new or changed files (sometimes called the 'deltas'). It will also provide an estimate of how much data has changed at the Source during that period, helping the user plan for the final True-Up and Cutover.
Most migration jobs involve several True-Ups before the final Cutover.
Important Note about Syncing Deletes:
Migration App syncs deleted data by default. This means that if a file exists on Egnyte but no longer exists on the Source, running a True-Up will delete the file on Egnyte.
During a True-Up, the following steps are performed:
- Scan the Source.
- Scan the Destination.
- Compare the Source and Destination to prepare to mirror the two.
- Migrate the new data.
- Delete the files that are no longer on the Source.
You can run True-Ups as many times as you like before you finally Cutover.
Phase 4:
Migrate Permissions (this is optional.) Permissions migration with Migration App doesn't recognize permissions that were previously migrated. So it's a fresh migration, whether or not the user has migrated permissions previously, and it commonly takes quite a while. Therefore, it is recommended as the final step in the migration project before Cutover. Review the Helpdesk article Permission Translation with Migration App before proceeding.
Phase 5:
Schedule Cutover. The final phase before Going Live is the “Cutover." Most customers plan it over a weekend and require users to stop using the Source data while they run the final Sync. This means running the same Migration Job True-Up one final time to capture the last changes and migrate them to Egnyte. Once it has been completed, you can announce Go Live, when all users start using Egnyte and disable the Source.
Planning for the Cutover
Imagine there are 900GB of data at the source. It's reasonable to estimate that only a small percentage will change over the course of a business day – for this example, it's assumed to be 5%, or 45GB. If the initial Data Seeding (Phase 1) took 18 hours to migrate, it can be estimated that the True-Up would take about an hour. To test this, a True-Up can be run after a day, then again after a 3- or 5-day interval. This will help gauge how much time it will take to migrate the deltas when planning the final Cutover.
Before scheduling the Cutover, remember to communicate with all content users well in advance. In addition, for this migration to be successful, users must stop using the source data permanently and switch over to the Egnyte data only after completing the final True-Up and Permissions Migration.
Finally, be aware that not all Cutovers are successful the first time. Lots of things can happen, so be sure to have a Plan B in place if it is not possible to finalize the Cutover when planned. Our Technical Support Agents are diligent and will do everything in their power to keep the process on track, but our Engineering team is not on call 24/7. If something comes up that requires Engineering intervention or troubleshooting, it may force rescheduling the Cutover. Be aware of the Terms of Use for customers using the Migration App in a Self-Service manner.
Questions? Feature Requests? Other Feedback?
If there are any feature suggestions or requests, feel free to submit them here, and make sure you mention Migration App in the text.
For more complex requests that would benefit from providing screenshots or other attachments, submit them to support@egnyte.com. Be sure to include Migration App in the email title. Our Support team will push your comments to the Product team for consideration.