Advanced Options may be customized directly from the User Interface for each individual migration job. But this can be tedious and time consuming if there are multiple jobs tied to a single Agent and all of them are required have the same settings. Using a Custom Config file, the user can control settings at the Agent level, which will then be applied to every job created from that Agent. Users can also use the Custom Config to address individual job exceptions
Admins have the ability to upload the custom configuration file for the agents they created and for the shared agents as well.
Possible Parameters to Configure Using Configuration File
- Skip Scan Period: Provides the option to skip re-scanning the source if a recent scan was already completed
- Exclusion Rules: Allows choosing which files or folders should not be migrated (for example, temporary files, logs, or specific folders)
- Confluence Mode: Allows multiple source locations to merge into the same destination folder in Egnyte. This is useful for combining data, but may overwrite duplicate files
- Scheduler: Automatically runs migration jobs at a scheduled date and time, or on a recurring basis (For example, daily True-Ups)
- Transfers (Upload Threads): Controls how many files are uploaded at the same time. More transfers can translate to potentially faster migration if system resources allow
- Checkers (Scan Threads): Controls the number of background process checks and verifies files before migration. This helps prepare and validate data efficiently
- Symbolic Links (Symlink Handling): Determines whether the migration should follow and copy data from shortcuts (symbolic links)
- Email Recipients: Adds additional email addresses to receive migration status updates
- Sync Deletes: Ensures that if files are deleted in the source, they are also deleted in the destination during a True-Up process
- Sanitize: Automatically cleans up unsupported or invalid file names so they can be successfully migrated
Customizing the Agent Configuration
- From the Migration Dashboard, click Settings to navigate to the Settings page.
- From the Settings page, select the Agent that needs to be customized.
This opens the settings panel for that Agent. - Click on Download Default Configuration File.
- Open the Configuration.json file in a text or code editor like Atom or Notepad++. The default configuration file looks like this:
Modifying the Agent Configuration
Edit the configuration file using JSON syntax.
- Specify a configName within the empty brackets. In the example below, it has been specified as LightConfig.
- The first section controls the Agent. The user can change Transfers and Checkers, but this will change it for all jobs associated with this Agent, so they need to be mindful of how much actual compute is available on the host machine.
- Anything specified under the config parameter will apply to all jobs associated with this agent.
In the example below, only the Agent configuration has been changed. It keeps the Agent at four Transfers, eight Checkers, but now Confluence and Symbolic Links are both enabled. - Once the configuration changes have been made, upload the custom config file to Migration App by dragging and dropping it on the Settings page.
Any future migration operations associated with this agent will reflect this custom configuration.
Modifying or Creating Individual Migration Job Configurations
Users can also specify unique settings for individual jobs – or exceptions to the Agent settings – by calling them out under migrations.
- Replace the “null” reference with brackets and follow the syntax provided below.
- For “name”, enter the exact migration name
- For “id”, copy the Migration Job ID from the Migration Details page.
-
Specify the unique exceptions for that job for Transfers, Checkers, Confluence and/or Symbolic Links.
In the example below, Job A and Job B have unique configurations, but any other jobs associated with Agent will follow the configuration specified for the Agent.
The migration job and scheduler options are summarized in the table belowField
Valid Field Value
Impact On Upload
Migration Config
Scheduler Config
transfers Between 1 to 20 Not Applicable Updates transfers
Result – Improves Migration speed
checkers Between 1 to 20 Not Applicable Updates checkers
Result – Improves Migration speed
min_diskspace_required Minimum default is 10 Not Applicable Not Applicable confluence Boolean value - true or false Not Applicable Sets confluence mode for migration job
Result - allows creating and performing copy/sync for conflicting destination job if confluence is set to true
symlink Boolean value - true or false Not Applicable Sets Symlink behaviour
Result - allow symlink data upload from source if set to true
sanitize Not Applicable Between 1 to 3.
1 - Never
2 - On the Fly
3 - On the Source
If value selected for sanitize is 2 then sanitize option will be set to “Always On Fly”and it cannot be changed further.
Result - If sanitize on fly/source is added then invalid file/folders will be sanitized and allowed to migrate.
emptyFolders Not Applicable Boolean value - true or false Sets the preference for empty folder migration.
Result - if set to true then allows performing empty folder operations
skipScanPeriod Integer Value: 0 to 24 Not Applicable Updates skip scan period
Result - If Migration or true-up is executed with the specified period then scan will be skipped. Else the scan will run as part of Migration or true-up
exclusionRules Refer to using exclusion rules section Not Applicable Result - files and folders that match the exclusion rule will be skipped during upload
startingFrom Not Applicable UTC time formatted as YYYY-MM-DDTHH:MM:SS Set the start date and time for a Migration job frequency Not Applicable Integer Value: 1, 3, 5, 7, 14, 30 Sets Interval between jobs in number of days if job is periodic times Not Applicable Any Integer Value Number of times job should run.
1 - for one time job
cleanfutureTimestamp Not Applicable Boolean value - true or false sets behaviour for future timestamps.
Result - Changes future timestamp to present if set to true
syncDeletes Not Applicable Boolean value - true or false sets behaviour for syncing deletes
Result - If set to true will delete files in destination that have been deleted on the source
allowInterruptionForSyncDeletes Not Applicable Boolean value - true or false Sets the behaviour for interrupting true-up when they are a lot of deletes on source
Result – will stop the true-up job if there are a lot of deletes on source allowing users to decide how to proceed.
emailrecipients Null (default)
Email addresses of the recipients
Not Applicable E-mail address updated for notifications.
Result - Email updates are shared with the recipients added.
-
Once the configuration changes have been made, upload the custom config file to Migration App by dragging and dropping it on the Settings page.
Any future migration operations associated with this agent and Jobs A and B will reflect this custom configuration.Power users do not have the ability to upload, download, or modify the configuration file and will not see these options.
To create a new migration job using the configuration file, copy the configuration of any existing job:
- For name, update the migration name
- Leave the migration_id field empty
- Update src-path (Source Path) and dst-path (Destination Path)
- Either use / instead of \ or use \\ instead of \