The Migration app includes a custom file exclusion feature that allows users to exclude specific files from migration using three methods: Extension-Based Exclusion, Glob Pattern-Based Exclusion, and Regex Pattern-Based Exclusion.
This feature provides flexibility in controlling which files are migrated from the source to the destination.
Additionally, exclusion rules allow the users to:
- Reduce migration time by excluding unnecessary files
- Avoid migrating files that may cause compatibility issues
- Maintain control over what data gets transferred during migration
- Apply complex filtering rules using patterns instead of manually specifying each file
Access Exclusion Rules Option
Navigate to the Migration job details -> Advanced options -> Exclusion Rules
Types of Exclusion Rules
Extension-Based Exclusion
Users can specify file types to be excluded by entering the undesired extensions in a single text box. This is the simplest type of file exclusion and may be used by users with any experience level.
Users can specify multiple file types in a comma-separated manner. Click on Save to apply the exclusion.
Glob Pattern-Based Exclusion
Users can exclude files based on glob patterns. The UI provides a separate text box for each pattern entry, and these entries are combined into a string array and validated before forwarding them to the agent. If a pattern cannot be validated, the UI will display an error.
Each glob pattern requires a separate text box. Click on + Add pattern to add a pattern. To delete a pattern, click on the Delete icon next to the pattern. Click on Save to apply the exclusion(s). The example below has two patterns added.
- *.exe - Excludes all .exe files
- ??BEX*.msi - Excludes .msi files with specific naming patterns
• Use {*.txt} instead of {{*.txt}} (nested curly braces are not supported)
• Use {*.\d\d\d} instead of *.\d\d\d for patterns with digits
• Use {*.jpg,*.png} instead of *.{jpg,png} for multiple extensions
Regex Pattern-Based Exclusion
Users can also exclude files based on regex patterns. Similar to the glob pattern-based exclusion, the UI provides a separate text box for each entry that is then combined into a string array, validated, and forwarded to the agent. If a pattern cannot be validated, the UI will display an error. Click on + Add pattern to add a pattern. To delete a pattern, click on the Delete icon next to the pattern. Click on Save to apply the exclusion(s). The example below has two patterns added.
- ^FF[0-9A-Za-z] - Excludes files starting with "FF" followed by alphanumeric characters
- ^ES[A-Z] - Excludes files starting with "ES" followed by uppercase letters
• Only POSIX/PCRE-based regex patterns supported by rclone are accepted.
• If the user wants to exclude the contents of a specific folder, they can add the folder names to the regex. For instance, to exclude the contents of a folder named Inheritance Only, enter that followed by a ‘/**’ as a regex pattern.
When the user saves exclusion rules, Egnyte validates all glob and regex patterns. For extension-based exclusions, Egnyte automatically appends *.{extension} to each entry. All patterns are then merged and validated before being applied.
Review and Verify Exclusion Rules
The count of exclusions rules are always displayed in the sidebar overview.
Users can execute a scan to verify that only the intended files are excluded. Review the default exclusion list files along with the custom exclusions.
Possible Errors and Resolutions
Invalid Patterns
Users might receive an error response with code 634 indicating Patterns Invalid.
Error Response Format:
{
"code": 634,
"fieldErrors": {
"glob": {
"code": 635,
"fields": [0, 1]
},
"regex": {
"code": 636,
"fields": [2, 4]
}
},
"message": "Patterns Invalid"
}
The fields array indicates which pattern indices have errors. In the example above, glob patterns at positions 0 and 1 are invalid, and regex patterns at positions 2 and 4 are invalid.
Steps to Resolve the error:
- Review the patterns at the indicated indices
- Common issues:
- Nested curly braces: Use {*.txt} not {{*.txt}}
- Missing curly braces for certain patterns: Use {*.\d\d\d} not *.\d\d\d
- Invalid regex syntax: Ensure your regex follows POSIX/PCRE standards
- Incorrect glob format: Use {*.jpg,*.png} not *.{jpg,png}
- Correct the invalid patterns and save again
- If a pattern appears valid but fails validation, try wrapping it in curly braces {}
Pattern Not Matching Expected Files
The error would occur if the exclusion rule is accepted but doesn't exclude the expected files.
Resolution:
1. Verify the pattern syntax matches rclone's filtering requirements
2. For extension matching: Use *.jpg and *.jpeg separately instead of *.jpe?g (which matches jpeXg where X is any character)
3. Test with simpler patterns first, then build complexity
4. Refer to rclone documentation:
o Glob patterns: Rclone Filtering
o Regex patterns: Rclone Filtering