Import media files

Media can be imported to the VIP File System with VIP-CLI. This is especially useful for the process of migrating a WordPress site to the VIP Platform.

Limitations

• The maximum size of the entire archive that holds the files to be imported is 200 GB.
• The maximum size for an individual file is 4 GB.
• Only one media import per environment can be run at a single time. Attempts to start a second import while a previous import is still in progress will result in an error.
• Files imported with VIP-CLI will not appear in a site’s WordPress media library unless the attachment data for the files exists in the database as a result of a database import or a data sync (for non-production environments).
• Media should only be imported to a production environment. Media files that are imported to a WordPress production environment are automatically shared with and available to associated non-production environments. For this reason, importing media to non-production environments is generally unnecessary.
• By default, media imports do not overwrite existing files that share the same name and file path. Including the --overwriteExistingFiles option (or -o ) when importing media files will override this behavior and force the silent overwriting of an existing file.
• Media files that are imported to a WordPress application without an associated database record cannot be deleted without assistance. To delete files without a database entry, contact VIP Support.

Requirements

• Files for import must be archived (.tar.gz, .tgz, or .zip).
• The archived file must decompress to a directory named uploads, and only contain the media required for a site(s).
• To import an archived file from a remote URL, the archived file must be hosted at a publicly accessible URL (e.g. https://www.example.com/uploads.tar.gz).
  • File-sharing services such as Dropbox, Box, or WeTransfer are able to provide publicly accessible URLs.
  • A downloadable Dropbox URL must include a ?dl=1 query parameter.
  • Google Drive URLs are not supported.

If the archived file for import cannot be reformatted or restructured to meet the above requirements, create a VIP Support ticket requesting assistance with the import and include a downloadable link for the archived file.

Directory structure

The directory to be imported must be named uploads.

In most cases, media uploaded to a WordPress install is stored in a directory named uploads/, which is a subdirectory of wp-content/. To prepare media files for import to VIP, an archived copy of the uploads/ directory is recommended.

If a non-VIP WordPress site has customized the name of the uploads/ directory, extra steps will be needed to duplicate the directory to a working location and re-name the duplicate directory to uploads before importing the media to VIP.

When an archive’s directory structure contains a directory named uploads/, only the files within the uploads/ directory will be imported. All other files will be ignored.

If no directories named uploads/ exist within an archive, the import tool will import the contents of the archive to an environment’s uploads/ directory with the archive’s existing directory structure.

Directory structure for multisites

Media import directories for multisites must also be named uploads.

The subdirectory structure of the uploads/ directory for sites belonging to a WordPress multisite will differ from WordPress single sites.

The uploads/ file path for the main site (ID 1) of the multisite will beuploads/, but all other sites on a multisite network will be in the uploads/sites/ directory structure, in their own subfolder based on their network site ID, designated by the database.

Upload paths for all sites other than the main site (ID 1) follow the pattern uploads/sites/{SITE ID}/{YYYY}/{MM}/{FILE NAME}. The values for {SITE ID} do not have any leading 0’s (i.e., use 2, not 02).

Unsupported file types

Imports to the VIP File System will fail for the following file types:

• The path to the file for import can be no more than 255 characters in length. This limitation includes the /wp-content/uploads/ prefix in every imported file path.
• Import will fail for files without extensions.
• Import will fail for files file paths that end with a forward-slash (i.e. /).
• Hidden files or files within hidden directories will be ignored. Hidden files will not be included in the count of files that are discovered by the vip import media command, and they will not be included in the final count of files that are successfully imported.

Special characters in file names

When files are uploaded through the WordPress media library, WordPress Core sanitizes file names and removes or replaces special characters, including: ~?,[]/\=<>:;'\&$#*()|~`!{}%+ and whitespace ().

When media files are imported to VIP, they bypass the WordPress sanitize_file_name() function, and all special characters are allowed, including: spaces ( ) [ ] ~ + & # % = ' ” \ < > : ; , / ? $ * | ` ! { } . Encoded or alternate whitespace, such as %20or + will be converted to proper spaces during the import.

Intermediate images

Traditional installations of WordPress create intermediate images; an array of smaller versions of each uploaded image. The generated intermediate files are unnecessary on VIP, as the VIP File System is able to dynamically generate and cache any requested dimension of the originally uploaded image. This reduces the amount of storage a site requires and provides image size flexibility at no loss of speed.

If needed, intermediate images can be imported by including the -i flag when running the vip import media command. When including intermediate images for an import, the process will take a longer time to complete.

SVGs

SVG files allow embedded JavaScript, which can pose a security risk. If SVGs exist in the uploads/ directory that will be imported, it is recommended to scan the file contents prior to media import.

The contents of an SVG file can be scanned by:

• Examining the contents within a file editor ( e.g. Visual Studio Code).
• A tool such as the SVG Sanitizer Test or VIP’s SVG Sanitizer.
• A local PHPCS scan using the WordPress-VIP-Go standard.

All SVG files introduced to a GitHub repository in a pull request will undergo an SVG analysis as part of the automated PHPCS scan using the WordPress-VIP-Go standard from the VIP Code Analysis Bot. Any discovered irregularities will be reported in the automated code review.

Import media files with VIP-CLI

Import media files to the VIP File System with the VIP-CLI command: vip import media [options] [command]

Example command for importing an archived file from the publicly available URL “https://www.example.com/uploads.tar.gz”:

vip @example-app.production import media "https://www.example.com/uploads.tar.gz"

During the import process, the CLI will display the real-time progress of the import. If desired, the real-time progress of an import can be hidden using the Ctrl + C key combination.

When all files in the archive have been processed, a status report for the import will be created within the current working directory. The exact path to this status report will be shown on screen.

Overwrite existing files during import

Including the --overwriteExistingFiles option (or -o) during import will force the silent overwriting of existing files.

For example:

vip @example-app.production import media "https://www.example.com/uploads.tar.gz" --overwriteExistingFiles

Retrieve error logs

If one or more files fail to import, an error log will be generated and able to be retrieved at the end of an import. Error logs include a list of failed files and the reasons for their failure.

By default, import logs are created in .txt format. Add the --exportFileErrorsToJson option (or -E) to write the error logs to a .json file instead.

vip @example-app.production import media "https://www.example.com/uploads.tar.gz" -E

An error log file can be dowloaded within 15 minutes of the completion of the import process from a URL provided in message output in the terminal. The file can be accessed from a new URL by running the vip import media status command. The file will remain available for up to 7 days after the import or until a new media import is performed.

Abort an import

VIP-CLI command: vip import media abort [options]

Abort an import process in progress. All files that were imported prior to executing the abort command cannot be deleted.

vip @example-app.production import media abort

Check the status of a media import

The status of the current or most recent media file import can be checked with the vip import media status VIP-CLI command.

• If an import is currently running, the output will be for the currently running import, and the status command will continually check and report back on the status of the import process until it is in a finished state (COMPLETED, ABORTED, or FAILED).
• If an import is not currently running, the output will be for the import that ran most recently.

Example output of the vip import media status command:

vip @example-app.production import media status
=============================================================
Checking the Media import status for your environment...
Imported Files: 95/100 - 95% ✓
=============================================================
Status: COMPLETED ✓ : The imported files should be visible on your App
App: mytestsite (PRODUCTION)
=============================================================

⚠️  5 file error(s) have been extracted

All errors have been exported to /Users/myuser/media-import-mytestsite-1642371503508.txt

The Imported Files: field displays the number and percentage of files that were successfully imported.

If any files failed to import, a warning (⚠️) message will appear in the output. The associated errors will be exported to an error log for review.

The Status: field will display the overall import status with the following possible states:

**The import can be in a FAILED status for a variety of reasons. Review status output, error logs, limitations, and the import checklist before attempting another media import.