Akamai NetStorage

At a glance

Capability	Support
rclone support tier	Tier 1: Core
Checksums	MD5
Modification times	Kept, to 1 s
Case-insensitive names	No
Duplicate names allowed	No
Server-side copy	No
Server-side move and rename	No
Folder move and rename	No: each file is moved in turn
Free space reporting	No
Empty folders	Yes
Change notifications	No
Uploads of unknown size	Yes
Public links	No

From rclone’s documentation

The rest of this page is adapted from rclone’s documentation for this backend (rclone v1.75.2, MIT licence). It is written for rclone’s command line: where it mentions a flag such as --netstorage-example, set the option example in the wizard’s options instead. Commands such as rclone copy aren’t part of CloudMount.

You may put subdirectories in too, e.g. remote:/path/to/dir. If you have a CP code you can use that as the folder after the domain such as <domain>/<cpcode>/<internal directories within cpcode>.

For example, this is commonly configured with or without a CP code:

• With a CP code. [your-domain-prefix]-nsu.akamaihd.net/123456/subdirectory/
• Without a CP code. [your-domain-prefix]-nsu.akamaihd.net

See all buckets

rclone lsd remote:

The initial setup for Netstorage involves getting an account and secret. Use rclone config to walk you through the setup process.

Example operations

Get started with rclone and NetStorage with these examples. For additional rclone commands, visit https://rclone.org/commands/.

See contents of a directory in your project

rclone lsd ns1:/974012/testing/

Sync the contents local with remote

rclone sync . ns1:/974012/testing/

Upload local content to remote

rclone copy notes.txt ns1:/974012/testing/

Delete content on remote

rclone delete ns1:/974012/testing/notes.txt

Move or copy content between CP codes

Your credentials must have access to two CP codes on the same remote. You can’t perform operations between different remotes.

rclone move ns1:/974012/testing/notes.txt ns1:/974450/testing2/

Features

Symlink Support

The Netstorage backend changes the rclone --links, -l behavior. When uploading, instead of creating the .rclonelink file, use the “symlink” API in order to create the corresponding symlink on the remote. The .rclonelink file will not be created, the upload will be intercepted and only the symlink file that matches the source file name with no suffix will be created on the remote.

This will effectively allow commands like copy/copyto, move/moveto and sync to upload from local to remote and download from remote to local directories with symlinks. Due to internal rclone limitations, it is not possible to upload an individual symlink file to any remote backend. You can always use the “backend symlink” command to create a symlink on the NetStorage server, refer to “symlink” section below.

Individual symlink files on the remote can be used with the commands like “cat” to print the destination name, or “delete” to delete symlink, or copy, copy/to and move/moveto to download from the remote to local. Note: individual symlink files on the remote should be specified including the suffix .rclonelink.

Note: No file with the suffix .rclonelink should ever exist on the server since it is not possible to actually upload/create a file with .rclonelink suffix with rclone, it can only exist if it is manually created through a non-rclone method on the remote.

Implicit vs. Explicit Directories

With NetStorage, directories can exist in one of two forms:

1. Explicit Directory. This is an actual, physical directory that you have created in a storage group.
2. Implicit Directory. This refers to a directory within a path that has not been physically created. For example, during upload of a file, nonexistent subdirectories can be specified in the target path. NetStorage creates these as “implicit.” While the directories aren’t physically created, they exist implicitly and the noted path is connected with the uploaded file.

Rclone will intercept all file uploads and mkdir commands for the NetStorage remote and will explicitly issue the mkdir command for each directory in the uploading path. This will help with the interoperability with the other Akamai services such as SFTP and the Content Management Shell (CMShell). Rclone will not guarantee correctness of operations with implicit directories which might have been created as a result of using an upload API directly.

--fast-list / ListR support

NetStorage remote supports the ListR feature by using the “list” NetStorage API action to return a lexicographical list of all objects within the specified CP code, recursing into subdirectories as they’re encountered.

• Rclone will use the ListR method for some commands by default. Commands such as lsf -R will use ListR by default. To disable this, include the --disable listR option to use the non-recursive method of listing objects.

• Rclone will not use the ListR method for some commands. Commands such as sync don’t use ListR by default. To force using the ListR method, include the --fast-list option.

There are pros and cons of using the ListR method, refer to rclone documentation. In general, the sync command over an existing deep tree on the remote will run faster with the “--fast-list” flag but with extra memory usage as a side effect. It might also result in higher CPU utilization but the whole task can be completed faster.

Note: There is a known limitation that “lsf -R” will display number of files in the directory and directory size as -1 when ListR method is used. The workaround is to pass “--disable listR” flag if these numbers are important in the output.

Purge

NetStorage remote supports the purge feature by using the “quick-delete” NetStorage API action. The quick-delete action is disabled by default for security reasons and can be enabled for the account through the Akamai portal. Rclone will first try to use quick-delete action for the purge command and if this functionality is disabled then will fall back to a standard delete method.

Note: Read the NetStorage Usage API for considerations when using “quick-delete”. In general, using quick-delete method will not delete the tree immediately and objects targeted for quick-delete may still be accessible.

Options

Standard options

The wizard asks for these when you add the drive.

host

Domain+path of NetStorage host to connect to.

Format should be <domain>/<internal folders>

Type: string · Required

account

Set the NetStorage account name

Type: string · Required

secret

Set the NetStorage account secret/G2O key for authentication.

Please choose the ‘y’ option to set your own password then enter your secret.

NB Input to this must be obscured - see rclone obscure.

Type: string · Required

Advanced options

The wizard shows these under Advanced options. Most drives work without changing any of them.

protocol

Select between HTTP or HTTPS protocol.

Most users should choose HTTPS, which is the default. HTTP is provided primarily for debugging purposes.

Type: string · Default: https

Choices:

• http: HTTP protocol
• https: HTTPS protocol

description

Description of the remote.

Type: string