Migrating GDrive Data with Rclone

Copying Shared Folders from Old GDrive to New GDrive

Requirements:

Setting up Rclone with both new and old GDrive accounts
New GDrive accounts recommended to have edit access
- Having only view access would prevent transfer of files where viewers are prevented from download, print, and copy

Command to use

1	rclone copy SOURCE_GDRIVE_NAME,shared_with_me:"SOURCE_FOLDER_NAME" DESTINATION_GDRIVE_NAME:"DESTINATION_FOLDER_NAME" --server-side-across-configs -vP

Where,
SOURCE_GDRIVE_NAME: The name for the source GDrive when Setting up Rclone
DESTINATION_GDRIVE_NAME: The name for the destination GDrive when Setting up Rclone

Pitfall Encountered:

Rclone should resume the migration just by running the same command line. But mine failed at first. I’ve identified that using --drive-shared-with-me flag at the end of the command is the culprit.

Using the flag at the end of the line makes it a global scope, forcing the destination drive also looked for a Shared Drive named DESTINATION_FOLDER_NAME, which of course it isn’t a Shared Drive to the new GDrive, hence it will re-migrate everything again, causing a duplication.

The erroneous command is as follow. (DO NOT USE)

1	rclone copy SOURCE_GDRIVE_NAME:"SOURCE_FOLDER_NAME" DESTINATION_GDRIVE_NAME:"DESTINATION_FOLDER_NAME" --drive-shared-with-me --server-side-across-configs -vP

Migrating Several Folders

While using & or && operator could definitely chain all the commands, this method is stupidly inelegant, and is a mess to read, and thus might cause human error.

rclone copy SOURCE_GDRIVE_NAME,shared_with_me:"SOURCE_FOLDER_NAME_1" DESTINATION_GDRIVE_NAME:"DESTINATION_FOLDER_NAME_1" --server-side-across-configs -vP && rclone copy SOURCE_GDRIVE_NAME,shared_with_me:"SOURCE_FOLDER_NAME_2" DESTINATION_GDRIVE_NAME:"DESTINATION_FOLDER_NAME_2" --server-side-across-configs -vP && rclone copy SOURCE_GDRIVE_NAME,shared_with_me:"SOURCE_FOLDER_NAME_3" DESTINATION_GDRIVE_NAME:"DESTINATION_FOLDER_NAME_3" --server-side-across-configs -vP

Fortunately, rclone provides built in filter to let us select which files or folders for it to act on (or not to). The filter we will be looking at would be --include and --include-from.

Option 1: `—include` Flag, the Inline Method

As per documentation example,

1 2	rclone copy /vol1/A remote:A rclone copy /vol1/B remote:B

would become:

1	rclone copy /vol1 remote: --include "{A,B}/**"

This tell us that, in order to transfer multiple folders, we can use --include "{A,B,C,D}/**", where each of the A, B, C, and D are the names of the folders we would want to transfer.

The resulting command would become:

1	rclone copy SOURCE_GDRIVE_NAME,shared_with_me: DESTINATION_GDRIVE_NAME: --include "{A,B,C,D}/**" --server-side-across-configs -vP

The /** tells rclone to copy everything within the subfolders recursively.

And yes, we removed the DESTINATION_FOLDER_NAME at the end of the DESTINATION_GDRIVE_NAME. Not doing so would result in all the A, B, C, and D folders to reside inside DESTINATION_FOLDER_NAME, as follow:

My Drive
|-- DESTINATION_FOLDER_NAME
    |-- A
    |-- B
    |-- C
    |-- D

Option 2: `—include-from` Flag, Read-from-a-Text-File Method

Alternatively, we could list down the folders we wanted to copy in a text file, and ask rclone to look at the text file and copy everything listed inside it.

First, we make a text file, says, source.txt, of course, at the directory where we run our CMD (ie: Command Prompt). For those new to using CMD, when we first opened our CMD, we will see the following text:

Microsoft Windows [Version 10.0.12345.6789]
(c) Microsoft Corporation. All rights reserved.

C:\Users\User>

The User part might change depending on our user name, so it might be C:\Users\James or C:\Users\Jessie

The text file will be located in this folder. So navigate to this folder and create a text file named, says, source.txt

source.txt

A/**
B/**
C/**
D/**

Yes, we will have to append /** at the end of every lines, but this method allows better readability. This is also the preferred choice if we are getting the the list in the form of text file or spreadsheet.

Then, we would run the following command:

1	rclone copy SOURCE_GDRIVE_NAME,shared_with_me: DESTINATION_GDRIVE_NAME: --include-from source.txt --server-side-across-configs -vP

Similarly, we removed the DESTINATION_FOLDER_NAME for the reason discussed earlier.

Easter Egg
On Windows, type in this command would change our CMD font color from white to green, giving us the hacker vibe.

color 0a

Rclone to Local Folder

In order to rclone to local folder, the instruction is similar to migrating fromm one GDrive to another.

We replace the second destination with C:\path\to\local\folder", and removed the --server-side-across-configs

1	rclone copy SOURCE_GDRIVE_NAME,shared_with_me:"SOURCE_FOLDER_NAME" "C:\path\to\local\folder" -vP