gsutil is a Python app developed by Google that lets you handle the GCP Cloud Storage service, anything from uploading, downloading, sync, mostly doing backup and migration operation. The other recommended CLI tool is gcloud storage. Let’s see how we could backup a Linux subdirectory to a GCP bucket while preserving the directory structure, using gsutil and gcloud storage.
Prerequisites
- GCP account
- gsutil CLI
- gcloud CLI
Solution
gsutil
gsutil -m cp -r -d /some/directory gs://some-bucket/
m: enables parallel (multithreaded) transfers. Faster uploads, in a way.r: recursive.d: directory path.
If you wish to exclude certain file extensions, for instance .jpg, run the following command:
gsutil -m cp -r -d -x "*.jpg" /some/directory gs://some-bucket/
But, if this doesn’t do it, alternately try using the gsutil rsync command. For instance:
gsutil rsync -r /some/directory gs://some-bucket/
Exclude jpg file extensions:
gsutil -m rsync -r -x '^(?!.*\.jpg$).*' /some/directory gs://some-bucket/
gcloud storage
I would prefer gcloud storage over gsutil any day. Overall, gcloud CLI provides a wider range of functionalities though, and it seems more simple and easier to use.
Example:
gcloud storage cp -r /some/directory gs://some-bucket
Note: There is no forward slash (/) in gs://some-bucket tho. Not adding a / makes a difference in terms of subdir replication.
Conclusion
If you get stuck at some step, feel free to leave a comment below and if you find this tutorial useful, follow our official channel on Telegram.