How to sync files to Google Coldline Storage
Jump to navigation
Jump to search
Creating a storage bucket
- Open the Cloud Storage browser:
- Click on "Create bucket":
- Name your bucket: Specify here the bucket name (all lowercase, numbers and “-“ or “_”)
- Note: Do not specify sensitive information on the bucket name since it is searchable via DNS
- Choose a default storage class: Select Coldline
- Location: Select a close location, such as Europe-west2 (London)
- Choose how to control access to objects: Leave the default settings
- Advanced settings: Leave the default settings (Google-managed key)
- Click on Create
Google Cloud SDK installation phase
- Login to a machine using privileged account
- Install Google Cloud SDK tools.
- Linux (Debian / Ubuntu):
- Linux (CentOS / RedHat):
- Windows:
- Run the following from command prompt to initialize the Cloud SDK:
- gcloud init --console-only
- Select a GCP project from the list
- Select a default Compute region and zone
Common Google Cloud SDK CLI Commands
- Login to Google Cloud Platform:
- gcloud auth application-default login --no-launch-browser
- Note: The command prompt will show you a link – copy the link to a new browser, login with your GCP project credentials and copy the verification code from the browser to the command prompt
- List all active GCP accounts:
- gcloud auth list
- Change the active account:
- gcloud config set account <Account_Name>
- Note: Replace <Account_Name> with the target GCP account
- Lists all available GCP projects:
- gcloud projects list
- Change the GCP project:
- gcloud config set project "<Project_ID>"
- Note: Replace <Project_ID> with the target GCP project ID
Backup phase
- List all Google Cloud Storage buckets inside the current GCP project:
- gsutil ls -p “<Project_ID>”
- Note: Replace <Project_ID> with the target GCP project ID
- Run the command below (one time) to enable file versioning:
- gsutil versioning set on gs://<bucketname>
- Note: Replace <bucketname> with the target bucket name
- Run the command below to sync recursively the content of /data folder into a target bucket:
- gsutil rsync -r /data gs://<bucketname>/target_folder
- Note 1: Replace /data with the relevant source folder name
- Note 2: Replace <bucketname> with the target bucket name
- Note 3: Replace /target_folder with the relevant target folder name
References
- Quickstart: Using the gsutil tool
- rsync - Synchronize content of two buckets/directories
- Easy, cheap and secure backup with Google Cloud Platform
- Top gsutil command lines to get started on Google Cloud Storage
- Use Cases and Different Ways to get Files Into Google Cloud Storage
- Google Cloud Storage backup tutorial