Sneakernet: Google Transfer Appliance
Before apps were available in the cloud, or delivered through virtual desktops, IT pro’s would go from desk to desk, sometimes after hours to install a new app for a user or move data off their desktop over to a new system. We had our trusty portable drive and sometimes USB stick but above all we made sure to wear our favorite sneakers.
Have times changed? well yes, but sneakernet has returned to the data center – with Google Transfer Appliance except this time we are dealing with more data and larger computers – the cloud.
Rackable high-capacity storage server
It can get costly and time-consuming (Bandwidth) when uploading data to your cloud provider. One way to speed up cloud migrations is using an appliance like Google Transfer Appliance. It’s a rackable high-capacity storage server that you set up in your data center.
If you are moving more than 20TB of data, or data that would take more than a week to upload – this is the preferred method to get the data from your data center to Google’s cloud.
To give you a sense of upload time based on network speed and dataset size for a typical enterprise, Google put together this graphic and I think its a good indicator of what to expect.
Upload Times with Google Transfer Appliance
Currently, you have two options when requesting an appliance, the two configurations are: 100TB or 480TB of raw storage capacity, or up to 200TB or 1PB compressed. If you have more than 500TB, one thing to consider is that multiple Transfer Appliances will be sent to you.
The 100TB appliance fits into a standard 19” rack, and is 2U in size, while the 480TB model is 4U and is a non-rackable unit case in a shipping crate.
After you’ve received the appliance and have it racked and networked, the steps to move your data are basic and straightforward. You capture the data using different options: Workstation, NFS capture or NFS export. Your data is encrypted with industry standard AES256 encryption.
Once you’ve moved your data, you send the appliance back to Google, they will upload your data to the cloud, and finally, you log into your console to decrypt your data and choose a storage bucket.
Some sample use cases
- Migrate your data center (or parts of it) to the cloud.
- Kick-start your ML or analytics project by transferring test data and staging it quickly.
- Move large archives of content like creative libraries, videos, images, regulatory or backup data to Cloud Storage.
- Collect data from research bodies or data providers and move it to Google Cloud for analysis.
One cool use case Google highlighted is Schmidt Ocean Institute who uses the Transfer Appliance to safely get the data back to shore (from ships out in the ocean) and make it publicly available to the research community as quickly as possible.
Shift your backup workflow
One other way to use the Transfer Appliance is to shift your backup workflow to the cloud quickly. You can move the bulk of your current backup data offline using the Transfer Appliance, and then incrementally back up to GCP over the network from there. I understand that partners like Commvault can help you do this.
Google Transfer Appliance is generally available in the U.S. currently, but if you are outside the U.S. you can fill out this forum and help Google decide where they should deploy next.
How did you manage to transfer data to your cloud provider? Use Google Cloud and have a different solution, let me know. If you’ve used the Google Transfer Appliance, share your thoughts and let me know on Twitter.