Does Cloud Files support the transfer of large files?
Yes, the Rackspace Cloud now supports the transfer and storage of larger files. Following is a list of frequently asked questions about our large file support.
How does Rackspace support the upload of large files to Cloud Files?
Although support for uploading content to Cloud Files through the Cloud Control Panel is limited to files smaller than 5 GB, we can accommodate the transfer of files larger than 5 GB by allowing you to segment your files into multiple file segments.
How large should my file segments be?
Rackspace does not enforce any lower limits on the file size. File segments cannot be larger than 5 GB, and we recommend not storing file segments that are smaller than 100 MB.
Can I serve my large files over the CDN?
At this time, you cannot serve files larger than 10 GB from the CDN.
Is there a simpler way to use this process?
We have created a tool called Swift to make this process easier. Swift segments your large file for you, creates a manifest file, and uploads the segments accordingly. After it uploads the segments, Swift manages the segments for you, deleting and updating them as needed. You can get information about the Swift Tool and download the Swift tool.
When should I use the API instead of the Swift tool?
If you are interested in developing against the Rackspace Large File Support code to incorporate into your application, you should work directly with the Cloud Files API. Use the following steps:
- Upload the segments:
curl -X PUT -H 'X-Auth-Token: <token>' \ http://<storage_url>/container/myobject/1 --data-binary '1' curl -X PUT -H 'X-Auth-Token: <token>' \ http://<storage_url>/container/myobject/2 --data-binary '2' curl -X PUT -H 'X-Auth-Token: <token>' \ http://<storage_url>/container/myobject/3 --data-binary '3'
- Create the manifest file:
curl -X PUT -H 'X-Auth-Token: <token>' \ -H 'X-Object-Manifest: container/myobject/' \ http://<storage_url>/container/myobject --data-binary ''
- Download the segments as a single object:
curl -H 'X-Auth-Token: <token>' \ http://<storage_url>/container/myobject
When should I use the Swift tool instead of the API, and what is the process?
If you want to upload large files but do not want to incorporate our code into an application, you might find it easier to use the Swift tool for your uploads and management. If you are using the tool, the process looks as follows:
The following code uploads largefile.iso to test_container in 10 segments (in this case, 10 MB per segment for a total of 100 MB).
$ st -A https://auth.api.rackspacecloud.com/v1.0 -U glh -K 3a25c2dc74f24c3122407a26566093c8 upload -S 1048576 test_container largefile.iso largefile.iso segment01 largefile.iso segment06 largefile.iso segment09 largefile.iso segment04 largefile.iso segment05 largefile.iso segment03 largefile.iso segment07 largefile.iso segment08 largefile.iso segment02 largefile.iso segment00 largefile.iso
The following code downloads the large file as a single object:
$ st -A https://auth.api.rackspacecloud.com/v1.0 -U glh -K 3a25c2dc74f24c3122407a26566093c8 download test_container largefile.iso largefile.iso
Alternatively, you can list the segments and download them separately:
$ st -A https://auth.api.rackspacecloud.com/v1.0 -U glh -K 3a25c2dc74f24c3122407a26566093c8 list test_container_segments largefile.iso/1300906481.0/10485760/00000000 largefile.iso/1300906481.0/10485760/00000001 largefile.iso/1300906481.0/10485760/00000002 largefile.iso/1300906481.0/10485760/00000003 largefile.iso/1300906481.0/10485760/00000004 largefile.iso/1300906481.0/10485760/00000005 largefile.iso/1300906481.0/10485760/00000006 largefile.iso/1300906481.0/10485760/00000007 largefile.iso/1300906481.0/10485760/00000008 largefile.iso/1300906481.0/10485760/00000009
You can list the main large object itself:
$ st -A https://auth.api.rackspacecloud.com/v1.0 -U glh -K 3a25c2dc74f24c3122407a26566093c8 list test_container largefile.iso
When you use the object name to list large object segments, the segment names are returned in alphabetical order. Plan segment names accordingly. For example, name10 would come before name9 alphabetically, so to preserve the correct order, the latter segment should be named name09.
What will the download experience be like?
After files are segmented and uploaded with a manifest file, your large file will be served as a single file, so the experience will mimic the download or service of any other object retrieval.
Do I have access to my file segments?
Yes, you can edit your file segments just like any other object within Cloud Files.
How do I ensure that my files are linked correctly?
Include your manifest file in your upload. You can change your file name by editing this manifest file as well. We recommend using prefixing in your file segments to easily map your manifest file to the portions of your large file. For example, you could name your segments as follows:
Myfavoritemovie-01 Myfavoritemovie-02 Myfavoritemovie-03 ..etc..
In this case, you would point your manifest file to the prefix:
Can I use this feature from the Cloud Control Panel?
At this time, Rackspace has not implemented this functionality into the Rackspace Cloud Control Panel.
© 2015 Rackspace US, Inc.
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License
See license specifics and DISCLAIMER