• Sales: 1-800-961-2888
  • Support: 1-800-961-4454

Mounting Rackspace Cloud Files to Linux Using CloudFuse


NOTE: This article is no longer updated, and the steps listed don't work completely with all current distributions. It is preserved for archival purposes. There are discussions of using CloudFuse in the comments here and on a blog post by the article's author. You can also get the latest code for CloudFuse from the CloudFuse github repository.

Please do not contact support for assistance with setting up CloudFuse.


Contents

Mounting Rackspace Cloud Files to Linux Using CloudFuse

This article shows how to mount Cloud Files as a directory on an Ubuntu 10.10 Cloud Server using CloudFuse software. This will allow you to access your Cloud Files container data inside a Linux server, just like any other file or folder.

Note: This process will give you easy access to your Cloud Files data, but in no way are we recommending that you use the mounted Cloud Files container as a location for any database/application. Running an application directly from the container would be exceptionally slow.

So, why would one want to mount a Cloud Files container to your Linux Cloud Server? Well, there are plenty of uses for having system-level access to your Cloud Files. For instance, if you have some scripts which create database or website backups, those scripts can create backups automatically into Cloud Files without you needing to use the API or File Manager in the Control Panel.

Note: The following commands are tried and tested on Ubuntu 10.10, but they should be applicable to other versions of Ubuntu or Debian. As long as you install the required packages, you should be able to compile CloudFuse code and use it.

Compiling and installing CloudFuse

First, download the CloudFuse code from cloudfuse-0.1.tar.gz. This tar.gz file has been uploaded and is maintained on github. It is shared here, and has been tested so that all the following steps will work fine. New code changes in the repository can cause compile-time errors. So, download the code from the above link. Extract the file, and then compile the software.

 
:~$ tar -xzvf cloudfuse-0.1.tar.gz

Once you have extracted the .tar.gz file, you should have following files under the cloudfuse-0.1 directory.

 
root@ubuntu-test:~/cloudfuse-0.1# ls -la
total 280
drwxr-xr-x 3 root root   4096 Feb 21 21:47 .
drwx------ 4 root root   4096 Feb 21 21:47 ..
drwxr-xr-x 8 root root   4096 Feb 21 21:47 .git
-rw-r--r-- 1 root root   1059 Feb 21 21:47 LICENSE
-rw-r--r-- 1 root root   1024 Feb 21 21:47 Makefile.in
-rw-r--r-- 1 root root   2332 Feb 21 21:47 README
-rw-r--r-- 1 root root  12014 Feb 21 21:47 cloudfsapi.c
-rw-r--r-- 1 root root   1043 Feb 21 21:47 cloudfsapi.h
-rw-r--r-- 1 root root  11240 Feb 21 21:47 cloudfuse.c
-rw-r--r-- 1 root root   4335 Feb 21 21:47 config.h.in
-rwxr-xr-x 1 root root 198521 Feb 21 21:47 configure
-rw-r--r-- 1 root root   1324 Feb 21 21:47 configure.in
-rwxr-xr-x 1 root root  13184 Feb 21 21:47 install-sh
root@ubuntu-test:~/cloudfuse-0.1#

Now it is time to compile CloudFuse and install it. You will need the following utilities and their dev packages installed in order to build it: libcurl, libfuse, and libxml2

CloudFuse is built and installed like any other autoconf-configured code. Normally:

 
root@ubuntu-test:~/cloudfuse-0.1# ./configure
root@ubuntu-test:~/cloudfuse-0.1# make
root@ubuntu-test:~/cloudfuse-0.1# sudo make install

But, first you need to install the required packages, otherwise the ./configure command will fail and generate errors.

 
root@ubuntu-test:~/cloudfuse-0.1# apt-get update
root@ubuntu-test:~/cloudfuse-0.1# apt-get install gcc
root@ubuntu-test:~/cloudfuse-0.1# apt-get install libcurl4-openssl-dev
root@ubuntu-test:~/cloudfuse-0.1# apt-get install libxml2 libxml2-dev
root@ubuntu-test:~/cloudfuse-0.1# apt-get install libfuse-dev

Now run the following command in the cloudfuse-0.1 source code directory

 
root@ubuntu-test:~/cloudfuse-0.1# ./configure
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for XML... yes
checking for CURL... yes
checking for FUSE... yes
.........
............
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
root@ubuntu-test:~/cloudfuse-0.1# make
gcc -g -O2 -I/usr/include/libxml2     -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -o cloudfuse cloudfsapi.c cloudfuse.c -lxml2   -lcurl   -pthread -lfuse -lrt -ldl
root@ubuntu-test:~/cloudfuse-0.1# make install
/usr/bin/install -c cloudfuse /usr/local/bin/cloudfuse
root@ubuntu-test:~/cloudfuse-0.1#

If everything went fine, you should now have CloudFuse installed properly. Confirm this by running the which command. It should show the location of the CloudFuse binary file.

 
root@ubuntu-test:~/cloudfuse-0.1# which cloudfuse
/usr/local/bin/cloudfuse
root@ubuntu-test:~/cloudfuse-0.1#

Mounting Cloud Files:

Now let’s use CloudFuse to mount our Cloud Files.

You'll have to create a configuration file for CloudFuse in your home directory and put your Rackspace Cloud Files username and API key in it, as shown below:

 
$HOME/.cloudfuse
    username=[username]
    api_key=[api key]
    authurl=[auth URL]

Auth URLs:

US Cloud Files account: https://auth.api.rackspacecloud.com/v1.0

UK Cloud Files account: https://lon.auth.api.rackspacecloud.com/v1.0

The following entries are optional. You can define these values in the .cloudfuse file.

 
     use_snet=[True to use snet for connections]
     cache_timeout=[seconds for directory caching, default 600]

After creating the above configuration file, run the cloudfuse command as follows. The syntax should be as simple as:

 
cloudfuse [mount point]

So, you should be able to mount Cloud Files like this

 
root@ubuntu-test:/# mkdir cloudfiles
root@ubuntu-test:/# cloudfuse /cloudfiles

If you run the # ls -la command inside the /cloudfiles directory you should see your Cloud Files containers.

If you are not logged in as the user root on the system, then your username will need to be part of the "fuse" group. This can be accomplished with the following command:

 
sudo usermod -a -G fuse [username]

If you are unable to see any containers inside the mountpoint, then one or more of the above steps didn't work properly. You will need to repeat the process, and verify that all of the above steps were followed precisely.



© 2011-2013 Rackspace US, Inc.

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License


See license specifics and DISCLAIMER

34 Comments

./configure output
checking for XML... configure: error: 'Unable to find libxml2. Please make sure library and header files are installed.'

apt-get install libxml2 output
Reading package lists... Done
Building dependency tree
Reading state information... Done
libxml2 is already the newest version.

Trying installing the "pkg-config" package, that might help configure find the library. You might also try installing "libxml2-dev" if installing pkg-config doesn't do the trick. If it still doesn't want to work, check configure.log (it should be created by configure) and see if any of the information in there can help us identify why it can't find libxml2.

Hi there,

I just tried out this tutorial on Ubuntu 10.10 image. It's very well written. Thanks! I followed all instructions and getting the follow two erros:

first. after I successfully executed 'cloudfuse cloudfiles'. I can add files to the directory but when I create a file, copy, or move, I get this error message:

for example:
# touch test
touch: setting times of `test': Function not implemented

This is an issue for me because some program I use won't save file in the mounted directory. I don't seem to find the right answer for it. I wonder if anyone here knows.

Thanks in advance for the help!

Li

PS: I also don't have the user group 'fuse' created by following above direction. I have some issues with "sudo usermod -a -G fuse [username]" but I can live with that. I want to mention it in case they are connected.


Odd behavior. I'll need to set up an environment and test this, I'll see if I can figure out what it's doing - I'll post the results a bit later. For the group, if it didn't get created you can run "sudo groupadd fuse" to add it, then try adding the user to the group again. If that isn't getting created by default then I'll make sure to get that instruction into the article.

Working on Ubuntu 10.04 LTS, I followed the tutorial and got the following error at the end:

fuse: device not found, try 'modprobe fuse' first

The solution was to do this:

mkdir -p /mnt/cloudfiles # <===or wherever you want to put your mount point
modprobe fuse
cloudfuse /mnt/cloudfiles

Ok, self-replying because it ate the formatting. The answer is to do the following (no quotes): "mkdir /mnt/cloudfiles; modprobe fuse; cloudfuse /mnt/cloudfiles;" If you want it mounted somewhere other than /mnt/cloudfiles, change it in both places.

Sorry about the formatting, but thanks for the advice about loading the kernel module. I'll get that incorporated into the article soon.

Another thing to be aware of: when you 'cloudfuse somedir', it scans your list of containers and makes them visible as directories inside 'somedir'. It will NOT continue to rescan, however. That means that if you issue the cloudfuse command at noon, then create a new container at 1pm, the new container will not be available through the cloudfuse'd directory. To make that directory / container visible via cloudfuse, do the following (no quotes): "umount somedir; cloudfuse somedir;"

Followed the instructions on a fresh Ubuntu Natty instance, 'fuse' group was not created.
Created fuse group, added unprivileged user to that group, unprivileged user is unable to access mounted cloudfiles dir. I'm about to change the group ownership of the cloud mounted files to see if that is the minimum of what is needed, otherwise an update to the article would be appreciated

Found a howto that worked.
http://tumblr.dderek.com/post/526900310/using-cloudfuse-to-mount-the-rackspace-cloud-files

Definitely looks like this needs updating. We'll try to get to it sooner than later. Thanks for letting us know about the trouble with Natty.

To autoload fuse at startup so that /etc/fstab works, add "fuse" (minus quotes) to /etc/modules, which is used for loading kernel modules automatically.

While the setting is briefly mentioned, it's not mentioned that enabling use_snet will route traffic via your "private" network interface, which means no bandwidth charges for transferring.

Found that out the hard way after transferring gigabytes of files, watching the transfer, and noticing it was moving out via my "public" interface.

Cheers

Thanks for the help Jon. I will be updating this article, and I'll definitely work in the adjustments and warnings that have been posted in the comments.

I was able to follow the installation instructions and everything appears to have worked correctly; no errors. However, when I attempt to run cloudfuse, I get the error "Unable to determine username and API key." I tried specifying my username and API key via command line as well as including the information in /root/.cloudfuse, but I still get the same error. I have cloudfuse installed on two other machines and they work without issue. The only difference I can see between the two is that the new machine is running Centos 6; the others, Centos 5.5.

Any assistance would be greatly appreciated.
Thank you.

Excuse me for saying this, but this tutorial is quite mucky! I have precisely tested the following procedure starting will a plain Ubuntu 10.04LTS instance. Just fill in your values in place of the "<*****>" sections:

sudo su

cd /home/<your_home_directory>

wget http://c16281.r81.cf2.rackcdn.com/cloudfuse-0.1.tar.gz

tar -xzvf cloudfuse-0.1.tar.gz
cd cloudfuse-0.1

apt-get update
apt-get upgrade

NOTE: Repeating "apt-get update" is sometimes advisable after upgrading.
apt-get update

apt-get install gcc
apt-get install libcurl4-openssl-dev
apt-get install libxml2 libxml2-dev
apt-get install libfuse-dev

./configure
make
make install

which cloudfuse

NOTE: The output should be:
/usr/local/bin/cloudfuse
root@ubuntu-test:~/cloudfuse-0.1#

cd /root

nano .cloudfuse

ADD THE FOLLOWING LINES (see "Your Account / API Access" in the Cloud Control Panel for <username> and <api_key> values):

username=<username>
api_key=<api_key>
authurl=https://auth.api.rackspacecloud.com/v1.0

cd /home/<your_home_directory>

mkdir cloudfiles

cloudfuse /home/<your_home_directory>/cloudfiles

NOTE: Do the following if needed.
sudo usermod -a -G fuse <your_system_user_name>

NOTE FINAL: Everyone should know the value of test servers. Linux is a "River of No Return" for many commands and operations. No system planned for production should have anything done on it without first thoroughly testing it on an identical test server.

I attempted to set this up on a highly configured server, and the "cloudfuse /home/<your_home_directory>/cloudfiles" command did not work.

I repeated the entire procedure, and when arriving at that step, I used the following instead. This worked:

cd /home/<your_home_directory>
cloudfuse cloudfiles

I don't know why ???

Please note that the code will not compile against newer cURL versions such as the ones in CentOS 6 or Fedora 16.

Followed all the steps mentioned above. Successfully Mounted the cloudfiles to me server. But when i am giving command "ls -l /media/cloudfiles" it hangs. Please suggest..

Hard to say what causes a problem like that, Tapas. I'd recommend going back to the cloudfuse source, running "./configure" again, then "make" and "make install". Watch for any warnings or errors during that process that might give any clues.

If nothing shows up there and "ls" hangs again, after doing that ls check your system logs (probably /var/log/messages, maybe /var/log/dmesg) to see if there's an entry there from cloudfuse.

Tapas, I am having the exact same issue. Have you found a fix? I am using CentOS 6 though. Could this be a problem as suggested by Florin?

same problem, cloudfuse doesn't seem to work for centos6. anyone have any tricks?

We haven't had a change to delve into the problem yet, but the original author has a blog post on the subject of using CloudFuse with Cloud Servers that's received a couple recent updates and has more than a few comments.

http://sandeepsidhu.wordpress.com/2011/03/07/mounting-cloud-files-using-cloudfuse-into-ubuntu-10-10-v2/

I am testing Cloudfuse for writing backups via my FTP server to the cloudfuse mount. I noticed that if the backup software creates some dirs and deletes them afterwards the dirs remain visible via the cloudfuse mount. If i then try to delete these dirs manualy i do get the error message "No such file or directory". That make sense cause the dir's don't exist anymore. Strange thing is that the cloudfile mount doesn't seem to refresh and still shows the removed dirs.
Conclusion: (unfortunatly) not suitable for production...:-(

If you're installing this on multiple RHEL-based boxes it's worth building an RPM. I've added a link to the spec file I used <a href="https://github.com/redbo/cloudfuse/issues/40">here</a>.

Doesnt work on CentOS 6 :(

For CentOS 5.5 use "yum" instead of apt-get
Also the package names end in "devel" rather than "dev"
Everything else is the same.

Using Ubuntu 12 LTS I had to remove the include for curl/types.h in one of the header files before compiling. I was getting an error that curl/types.h is long deprecated and no longer installed on the system at all. Removed that line and everything still seems to work just fine.

This tutorial could be MUCH more useful it this:
use_snet=[True to use snet for connections]

were defined more clearly.

If you are mounting a cloudfile share from a Rackspace server, you WANT this, else you will be charged for outgoing bandwidth when your server accesses the the cloudfiles.

-D

Hi all, I am trying to install CloudFuse on Ubuntu 12.04. I removed curl/types.h include line as suggested but I am getting strange errors:
/home/cloudfuse-0.1/cloudfsapi.c:46: undefined reference to `CRYPTO_num_locks'
/home/cloudfuse-0.1/cloudfsapi.c:46: undefined reference to `CRYPTO_malloc'
/home/cloudfuse-0.1/cloudfsapi.c:48: undefined reference to `CRYPTO_num_locks'
/home/cloudfuse-0.1/cloudfsapi.c:50: undefined reference to `CRYPTO_set_id_callback'
/home/cloudfuse-0.1/cloudfsapi.c:51: undefined reference to `CRYPTO_set_locking_callback'

Has anyone come across those before?

Another way to skin this cat: I've been developing a Unix backup program, HashBackup, for a few years now that supports Cloud Files. And you can mount the remote backup. You can't write directly to the mount like Cloud Fuse, but it might be an option if you are basically using Cloud Files to do backups and want to view files through a regular file system.

The other nice thing: all your data is deduped, compressed, and encrypted with a private key kept on your local system and known only to you.

It's at http://www.hashbackup.com if you want to give it a try.

Hi All,

I'd suggest to download the recent cloudfuse version from:

git clone https://github.com/redbo/cloudfuse.git

It works fine on RHEL 6x64 version without any "hangs"

My goal in using cloudfuse in a CentOS 6 environment is to have an unlimited storage device for photo and video storage uploaded by users from a Joomla Content Management System (CMS). I am using a component called JOM Social for user upload of photos and media, but may be running into a permissions issue for files written to the cloud container. When I display container contents of the mounted cloud file, it shows all files are owned by root:root. When I create a test file in the container, it is created as root:root. When I su to another user, I lose access to the container completely.

So my question is: How do I mount the cloud files and own files in a container as another user (apache:apache ideally) other than root?

I'm not very familiar with cloudfuse, but I do believe that you can pass a umask in the options when mounting the container. Something like:

cloud fuse umask=0007,username="blah",api_key="blahblah" ...

That should set the container to be group-writeable. A umask of "0000" should make the files created in the container writeable by anyone, but that isn't a pretty approach from a security standpoint.

An alternative might be a program called fileconveyor that syncs files to a container from a directory:

http://www.rackspace.com/knowledge_center/article/syncing-to-cloud-files-with-fileconveyor

That way the orignal files would be natively in the file system, bypassing the need to set up a separate mount for the container.

Add new comment