Setting up the GlusterFS Servers

server1.example.com/server2.example.com:

The latest GlusterFS version is available as Ubuntu package from gluster.org. We can install it as follows:

We add the public key of the gluster.org Ubuntu repository on both servers.

wget -O – http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.9/rsa.pub | apt-key add –

Then add the GlusterFS repository (the command is one line!)

echo deb http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.9/ubuntu/jessie/apt jessie main > /etc/apt/sources.list.d/gluster.list

and update the repository list.

apt-get update

Now we can install the GlusterFS server software with apt.

apt-get -y install glusterfs-server

The command

glusterfsd –version

should now show the GlusterFS version that you’ve just installed (3.7.9 in this case):

root@server1:/# glusterfsd –version
glusterfs 3.7.9 built on Mar 23 2016 05:24:49
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2013 Red Hat, Inc. <http://www.redhat.com/>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
It is licensed to you under your choice of the GNU Lesser
General Public License, version 3 or any later version (LGPLv3
or later), or the GNU General Public License, version 2 (GPLv2),
in all cases as published by the Free Software Foundation.
root@server1:/#

If you use a firewall, ensure that TCP ports 111, 24007, 24008, 24009-(24009 + number of bricks across all volumes) are open onserver1.example.com and server2.example.com.

Glusterfs shall store its data in the directory /data on the servers. This location can be a normal directory if you have a smaller installation or you use a separate hard disk partition and mount it as /data.

Run on both servers:

mkdir /data

to create the data directory.

Next, we must add server2.example.com to the trusted storage pool (please note that I’m running all GlusterFS configuration commands fromserver1.example.com, but you can as well run them from server2.example.com because the configuration is repliacted between the GlusterFS nodes – just make sure you use the correct hostnames or IP addresses):

server1.example.com:

On server1.example.com, run

gluster peer probe server2.example.com

root@server1:/# gluster peer probe server2.example.com
peer probe: success.
root@server1:/#

The status of the trusted storage pool should now be similar to this:

gluster peer status

root@server1:/# gluster peer status
Number of Peers: 1

Hostname: server2.example.com
Uuid: 0f7ee46c-6a71-4a31-91d9-6076707eff95
State: Peer in Cluster (Connected)
root@server1:/#

Next we create the share named testvol with two replicas (please note that the number of replicas is equal to the number of servers in this case because we want to set up mirroring) on server1.example.com and server2.example.com in the /data/testvol directory (this will be created if it doesn’t exist):

gluster volume create testvol replica 2 transport tcp server1.example.com:/data/testvol server2.example.com:/data/testvol force

root@server1:/# gluster volume create testvol replica 2 transport tcp server1.example.com:/data/testvol server2.example.com:/data/testvol force
volume create: testvol: success: please start the volume to access data
root@server1:/#

Start the volume:

gluster volume start testvol

root@server1:/# gluster volume start testvol
volume start: testvol: success
root@server1:/#

Our test volume has been started successfully.

It is possible that the above command tells you that the action was not successful:

root@server1:~# gluster volume start testvol
Starting volume testvol has been unsuccessful
root@server1:~#

In this case you should check the output of…

server1.example.com/server2.example.com:

netstat -tap | grep glusterfsd

on both servers.

If you get output like this…

root@server1:/# netstat -tap | grep glusterfsd
tcp 0 0 *:49152 *:* LISTEN 8007/glusterfsd
tcp 0 0 server1.example.c:65533 server1.example.c:24007 ESTABLISHED 8007/glusterfsd
tcp 0 0 server1.example.c:49152 server2.example.c:65531 ESTABLISHED 8007/glusterfsd
tcp 0 0 server1.example.c:49152 server1.example.c:65532 ESTABLISHED 8007/glusterfsd
tcp 0 0 server1.example.c:49152 server1.example.c:65531 ESTABLISHED 8007/glusterfsd
tcp 0 0 server1.example.c:49152 server2.example.c:65526 ESTABLISHED 8007/glusterfsd
root@server1:/#

… everything is fine, but if you don’t get any output…

root@server2:~# netstat -tap | grep glusterfsd
root@server2:~#

… restart the GlusterFS daemon on the corresponding server (server1.example.com in this case):

server2.example.com:

service glusterfs-server restart

Then check the output of…

netstat -tap | grep glusterfsd

… again on that server – it should now look like this:

root@server2:/# netstat -tap | grep glusterfsd
tcp 0 0 *:49152 *:* LISTEN 7852/glusterfsd
tcp 0 0 server2.example.c:49152 server2.example.c:65532 ESTABLISHED 7852/glusterfsd
tcp 0 0 server2.example.c:49152 server1.example.c:65526 ESTABLISHED 7852/glusterfsd
tcp 0 0 server2.example.c:49152 server2.example.c:65525 ESTABLISHED 7852/glusterfsd
tcp 0 0 server2.example.c:65533 server2.example.c:24007 ESTABLISHED 7852/glusterfsd
tcp 0 0 server2.example.c:49152 server1.example.c:65524 ESTABLISHED 7852/glusterfsd
root@server2:/#

Now back to server1.example.com:

server1.example.com:

You can check the status of the volume with the command

gluster volume info

root@server1:/# gluster volume info

Volume Name: testvol
Type: Replicate
Volume ID: 3fc9af57-ca56-4a72-ad54-3d2ea03e5883
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: server1.example.com:/data/testvol
Brick2: server2.example.com:/data/testvol
Options Reconfigured:
performance.readdir-ahead: on
root@server1:/#

By default, all clients can connect to the volume. If you want to grant access to client1.example.com (= 192.168.1.102) only, run:

gluster volume set testvol auth.allow 192.168.1.102

root@server1:/# gluster volume set testvol auth.allow 192.168.1.102
volume set: success
root@server1:/#

Please note that it is possible to use wildcards for the IP addresses (like 192.168.*) and that you can specify multiple IP addresses separated by comma (e.g. 192.168.1.102,192.168.1.103).

The volume info should now show the updated status:

gluster volume info

root@server1:/# gluster volume info

Volume Name: testvol
Type: Replicate
Volume ID: 3fc9af57-ca56-4a72-ad54-3d2ea03e5883
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: server1.example.com:/data/testvol
Brick2: server2.example.com:/data/testvol
Options Reconfigured:
auth.allow: 192.168.1.102
performance.readdir-ahead: on
root@server1:/#