Aoe

Serversetup with AOE

install vblade-persist

Install vblade-persist if not already.

If you want to expose a complete LVM first do the usual stuff with (assuming /dev/md1 will be the exposed storage):

pvcreate /dev/md1
vgcreate storage /dev/md1

Setup vblade-persist

Then setup the appropriate shelf and slots on the netdevice you want with the storage backend you want:

/usr/sbin/vblade-persist setup 0 1 br0 /dev/md1

Then make sure it automatically starts on boot:

/usr/sbin/vblade-persist auto all

Lastly start it now (or reboot)

/usr/sbin/vblade-persist start all

You can always check status with

/usr/sbin/vblade-persist ls

#shelf slot netif source auto? stat
0 1 br0 /dev/md1 auto run

Client setup with AOE

Install tools

Install aoetools

apt-get install aoetools

Run aoe-stat and it should find the exposed storage*

aoe-stat 
e0.1      4000.526GB    br0 up            
e1.1      1500.300GB    br0 up            
e2.0       920.198GB    br0 up            

* - keep in mind aoe is not TCP - so it has to be on the same network as all the servers using it. All servers on that network can then see the exported data.

Now you should be able to run lvscan, vgs etc on all clients on this network and see all lvm-data.

If you want to automagically start various volumes, you could change

/etc/default/aoetools

INTERFACES="br0"
LVMGROUPS="storage nas mirrordata"

for instance

Upgrading AOE-driver

Especially the driver on debian is pretty old and newer much better ones are out.

First, install prereq, grab source, compile, install, check it is installed and reboot your machine:

# Install prereq
apt-get update; apt-get -y install build-essential linux-headers-$(uname -r)

# Grab source (grab latest version)
mkdir aoe
cd aoe
wget http://support.coraid.com/support/linux/aoe6-84.tar.gz

# Extract and Compile
tar zxvf aoe6-84.tar.gz
make

# Install
make install

# Check version
aoe-version

# Reboot (if you want to load module at startup you could create a script for it or just put it in /etc/rc.local)
modprobe aoe

Optimizations for lowmem systems and avoid large pagecache being flushed to disk

## Some simple aoe-settings to avoid oom and more cleanly writes (from coraid hp: http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19 )
echo 3 > /proc/sys/vm/dirty_ratio 
echo 4 > /proc/sys/vm/dirty_background_ratio 
echo 32768 > /proc/sys/vm/min_free_kbytes