OCFS2 + ISCSI Centralized storage in Ubuntu 10.10

In this article I will mention the steps to mount an ISCSI target on two ubuntu machines and then cluster it using oracle clustered file system. The newly mounted partition can be used as a centralized storage location in High availability, Failover or load-balancing setup.

The step by step howto is provided below,

1) Setup an ISCSI Server using Openfiler and create SAN LUN, and assign the IP 192.168.1.11 to it.

For setting up openfiler based ISCI target, you can refer steps 1 to 8 mentioned in the url pasted below.

http://www.techhead.co.uk/how-to-configure-openfiler-v23-iscsi-storage-for-use-with-vmware-esx

2) Setup two servers with Ubuntu 10.10 in it.
server1 ==> name: heuristics –> IP 192.168.1.30
server2 ==> name: heuristics2 –> IP 192.168.1.32

3) Install open-iscsi tool in both servers
apt-get install open-iscsi open-iscsi-utils
/etc/init.d/open-iscsi start

4) List out the ISCSI targets available in both servers.
iscsiadm -m discovery -t sendtargets -p 192.168.1.11

In my case the above command produced the following output,
root@heuristics2:~# iscsiadm -m discovery -t sendtargets -p 192.168.1.11
192.168.1.11:3260,1 iqn.2006-01.com.openfiler:tsn.0d0c0c810c57
root@heuristics2:~#

5) Mount the ISCSI target “iqn.2006-01.com.openfiler:tsn.0d0c0c810c57″(Lets call it TG57) to the local machine
iscsiadm -m node -T iqn.2006-01.com.openfiler:tsn.0d0c0c810c57 -p 192.168.1.11 –login

6) Step5 will make the iscsi target TG57 as a device to the system. Which can be viewed as a block device.
root@heuristics# fdisk -l


Disk /dev/sdb: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003f05c

Device Boot Start End Blocks Id System
/dev/sdb1 1 1011 1047383 83 Linux

7) Install OCFS2 – Oracle Cluster File System for Linux
apt-get install ocfs2 ocfs2-tools
if you unable to install then you can also install ocfs2 & ocfs2-tool by using "synaptic" in ubuntu

Configure OCFS2

Create a configuration file with proper indendation and copy it to both the servers. In my case “ocfs2″ is the cluster name.

root@heuristics:~# cat /etc/ocfs2/cluster.conf
node:
ip_port = 7777
ip_address = 192.168.1.30
number = 0
name = heuristics
cluster = ocfs2

node:
ip_port = 7777
ip_address = 192.168.1.32
number = 1
name = heuristics2
cluster = ocfs2

cluster:
node_count = 2
name = ocfs2
root@heuristics:~#

If proper indentation is not provided the following error will be shown,
Starting cluster oracle: Failed
o2cb_ctl: Unable to load cluster configuration file “/etc/ocfs2/cluster.conf”
Stopping cluster oracle: Failed
o2cb_ctl: Unable to load cluster configuration file “/etc/ocfs2/cluster.conf”

9) Start the cluster service in both the machines
/etc/init.d/ocfs2 start
/etc/init.d/o2cb start
/etc/init.d/o2cb enable


10) Create one partition named /dev/sdb1 in the ISCSI target
fdisk /dev/sdb

(to ensure that /dev/sdb is the target)

shell> fdisk /dev/sdb
at the fdisk prompt, run the following key sequence:
o to create a new partition table
n to create a new partition
p to create a primary partiton
1 to create it as the first partition
[enter] twice to accept default first and last cylinders
t to autoselect only partition (what we just created) and change type
8e to change the type to Linux LVM
w to write the changes to the partition table and exit fdisk


11) Make ocfs cluster file system using the following command(need to execute only on one machine)
mkfs.ocfs2 -b 4k -C 32k -N3 -L cluster-storage /dev/sdb1

This creates a file system with 4096 block size and 32768 (32k) cluster size.
NOTE: N= 3 , for a cluster with 2 machines, N=3 and for a cluster with ‘n’ machines N=(n+1)

12) Update partition table on all servers in the cluster. In this case all the servers have /dev/sdb as the iSCSI target.

We will run the following to re-read the partition:
blockdev –rereadpt /dev/sdb

Next, we will want to create a mount point on the servers for this cluster.
mkdir /mycluster

Mount the partition,
mount -L cluster-storage /cluster-storage

13) Show results and test
root@heuristics2:~# hostname
heuristics2
root@heuristics2:~# df -TH /cluster-storage/
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 ocfs2 1.1G 330M 744M 31% /cluster-storage
root@heuristics2:~#
root@heuristics:~# hostname
heuristics
root@heuristics:~# df -TH /cluster-storage/
Filesystem Type Size Used Avail Use% Mounted on
/dev/sdb1 ocfs2 1.1G 330M 744M 31% /cluster-storage
root@heuristics:~#



ORIGINAL LINK OF THIS TUTORIAL with Thanks
http://jackal777.wordpress.com/2011/08/10/ocsf2-iscsi-centralized-storage-in-ubuntu-10-10/#comment-69


REFERENCE

http://www.idevelopment.info/data/Unix/Linux/LINUX_ConnectingToAniSCSITargetWithOpen-iSCSIInitiatorUsingLinux.shtml

http://knowledgelayer.softlayer.com/questions/265/Can+I+connect+multiple+servers+to+a+single+iSCSI+LUN%3F