In this little tutorial I’m going to explain the basic steps to set up an iscsi target on a debian machine.
Install required packages:
# apt-get install open-iscsi multipath-tools
I always use multipath to get path availability.
Configure /etc/iscsi/iscsid.conf file according to your storage server. It is very common the use of chap authentication for logging into a specified target. In order to configure the user and password, edit /etc/iscsi/iscsid.conf file and change these parameters:
node.session.auth.authmethod = CHAP node.session.auth.username_in = user node.session.auth.password_in = password
Discover iscsi targets:
# iscsiadm -m discovery -t sendtargets -p <PORTAL_IP>
You are going to get an output like this:
192.168.0.50:3260,1 iscsi-target1 192.168.0.150:3260,1 iscsi-target1
In this case we have two paths to the same target. It can be, because your storage server has two differents controllers, that can be accessed to reach same target or because your storage server has two network cards but only one internal controller. Any way, we have two paths to reach the same target.
Now, that we have the target discovered we can log in on each portal:
# /usr/bin/iscsiadm --mode node --portal 192.168.0.50:3260 --targetname iscsi-target1 --login # /usr/bin/iscsiadm --mode node --portal 192.168.0.150:3260 --targetname iscsi-target1 --login
To view your current iscsi connections:
# iscsiadm -m session tcp: [2] 172.16.64.10:3260,1 iscsi-target1 tcp: [3] 172.16.66.10:3260,1 iscsi-target1
or
# /etc/init.d/open-iscsi status Current active iSCSI sessions: tcp: [2] 172.16.64.10:3260,1 iscsi-target1 tcp: [3] 172.16.66.10:3260,1 iscsi-target1
To see what luns provides each iscsi connection you can run the following command, with the parameter -r we specify the session number we want to see:
~# iscsiadm -m session -P3 -r3 iSCSI Transport Class version 2.0-870 version 2.0-873 Target: iscsi-target1 (non-flash) Current Portal: 192.168.0.50:3260,1 Persistent Portal: 192.168.0.50:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1993-08.org.debian:01:6a8c80ddf449 Iface IPaddress: 192.168.0.20 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 8 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ********* Timeouts: ********* Recovery Timeout: 120 Target Reset Timeout: 30 LUN Reset Timeout: 20 Abort Timeout: 15 ***** CHAP: ***** username: chapuser password: ******** username_in: <empty> password_in: ******** ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 1048576 FirstBurstLength: 65536 MaxBurstLength: 1048576 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 9 State: running scsi9 Channel 00 Id 0 Lun: 0 Attached scsi disk sda State: running
If you check the available devs under /dev/disk/by-path you are going to see your new disks provided by iscsi luns. You can see how each lun is detected twice, one for each available path, for example lun0 can be accesed through sda and sdb:
# ls -l /dev/disk/by-path/ total 0 lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-0 -> ../../sda lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-1 -> ../../sdc lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-2 -> ../../sde lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-0 -> ../../sdb lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-1 -> ../../sdd lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-2 -> ../../sdf
We are going to use multipath tool to unify the two paths in an unified one. To do so check the properties of the new device:
~# udevadm info -q property /dev/sda DEVLINKS=/dev/disk/by-id/scsi-2616c304142756451 /dev/disk/by-path/ip-192.168.0.50:3260-iscsi-target1-lun-0 DEVNAME=/dev/sda DEVPATH=/devices/platform/host8/session3/target8:0:0/8:0:0:0/block/sda DEVTYPE=disk ID_BUS=scsi ID_MODEL=al0ABudQ263abHG3 ID_MODEL_ENC=al0ABudQ263abHG3 ID_PATH=ip-192.168.0.50:3260-iscsi-target1-lun-0 ID_PATH_TAG=ip-172_16_68_50_3260-iscsi-target1-lun-0 ID_REVISION=220 ID_SCSI=1 ID_SCSI_SERIAL=731ab7e9 ID_SERIAL=2616c304142756451 ID_SERIAL_SHORT=616c304142756451 ID_TYPE=disk ID_VENDOR=SCST_BIO ID_VENDOR_ENC=SCST_BIO MAJOR=8 MINOR=128 SUBSYSTEM=block TAGS=:systemd: USEC_INITIALIZED=40848863230
And configure multipath accordingly adding this section to the file /etc/multipatch.conf
devices {
device {
vendor "SCST_FIO|SCST_BIO"
product "*"
path_selector "round-robin 0"
path_grouping_policy multibus
rr_min_io 100
}
}
To check the unified paths run the following command:
# multipath -ll mpath12 (2507a637562517a71) dm-4 SCST_BIO,PzcubQzqZx3JHfm1 size=9.3T features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 3:0:0:2 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 2:0:0:2 sde 8:64 active ready running mpath11 (26b59586b61536156) dm-2 SCST_BIO,kYXkaSaVkZZalcKP size=4.9T features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 2:0:0:1 sdc 8:32 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 3:0:0:1 sdd 8:48 active ready running mpath10 (24b4f4575564c6b4b) dm-3 SCST_BIO,KOEuVLkKIxommozh size=4.9T features='0' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 2:0:0:0 sda 8:0 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 3:0:0:0 sdb 8:16 active ready running
We can see multipath topology, mpath12 is the multipath device for sde and sdf accesed through dm-4. Now, we can partition /dev/mapper/mpath12 as needed and use it as other disk in our system. Multipath has some parameters that can be changed for each device, for example the path policy (see multipath help). In our example the policy is set to round-robin.
if you don’t see any multipath devices with the above command try running multipath -v3 to see verbose messages and see if are showing up messages like this:
Apr 12 20:07:51 | sda: blacklisted, udev property missing
if so, add these lines to multipath.conf and run again multipath -v3 command
blacklist_exceptions {
property "(ID_WWN|SCSI_IDENT_.*|ID_SERIAL)"
}
And now the paths should be discovered.
Thank you.
A great tutorial for debian
I had troubles where I was trying to add a device (with already a partition and filesystem) from Dell Storage and it kept coming as /dev/mapper/mpatha-part1 /dev/mapper/mpatha etc.
Which means I created a new block volume and did not create partition on it. and directly created filesystem (mkfs.ext4 /dev/mapper/mpathb) and then it started showing as
mpathb (36000d31003660a00000000000000001b) dm-5 COMPELNT,Compellent Vol
size=30T features=’1 queue_if_no_path’ hwhandler=’0′ wp=rw
`-+- policy=’service-time 0′ prio=1 status=active
|- 67:0:0:2 sdb 8:16 active ready running
|- 68:0:0:2 sde 8:64 active ready running
|- 69:0:0:2 sdg 8:96 active ready running
`- 65:0:0:2 sdf 8:80 active ready running
Although not sure why policy is “service-time 0” even though I have path_selector “round-robin 0” in /etc/multipath.conf
Also we will mount it as follow in /etc/fstab (with _netdev flag)
dev/mapper/mpathb /share ext4 _netdev 0 0