Skip to main content

Category: Linux

Nagios plugin for checking mysql synchronization state

sysadmin Leave a comment

I want to share this simple nagios plugin for monitoring mysql slaves replications states:

check_mysql_sync

This plugin can be downloaded here. To install copy this script to nagios plugins directory (/usr/lib/nagios/plugins/). View script help for full options list:

# /usr/lib/nagios/plugins/check_mysql_sync.pl -h

Check mysql sync between master and slave for Nagios version 0.1
Usage: /usr/lib/nagios/plugins/check_mysql_sync.pl [-v] -H <host> -p <port> -l <username> -x <password> -w <warn level> -c <crit level>  [-V]
-v, --verbose
   print extra debugging information 
-h, --help
   print this help message
-H, --hostname=HOST
   name or IP address of host to check
-l, --login
    username for mysql database (Default nagios)
-x, --passwd=PASSWD
   password
-p, --port=PORT
   MYSQL port (Default 3361)
-w, --warn=INTEGER
   warning level for seconds behind sync
-c, --crit=INTEGER 
   critical level for seconds behind sync
-V, --version
   prints version number

Run this command in your mysql master for grant to nagios user the required privileges:

 GRANT PROCESS, REPLICATION CLIENT on *.*  TO 'nagios'@'192.168.1.25' IDENTIFIED BY 'yourpass';

Using rpaf Apache module to preserve client IP

sysadmin 8 Comments

Suppose you have two web servers on your network, public Server A and private Server B. Maybe you want to publish some sections of your private server on internet using Apache proxy module. In this example, “users” path on www.server_a.com is redirected to www.server_b.com/users , because Server A is in the same private network as Server B is, Server A knows how to reach Server B and redirects the web requests to it. The problem here is that Server B will always have the IP of Server A in all requests (192.168.20.50).

Server A
Public IP: 85.164.26.20
Private IP: 192.168.20.50

 ProxyRequests Off
 ProxyPreserveHost On
 ProxyPass /users http://www.server_b.com/users
 ProxyPassReverse /users http://www.server_b.com/users

Server B
Private IP: 192.168.20.51

To solve this problem you can use rpaf module for Apache2. Explanation from the author:

“It changes the remote address of the client visible to other Apache modules when two conditions are satisfied. First condition is that the remote client is actually a proxy that is defined in httpd.conf. Secondly if there is an incoming X-Forwarded-For header and the proxy is in it's list of known proxies it takes the last IP from the incoming X-Forwarded-For header and changes the remote address of the client in the request structure.”

In this case rpaf translates the remote address 192.168.20.50 to real client address sent in X-Forwarded-For header.

1- Install rpaf module on Server B

# apt-get install libapache2-mod-rpaf

2- Configure your virtualhost in Server B to use rpaf module:

RPAFenable On
RPAFsethostname On
RPAFproxy_ips 192.168.20.50

3- Restart apache

# apachect restart

Now, you will see the real client IP in your apache access logs.

550 Message does not pass DomainKeys requirements for domain

sysadmin 15 Comments

I got this error from my mail server log when attempting to send a mail to one of our clients: 

/var/log/syslog.4.gz:Dec 14 18:13:01 XXXXXX postfix/smtp[26150]: 01BA812B22C: to=<xxxxx@yyyyy.com>, 
relay=smtp.xxxxxx.com[1.2.3.4]:25, delay=0.11, delays=0.03/0.01/0.02/0.05, dsn=5.0.0,
status=bounced (host smtp.xxxxxx.com[1.2.3.4] said: 550 Message does not pass DomainKeys requirements
 for domain zzzz.com (in reply to end of DATA command))

I’ve not implemented DomainKeys in my mail servers (but they was in the past), but I noticed that my DNS servers was wrong configured to support this protocol. DomainKeys needs two TXT records, one for the policy and one for the selector.

The policy is set with a TXT record for _domainkey.yourdomain.com, “o=~;” means that some mails can be signed and “o=-;” means all mails must be signed for domain yourdomain.com. I my case I had to change “o=-;” to “o=~;” because now, I was not using Domainkeys in my MTAs.

_domainkey                TXT    "o=~;"

The selector is implemented with other TXT record, in which you set your public key. According to RFC:

Selectors are arbitrary names below the “_domainkey.” namespace. A selector value
and length MUST be legal in the DNS namespace and in email headers
with the additional provision that they cannot contain a semicolon.

brisbane._domainkey IN TXT "g=; k=rsa; p=MHww ... IDAQAB"

The flags you can set are explained below:

g = granularity of the key. If present with a non-zero length
value, this value MUST exactly match the local part of the
sending address. This tag is optional.

The intent of this tag is to constrain which sending address
can legitimately use this selector. An email with a sending
address that does not match the value of this tag constitutes
a failed verification.

k = key type (rsa is the default). Signers and verifiers MUST
support the ‘rsa’ key type. This tag is optional.

n = Notes that may be of interest to a human. No interpretation
is made by any program. This tag is optional.

p = public key data, encoded as a Base64 string. An empty value
means that this public key has been revoked. This tag MUST be
present.

t = a set of flags that define boolean attributes. Valid
attributes are as follows:

y = testing mode. This domain is testing DomainKeys and
unverified email MUST NOT be treated differently from
verified email. Recipient systems MAY wish to track
testing mode results to assist the sender.

This tag is optional.

For example, valid entries for selectors can be:

“coolumbeach._domainkey.example.net”
“sebastopol._domainkey.example.net”
“reykjavik._domainkey.example.net”
“default._domainkey.example.net”

Here, you can use these links to test if your DNS records are well formed. the first one is to check your policy and the last one to check the selector:

http://domainkeys.sourceforge.net/policycheck.html
http://domainkeys.sourceforge.net/selectorcheck.html

Setting up multipath iscsi targets on debian

sysadmin 3 Comments

In this little tutorial I’m going to explain the basic steps to set up an iscsi target on a debian machine.

Install required packages:

# apt-get install open-iscsi multipath-tools

I always use multipath to get path availability.

Configure /etc/iscsi/iscsid.conf file according to your storage server. It is very common the use of chap authentication for logging into a specified target. In order to configure the user and password, edit /etc/iscsi/iscsid.conf file and change these parameters:

node.session.auth.authmethod = CHAP
node.session.auth.username_in = user
node.session.auth.password_in = password

Discover iscsi targets:

# iscsiadm -m discovery -t sendtargets -p <PORTAL_IP>

You are going to get an output like this:

192.168.0.50:3260,1 iscsi-target1
192.168.0.150:3260,1 iscsi-target1

In this case we have two paths to the same target. It can be, because your storage server has two differents controllers, that can be accessed to reach same target or because your storage server has two network cards but only one internal controller. Any way, we have two paths to reach the same target.

Now, that we have the target discovered we can log in on each portal:

# /usr/bin/iscsiadm --mode node --portal 192.168.0.50:3260 --targetname iscsi-target1 --login
# /usr/bin/iscsiadm --mode node --portal 192.168.0.150:3260 --targetname iscsi-target1 --login

To view your current iscsi connections:

# iscsiadm -m session
tcp: [2] 172.16.64.10:3260,1 iscsi-target1
tcp: [3] 172.16.66.10:3260,1 iscsi-target1

or

# /etc/init.d/open-iscsi status
Current active iSCSI sessions:
tcp: [2] 172.16.64.10:3260,1 iscsi-target1
tcp: [3] 172.16.66.10:3260,1 iscsi-target1

To see what luns provides each iscsi connection you can run the following command, with the parameter -r we specify the session number we want to see:

~# iscsiadm -m session -P3 -r3
iSCSI Transport Class version 2.0-870
version 2.0-873
Target: iscsi-target1 (non-flash)
	Current Portal: 192.168.0.50:3260,1
	Persistent Portal: 192.168.0.50:3260,1
		**********
		Interface:
		**********
		Iface Name: default
		Iface Transport: tcp
		Iface Initiatorname: iqn.1993-08.org.debian:01:6a8c80ddf449
		Iface IPaddress: 192.168.0.20
		Iface HWaddress: <empty>
		Iface Netdev: <empty>
		SID: 8
		iSCSI Connection State: LOGGED IN
		iSCSI Session State: LOGGED_IN
		Internal iscsid Session State: NO CHANGE
		*********
		Timeouts:
		*********
		Recovery Timeout: 120
		Target Reset Timeout: 30
		LUN Reset Timeout: 20
		Abort Timeout: 15
		*****
		CHAP:
		*****
		username: chapuser
		password: ********
		username_in: <empty>
		password_in: ********
		************************
		Negotiated iSCSI params:
		************************
		HeaderDigest: None
		DataDigest: None
		MaxRecvDataSegmentLength: 262144
		MaxXmitDataSegmentLength: 1048576
		FirstBurstLength: 65536
		MaxBurstLength: 1048576
		ImmediateData: Yes
		InitialR2T: No
		MaxOutstandingR2T: 1
		************************
		Attached SCSI devices:
		************************
		Host Number: 9	State: running
		scsi9 Channel 00 Id 0 Lun: 0
			Attached scsi disk sda		State: running

If you check the available devs under /dev/disk/by-path you are going to see your new disks provided by iscsi luns. You can see how each lun is detected twice, one for each available path, for example lun0 can be accesed through sda and sdb:

# ls -l /dev/disk/by-path/
total 0
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-0 -> ../../sda
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-1 -> ../../sdc
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.150:3260-iscsi-target1-lun-2 -> ../../sde
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-0 -> ../../sdb
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-1 -> ../../sdd
lrwxrwxrwx 1 root root 9 dic 13 23:47 ip-192.168.0.50:3260-iscsi-target1-lun-2 -> ../../sdf

We are going to use multipath tool to unify the two paths in an unified one. To do so check the properties of the new device:

~# udevadm info -q property /dev/sda
DEVLINKS=/dev/disk/by-id/scsi-2616c304142756451 /dev/disk/by-path/ip-192.168.0.50:3260-iscsi-target1-lun-0
DEVNAME=/dev/sda
DEVPATH=/devices/platform/host8/session3/target8:0:0/8:0:0:0/block/sda
DEVTYPE=disk
ID_BUS=scsi
ID_MODEL=al0ABudQ263abHG3
ID_MODEL_ENC=al0ABudQ263abHG3
ID_PATH=ip-192.168.0.50:3260-iscsi-target1-lun-0
ID_PATH_TAG=ip-172_16_68_50_3260-iscsi-target1-lun-0
ID_REVISION=220
ID_SCSI=1
ID_SCSI_SERIAL=731ab7e9
ID_SERIAL=2616c304142756451
ID_SERIAL_SHORT=616c304142756451
ID_TYPE=disk
ID_VENDOR=SCST_BIO
ID_VENDOR_ENC=SCST_BIO
MAJOR=8
MINOR=128
SUBSYSTEM=block
TAGS=:systemd:
USEC_INITIALIZED=40848863230

And configure multipath accordingly adding this section to the file /etc/multipatch.conf

devices {
   device {
                vendor                          "SCST_FIO|SCST_BIO"
                product                         "*"
                path_selector                   "round-robin 0"
                path_grouping_policy            multibus
                rr_min_io                       100
        }
}

To check the unified paths run the following command:

# multipath -ll
mpath12 (2507a637562517a71) dm-4 SCST_BIO,PzcubQzqZx3JHfm1
size=9.3T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 3:0:0:2 sdf 8:80   active ready  running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 2:0:0:2 sde 8:64   active ready  running
mpath11 (26b59586b61536156) dm-2 SCST_BIO,kYXkaSaVkZZalcKP
size=4.9T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 2:0:0:1 sdc 8:32   active ready  running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:1 sdd 8:48   active ready  running
mpath10 (24b4f4575564c6b4b) dm-3 SCST_BIO,KOEuVLkKIxommozh
size=4.9T features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=1 status=active
| `- 2:0:0:0 sda 8:0    active ready  running
`-+- policy='round-robin 0' prio=1 status=enabled
  `- 3:0:0:0 sdb 8:16   active ready  running

We can see multipath topology, mpath12 is the multipath device for sde and sdf accesed through dm-4. Now, we can partition /dev/mapper/mpath12 as needed and use it as other disk in our system. Multipath has some parameters that can be changed for each device, for example the path policy (see multipath help). In our example the policy is set to round-robin.

if you don’t see any multipath devices with the above command try running multipath -v3 to see verbose messages and see if are showing up messages like this:

Apr 12 20:07:51 | sda: blacklisted, udev property missing

if so, add these lines to multipath.conf and run again multipath -v3 command

blacklist_exceptions {
        property "(ID_WWN|SCSI_IDENT_.*|ID_SERIAL)"
}

And now the paths should be discovered.

How to clear iptables rules

sysadmin Leave a comment

I am using the following script to clear all iptables rules. I use the script when I am debugging my iptables rules.

#!/bin/bash


/sbin/iptables -P INPUT   ACCEPT
/sbin/iptables -P OUTPUT  ACCEPT
/sbin/iptables -P FORWARD ACCEPT
/sbin/iptables -F
/sbin/iptables -X

for table in filter nat mangle; do
    /sbin/iptables -t $table -F
    /sbin/iptables -t $table -X
    /sbin/iptables -t $table -Z
done