I had some free slots in two on my ceph nodes and I used them to set a new SSD only pool. Because the ssd OSDs are sharing the server with an existing SATA pool we have to do some additional steps. There are some good posts out there that explain how to set up two pools sharing the same server, but you have to edit manually the ceph crushmap. Although it’s not very difficult to do it in that way I achieve the same goal setting the crush location parameter for those OSDs. I’ve tested it in Hammer release.
First create a new root bucket for the ssd pool. This bucket will be used to set the ssd pool location using a crush rule.
ceph osd crush add-bucket ssds root
We already have some servers with SATA OSDs in production, but we have to add two new host buckets for the faked hostnames that we are going to use to set the ssd OSDs.
ceph osd crush add-bucket ceph-node1-ssd host ceph osd crush add-bucket ceph-node2-ssd host
Move the host buckets to the ssds root:
ceph osd crush move ceph-node1-ssd root=ssds ceph osd crush move ceph-node2-ssd root=ssds
In the ceph configuration file (ceph.conf) set the crush location for the SSD OSDs. This is necessary because the default location is always the hostname obtained with the command hostname -s
[osd.35] host = ceph-node1 osd_journal = /dev/disk/by-id/ata-INTEL_SSDSC2BB016T6_BTWA543204R11P6KGN-part1 crush_location = root=ssds host=ceph-node1-ssd
You can check the location of the osd running this command:
$ ceph-crush-location --id 35 --type osd root=ssds host=ceph-node1-ssd
For each new ssd OSD move the osd to ssds root:
ceph osd crush add 35 1.5 root=ssds ceph osd crush set osd.35 1.5 root=ssds host=ceph-node1-ssd
Create a new SSD pool:
ceph osd pool create ssdpool 128 128
Crate a crush rule in the ssds root:
ceph osd crush rule create-simple ssdpool ssds host
Finally assign the new rule to the ssdpool:
$ ceph osd pool set ssdpool crush_ruleset 4 set pool 5 crush_ruleset to 4
Got! Now, we have a new only SSD pool:
$ ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -25 3.00000 root ssds -26 1.50000 host ceph-node1-ssd 35 1.50000 osd.35 up 1.00000 1.00000 -27 1.50000 host ceph-node2-ssd 36 1.50000 osd.36 up 1.00000 1.00000 -21 48.22978 root sata -24 6.50995 host ceph-node1 1 0.92999 osd.1 up 1.00000 1.00000 5 0.92999 osd.5 up 1.00000 1.00000 16 0.92999 osd.16 up 1.00000 1.00000 17 0.92999 osd.17 up 1.00000 1.00000 18 0.92999 osd.18 up 1.00000 1.00000 19 0.92999 osd.19 up 1.00000 1.00000 20 0.92999 osd.20 up 1.00000 1.00000 -22 6.50995 host ceph-node2 21 0.92999 osd.21 up 1.00000 1.00000 22 0.92999 osd.22 up 1.00000 1.00000 23 0.92999 osd.23 up 1.00000 1.00000 24 0.92999 osd.24 up 1.00000 1.00000 25 0.92999 osd.25 up 1.00000 1.00000 26 0.92999 osd.26 up 1.00000 1.00000 27 0.92999 osd.27 up 1.00000 1.00000 -7 13.29996 host ceph-node3 0 1.89999 osd.0 up 1.00000 1.00000 6 1.89999 osd.6 up 1.00000 1.00000 9 1.89999 osd.9 up 1.00000 1.00000 11 1.89999 osd.11 up 1.00000 1.00000 14 1.89999 osd.14 up 1.00000 1.00000 15 1.89999 osd.15 up 1.00000 1.00000 2 1.89999 osd.2 up 1.00000 1.00000 -20 6.50995 host ceph-node4 28 0.92999 osd.28 up 1.00000 1.00000 29 0.92999 osd.29 up 1.00000 1.00000 30 0.92999 osd.30 up 1.00000 1.00000 31 0.92999 osd.31 up 1.00000 1.00000 32 0.92999 osd.32 up 1.00000 1.00000 33 0.92999 osd.33 up 1.00000 1.00000 34 0.92999 osd.34 up 1.00000 1.00000 -14 15.39998 host ceph-node5 3 2.20000 osd.3 up 1.00000 1.00000 4 2.20000 osd.4 up 1.00000 1.00000 7 2.20000 osd.7 up 1.00000 1.00000 8 2.20000 osd.8 up 1.00000 1.00000 10 2.20000 osd.10 up 1.00000 1.00000 12 2.20000 osd.12 up 1.00000 1.00000 13 2.20000 osd.13 up 1.00000 1.00000 [...]