Share volume between machines-白红宇

共享存储卷在集群中应用广泛, 一般用作集群文件系统, 例如rhcs套件中间gfs.

但是如果不想使用集群文件系统的话, 同一时间不能同时被2台或者2台以上的系统挂载. 否则可能将文件系统破坏.

使用逻辑卷来管理是比较好的选择之一.

例如存储上划分了1个卷, 同时指派给2台主机.

在2台主机上都能看到这个存储卷.

例如 :

A主机 :

[root@db-xxx-xxx-xxx-xxx ~]# multipath -llmsa_60_vd01vol01 (3600c0ff000da60c23d8a375201000000) dm-3 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:4:1 sdh 8:112 [active][ready] \_ 1:0:4:1 sds 65:32 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:3:1 sdf 8:80  [active][ready] \_ 1:0:3:1 sdq 65:0  [active][ready]msa_62_vd01vol01 (3600c0ff000da60e2f705385201000000) dm-5 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:6:1 sdk 8:160 [active][ready] \_ 1:0:6:1 sdv 65:80 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:5:1 sdj 8:144 [active][ready] \_ 1:0:5:1 sdu 65:64 [active][ready]msa_58_vd01vol01 (3600c0ff000da61d7258a375201000000) dm-1 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:2:1 sdd 8:48  [active][ready] \_ 1:0:2:1 sdo 8:224 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:1:1 sdb 8:16  [active][ready] \_ 1:0:1:1 sdm 8:192 [active][ready]msa_60_vd02vol01 (3600c0ff000da60ca488a375201000000) dm-4 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:3:2 sdg 8:96  [active][ready] \_ 1:0:3:2 sdr 65:16 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:4:2 sdi 8:128 [active][ready] \_ 1:0:4:2 sdt 65:48 [active][ready]msa_58_vd02vol01 (3600c0ff000da6318358a375201000000) dm-2 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:1:2 sdc 8:32  [active][ready] \_ 1:0:1:2 sdn 8:208 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:2:2 sde 8:64  [active][ready] \_ 1:0:2:2 sdp 8:240 [active][ready]msa_57_vd01vol01 (3600c0ff000da62c0be8d375201000000) dm-0 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:0:1 sda 8:0   [active][ready] \_ 1:0:0:1 sdl 8:176 [active][ready]

B主机 :

[root@db-xxx-xxx-xxx-xxx ~]# multipath -llmsa_60_vd01vol01 (3600c0ff000da60c23d8a375201000000) dm-3 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:4:1 sdh 8:112 [active][ready] \_ 1:0:4:1 sds 65:32 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:2:1 sdd 8:48  [active][ready] \_ 1:0:2:1 sdo 8:224 [active][ready]msa_62_vd01vol01 (3600c0ff000da60e2f705385201000000) dm-5 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:5:1 sdj 8:144 [active][ready] \_ 1:0:6:1 sdv 65:80 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:6:1 sdk 8:160 [active][ready] \_ 1:0:5:1 sdu 65:64 [active][ready]msa_58_vd01vol01 (3600c0ff000da61d7258a375201000000) dm-1 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:3:1 sdf 8:80  [active][ready] \_ 1:0:1:1 sdm 8:192 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:1:1 sdb 8:16  [active][ready] \_ 1:0:3:1 sdq 65:0  [active][ready]msa_60_vd02vol01 (3600c0ff000da60ca488a375201000000) dm-4 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:2:2 sde 8:64  [active][ready] \_ 1:0:2:2 sdp 8:240 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:4:2 sdi 8:128 [active][ready] \_ 1:0:4:2 sdt 65:48 [active][ready]msa_58_vd02vol01 (3600c0ff000da6318358a375201000000) dm-2 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:1:2 sdc 8:32  [active][ready] \_ 1:0:3:2 sdr 65:16 [active][ready]\_ round-robin 0 [prio=20][enabled] \_ 0:0:3:2 sdg 8:96  [active][ready] \_ 1:0:1:2 sdn 8:208 [active][ready]msa_57_vd01vol01 (3600c0ff000da62c0be8d375201000000) dm-0 HP,MSA2312fc[size=1.4T][features=1 queue_if_no_path][hwhandler=0][rw]\_ round-robin 0 [prio=100][active] \_ 0:0:0:1 sda 8:0   [active][ready] \_ 1:0:0:1 sdl 8:176 [active][ready]

在A主机上创建pv, vg, lv. 并格式文件系统.

创建完后, 在B主机上扫描 :

pvscanvgscanlvscan

扫描完后可以看到创建的pv, vg, lv.

此时在B主机上, lv处于未激活状态 :

[root@db-xxx-xxx-xxx-xxx ~]# lvs  LV   VG       Attr   LSize Origin Snap%  Move Log Copy%  Convert  lv01 vgdata01 -wi--- 8.17T[root@db-xxx-xxx-xxx-xxx ~]# ll /dev/mapper/vgdata01-lv01ls: /dev/mapper/vgdata01-lv01: No such file or directory

而在A主机上, lv处于激活状态 :

[root@db-xxx-xxx-xxx-xxx ~]# lvs  LV   VG       Attr   LSize Origin Snap%  Move Log Copy%  Convert  lv01 vgdata01 -wi-a- 8.17T[root@db-xxx-xxx-xxx-xxx ~]# ll /dev/mapper/vgdata01-lv01 brw-rw---- 1 root disk 253, 6 Sep 17 08:29 /dev/mapper/vgdata01-lv01

在B主机上激活逻辑卷的命令,

激活后map device 被自动创建 :

[root@db-xxx-xxx-xxx-xxx ~]# lvchange -ay vgdata01/lv01[root@db-xxx-xxx-xxx-xxx ~]# ll /dev/mapper/vgdata01-lv01brw-rw---- 1 root disk 253, 6 Sep 17 08:54 /dev/mapper/vgdata01-lv01

此时可以将A主机卸载,

[root@db-A ~]# umount /dev/mapper/vgdata01-lv01

然后在B主机挂载该文件系统

[root@db-B ~]# mount /dev/mapper/vgdata01-lv01 /database1

在任意主机上创建的文件, 在另一主机挂载该卷后可以正常访问.

[root@db-B database1]# date +%F%T >>./test[root@db-B database1]# lltotal 20drwx------ 2 root root 16384 Sep 17 08:29 lost+found-rw-r--r-- 1 root root    19 Sep 17 08:57 test[root@db-B database1]# cat test2013-09-1708:57:22

将B卸载并冻结lv后, 在A挂载.

[root@db-B ~]# umount /database1[root@db-B ~]# lvchange -an vgdata01/lv01[root@db-A ~]# lvchange -ay vgdata01/lv01[root@db-A ~]# mount /dev/mapper/vgdata01-lv01 /database[root@db-A ~]# cd /database/[root@db-A database]# lltotal 20drwx------ 2 root root 16384 Sep 17 08:29 lost+found-rw-r--r-- 1 root root    19 Sep 17 08:57 test[root@db-A database]# cat test2013-09-1708:57:22

[注意]

对于不使用集群文件系统共享卷的情况, 请务必小心, 切忌不要同时挂载该文件系统.

2. 不建议使用fstab自动挂载这类文件系统. 同时确保只有一个系统中的lv处于激活状态, 避免误挂载.

[参考]

1. man lvs

The lv_attr bits are:              1  Volume  type:  (m)irrored, (M)irrored without initial sync, (o)rigin, (O)rigin with merging snapshot,                 (s)napshot, merging (S)napshot, (p)vmove, (v)irtual,  mirror  (i)mage,  mirror  (I)mage  out-of-sync,                 under (c)onversion              2  Permissions: (w)riteable, (r)ead-only              3  Allocation  policy:  (c)ontiguous,  c(l)ing, (n)ormal, (a)nywhere, (i)nherited This is capitalised if                 the volume is currently locked against allocation changes, for example during pvmove (8).              4  fixed (m)inor              5  State: (a)ctive, (s)uspended, (I)nvalid  snapshot,  invalid  (S)uspended  snapshot,  mapped  (d)evice                 present without tables, mapped device present with (i)nactive table              6  device (o)pen