swift分布式存储添加存储节点

swift分布式存储添加节点需要新的节点信息加入到ring中记录下来

1、在ring中添加两个节点的信息

swift@server1:/etc/swift> export ZONE=4

swift@server1:/etc/swift> export STORAGE_LOCAL_NET_IP=192.168.1.215

swift@server1:/etc/swift> export WEIGHT=100

swift@server1:/etc/swift> export DEVICE=sdb1

swift@server1:/etc/swift> sudo swift-ring-builder account.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6002/$DEVICE $WEIGHT

swift@server1:/etc/swift> sudo swift-ring-builder container.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6001/$DEVICE $WEIGHT

swift@server1:/etc/swift> sudo swift-ring-builder object.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6000/$DEVICE $WEIGHT

swift@server1:/etc/swift> export ZONE=5

swift@server1:/etc/swift> export STORAGE_LOCAL_NET_IP=192.168.1.216

swift@server1:/etc/swift> sudo swift-ring-builder account.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6002/$DEVICE $WEIGHT

swift@server1:/etc/swift> sudo swift-ring-builder container.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6001/$DEVICE $WEIGHT

swift@server1:/etc/swift> sudo swift-ring-builder object.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6000/$DEVICE $WEIGHT

查看ring的信息

swift@server1:/etc/swift> sudo swift-ring-builder account.builder

account.builder,build version 38

262144 partitions,3.000000 replicas,1 regions,5 zones,5 devices,100.00 balance

The minimum number of hours before a partition can be reassigned is 1

Devices: id region zone ip address port replication ip replication port name weight partitions balance meta

13 1 1 192.168.1.212 6002 192.168.1.212 6002 sdb1 100.00 262144 66.67

14 1 2 192.168.1.213 6002 192.168.1.213 6002 sdb1 100.00 262144 66.67

15 1 3 192.168.1.214 6002 192.168.1.214 6002 sdb1 100.00 262144 66.67

16 1 4 192.168.1.215 6002 192.168.1.215 6002 sdb1 100.00 0 -100.00

17 1 5 192.168.1.216 6002 192.168.1.216 6002 sdb1 100.00 0 -100.00

swift@server1:/etc/swift> sudo swift-ring-builder container.builder

container.builder,build version 13

262144 partitions,100.00 balance

The minimum number of hours before a partition can be reassigned is 1

Devices: id region zone ip address port replication ip replication port name weight partitions balance meta

3 1 1 192.168.1.212 6001 192.168.1.212 6001 sdb1 100.00 262144 66.67

4 1 2 192.168.1.213 6001 192.168.1.213 6001 sdb1 100.00 262144 66.67

5 1 3 192.168.1.214 6001 192.168.1.214 6001 sdb1 100.00 262144 66.67

6 1 4 192.168.1.215 6001 192.168.1.215 6001 sdb1 100.00 0 -100.00

7 1 5 192.168.1.216 6001 192.168.1.216 6001 sdb1 100.00 0 -100.00

swift@server1:/etc/swift> sudo swift-ring-builder object.builder

object.builder,100.00 balance

The minimum number of hours before a partition can be reassigned is 1

Devices: id region zone ip address port replication ip replication port name weight partitions balance meta

3 1 1 192.168.1.212 6000 192.168.1.212 6000 sdb1 100.00 262144 66.67

4 1 2 192.168.1.213 6000 192.168.1.213 6000 sdb1 100.00 262144 66.67

5 1 3 192.168.1.214 6000 192.168.1.214 6000 sdb1 100.00 262144 66.67

6 1 4 192.168.1.215 6000 192.168.1.215 6000 sdb1 100.00 0 -100.00

7 1 5 192.168.1.216 6000 192.168.1.216 6000 sdb1 100.00 0 -100.00

2、平衡ring

swift@server1:/etc/swift> sudo swift-ring-builder account.builder rebalance

Reassigned 262144 (100.00%) partitions. Balance is now 16.67.

-------------------------------------------------------------------------------

NOTE: Balance of 16.67 indicates you should push this

ring,wait at least 1 hours,and rebalance/repush.

-------------------------------------------------------------------------------

swift@server1:/etc/swift> sudo swift-ring-builder container.builder rebalance

Reassigned 262144 (100.00%) partitions. Balance is now 16.67.

-------------------------------------------------------------------------------

NOTE: Balance of 16.67 indicates you should push this

ring,and rebalance/repush.

-------------------------------------------------------------------------------

swift@server1:/etc/swift> sudo swift-ring-builder object.builder rebalance

Reassigned 262144 (100.00%) partitions. Balance is now 16.67.

-------------------------------------------------------------------------------

NOTE: Balance of 16.67 indicates you should push this

ring,and rebalance/repush.

-------------------------------------------------------------------------------

swift@server1:/etc/swift> sudo chown swift:swift *

拷贝ring文件到所有的存储节点

3、配置添加的存储节点

参考之前的文章配置配置215和216，注意目录和文件的权限

启动215和216的服务

之后记得其他节点重启服务

proxy节点

swift@server1:/etc/swift> sudo swift-init proxy restart

Signal proxy-server pid: 23568 signal: 15

proxy-server (23568) appears to have stopped

Starting proxy-server...(/etc/swift/proxy-server.conf)

storage节点

# swift-init all restart

注意

server5:/ # chown -R swift:swift /srv/node/sdb1

server6:/ # chown -R swift:swift /srv/node/sdb1

刚开始前三台存储节点上的数据

swift@server2:/srv/node/sdb1> du -sh *

16K accounts

356M async_pending

129M containers

5.0G objects

0 tmp

添加后两台存储节点之后的数据分布

wift@server2:/srv/node/sdb1> du -sh *

16K accounts

197M async_pending

129M containers

2.5G objects

0 tmp

swift@server3:/srv/node/sdb1> du -sh *

16K accounts

175M async_pending

129M containers

2.5G objects

0 tmp

swift@server4:/srv/node/sdb1> du -sh *

0 accounts

244M async_pending

129M containers

3.0G objects

0 tmp

server5:/srv/node/sdb1 # du -sh *

68M containers

2.7G objects

0 tmp

server6:/srv/node/sdb1 # du -sh *

16K accounts

32K containers

2.8G objects

0 tmp