First, zk data migration, there are the following two options:

1, the use of zk cluster more than half of the characteristics of still available, such as the cluster has 5 nodes, you can cut out 1 ~ 2 of the nodes, and then add a new node, the composition of the new cluster, in order to realize the migration of data;

2, direct Copy the metadata files of the cluster to the new cluster;

But the 1st option is not the best choice, for example, zk cluster connection number load is high, if you reduce the number of nodes at this time, it will lead to the cluster load becomes higher, or even the cluster crash. Therefore, the 2nd option is used to realize cluster data migration and recovery by copying metadata.

The idea of zk data migration and recovery

1, build a new zk cluster and start the cluster (at this time, there is no data in the cluster);

2, stop the zk process on all nodes of the new cluster;

3, delete files in the data directory of all nodes of the new cluster, including transaction logs, snapshots, epoch files
4, delete files in the data directory of all nodes of the new cluster.

4. Copy the transaction logs, snapshots, and epoch files of the leader node of the old cluster to the corresponding data directories of all the nodes of the new cluster;

5. Restart the new cluster;

Third, considerations:

If the two epoch files of the new cluster are not deleted, it will result in the new cluster not being able to start; the reason is: If you just copy the snapshots and transaction logs from the old cluster to the new cluster, the nodes in the new cluster will recognize the current epoch value recorded in the epoch file when they start up, and then compare this epoch value with the transaction ID (zxid) in the metadata copied from the old cluster, and find that it doesn't match, which will result in the new cluster not being able to start up normally. Therefore, you need to delete the epoch file of each node in the new cluster, and copy the epoch file, snapshot file, and transaction log file of the old cluster to each node in the new cluster.

Four, zk data migration and recovery of the specific operation steps:

1, build a new cluster:

1), rpm?-ivh?jdk-8u20-linux-x64.rpm?

2), cd?/data/? &&?tar?-zxvf?zk_server.tgz?#### Extract to /data or /data1

3), cd?/data/? &&?mv?zk_server?zk.1?### For nodes with myid 1, the home directory is /data/zk.1, and for nodes with myid 2, the home directory is /data/zk.2

4), after unpacking, you can see 3 directories:

cd?/data/zk.1?& ;&?ls? ;&?ls?-l

zk_data?#### The main directory where zk snapshot data is stored

zk_log### The main directory where zk transaction logs are stored

zookeeper### The program path that contains the configuration files

5), cd?/data/zk.1/zk_ data?&&?echo?1?>?myid?### Configure node myid, nodes with myid 1 are configured as 1, nodes with myid 2 are configured as 2, and nodes with myid 3 are configured as 3

6), cd?/data/zookeeper/conf?&&?cp? amp;?cp?-ar?zoo.cfg.template?zoo.cfg

7), vim?zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

clientPort=2181

autopurge.snapRetainCount=500

autopurge.purgeInterval?=?48?

dataDir=/data/zk.1/zk_data##### myid as 2 then configured as /data/zk.2/zk_data

dataLogDir=/data/zk.1/zk_log?#### myid is 2 then configured as /data/zk.2/zk_log

server.1=Node 1's IP:8880:7770? >

server.2=IP:8880:7770 for node 2 ? # configuration of node 2

server.3=IP:8880:7770 of node 3 # configuration of node 3

8), the installation and deployment of the remaining two nodes is the same

9), start the three nodes in turn and check the status

Startup:

cd?/data/zk.1/ zookeeper/bin/? &&nohup?sh?zkServer.sh?start?>?zookeeper.out?&

Check node status:

cd?/data/zk.1/zookeeper/bin/? &&? /zkServer.sh?status?

Connect to the local node and view the data:

cd?/data/zk.1/zookeeper/bin/? &&? /zkCli.sh?-server?127.0.0.1:2181

2. Stop the zk process on all nodes in the new cluster:

cd?/data/zk.1/zookeeper/bin/? &&?sh?zkServer.sh?stop

cd?/data/zk.2/zookeeper/bin/? &&?sh?zkServer.sh?stop

cd?/data/zk.3/zookeeper/bin/? &&?sh?zkServer.sh?stop

3. Delete the files in the data directory of all nodes in the new cluster, including: transaction logs, snapshots, epoch files (take node 1 as an example):

cd?/data/zk.1/zk_data/version-2?&&. amp;?rm?-f?snapshot.*? &&?rm?-f?acceptedEpoch?&&?rm?-f?currentEpoch

cd?/data/zk.1/zk_log/version-2?&&?rm?-f?log.*

4. Move the Copy the transaction log, snapshot, and epoch files from the old cluster leader node to the corresponding data directories of all nodes in the new cluster (whichever is the leader node's data):

1), Backup the files in the data directory of the old cluster leader node (copy them to the newly created zk_meta_dir directory locally)

Latest log transaction log files? #### Not all log files, but the latest log file

Latest snapshot file #### Not all snapshot files, but the latest snapshot file

acceptedEpoch file

currentEpoch file

2) Distribute the log file, snapshot file, and Epoch file from the leader node's zk_meta_dir directory to the directories corresponding to each node of the new cluster, for example, /data/zk.1/zk_data/version-2, /data/zk.1/zk_log/version-2 of node 1