生活的道路一旦选定，就要勇敢地走到底，决不回头。——左拉
坚强的信心，能使平凡的人做出惊人的事业。——马尔顿
人不可有傲气，但不可无傲骨。 --徐悲鸿
古之立大志者，不惟有超世之才，亦必有坚韧不拔之志。 --苏轼
时间像海绵里的水，只要你愿意挤，总还是有的。 --鲁迅

> DevOps > ELK > Elasticsearch数据备份与恢复

Elasticsearch数据备份与恢复

ELK zkinogg 1年前 (2020-08-12) 220次浏览 0个评论

一.本地备份与恢复

# 1.下载nodejs安装包
[root@db04 ~]# wget https://mirrors.sjtug.sjtu.edu.cn/nodejs-release/v10.13.0/node-v10.13.0-linux-x64.tar.gz

# 2.解压并创建软链接
[root@db04 ~]# tar xf node-v10.13.0-linux-x64.tar.gz
[root@db04 ~]# ln -s ~/node-v10.13.0-linux-x64/bin/node /usr/bin/node
[root@db04 ~]# ln -s ~/node-v10.13.0-linux-x64/bin/npm /usr/bin/npm

# 3.更换npm源
[root@db04 ~]# npm config set registry https://registry.npm.taobao.org

# 4.安装elasticdump
[root@db04 ~]# npm install elasticdump -g

# 5.进入elasticdump
[root@db04 ~]# cd /root/node-v10.13.0-linux-x64/lib/node_modules/elasticdump/bin



# 6.本地目录备份

## 第一次进行的是索引的备份
## 第二次进行的是数据的备份
[root@db04 bin]# ./elasticdump --input=http://10.0.0.54:9200/student/user --output=/tmp/student_mapping.json --type=mapping
Wed, 12 Aug 2020 07:41:59 GMT | starting dump
Wed, 12 Aug 2020 07:41:59 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 07:41:59 GMT | sent 1 objects to destination file, wrote 1
Wed, 12 Aug 2020 07:41:59 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 07:41:59 GMT | Total Writes: 1
Wed, 12 Aug 2020 07:41:59 GMT | dump complete
[root@db04 bin]# ./elasticdump --input=http://10.0.0.54:9200/student/user --output=/tmp/student_data.json --type=data
Wed, 12 Aug 2020 07:42:21 GMT | starting dump
Wed, 12 Aug 2020 07:42:21 GMT | got 8 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 07:42:21 GMT | sent 8 objects to destination file, wrote 8
Wed, 12 Aug 2020 07:42:21 GMT | got 0 objects from source elasticsearch (offset: 8)
Wed, 12 Aug 2020 07:42:21 GMT | Total Writes: 8
Wed, 12 Aug 2020 07:42:21 GMT | dump complete


##备份之后用Elasticsearch Head插件模拟数据丢失，误删除student索引和数据并用elasticdump恢复


[root@db04 bin]# ./elasticdump --output=http://10.0.0.54:9200/student --input=/tmp/student_mapping.json --type=mapping
Wed, 12 Aug 2020 07:46:21 GMT | starting dump
Wed, 12 Aug 2020 07:46:21 GMT | got 1 objects from source file (offset: 0)
Wed, 12 Aug 2020 07:46:21 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 07:46:21 GMT | got 0 objects from source file (offset: 1)
Wed, 12 Aug 2020 07:46:21 GMT | Total Writes: 1
Wed, 12 Aug 2020 07:46:21 GMT | dump complete
[root@db04 bin]# ./elasticdump --output=http://10.0.0.54:9200/student --input=/tmp/student_data.json --type=data
Wed, 12 Aug 2020 07:46:40 GMT | starting dump
Wed, 12 Aug 2020 07:46:40 GMT | got 8 objects from source file (offset: 0)
Wed, 12 Aug 2020 07:46:40 GMT | sent 8 objects to destination elasticsearch, wrote 8
Wed, 12 Aug 2020 07:46:40 GMT | got 0 objects from source file (offset: 8)
Wed, 12 Aug 2020 07:46:40 GMT | Total Writes: 8
Wed, 12 Aug 2020 07:46:40 GMT | dump complete

#  根据索引名和类型进行备份

#备份成功。数据恢复成功

二.备份到另一台ES节点（如果是同一集群的节点索引不能重名）

# 备份到另一台ES节点（同一集群索引不能重名）
[root@db04 bin]# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=analyzer
Wed, 12 Aug 2020 08:14:00 GMT | starting dump
Wed, 12 Aug 2020 08:14:00 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:01 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 08:14:01 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 08:14:01 GMT | Total Writes: 1
Wed, 12 Aug 2020 08:14:01 GMT | dump complete
[root@db04 bin]# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=mapping
Wed, 12 Aug 2020 08:14:10 GMT | starting dump
Wed, 12 Aug 2020 08:14:10 GMT | got 1 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:10 GMT | sent 1 objects to destination elasticsearch, wrote 1
Wed, 12 Aug 2020 08:14:10 GMT | got 0 objects from source elasticsearch (offset: 1)
Wed, 12 Aug 2020 08:14:10 GMT | Total Writes: 1
Wed, 12 Aug 2020 08:14:10 GMT | dump complete
[root@db04 bin]# ./elasticdump --input=http://10.0.0.54:9200/student --output=http://10.0.0.55:9200/students --type=data
Wed, 12 Aug 2020 08:14:15 GMT | starting dump
Wed, 12 Aug 2020 08:14:15 GMT | got 8 objects from source elasticsearch (offset: 0)
Wed, 12 Aug 2020 08:14:15 GMT | sent 8 objects to destination elasticsearch, wrote 8
Wed, 12 Aug 2020 08:14:15 GMT | got 0 objects from source elasticsearch (offset: 8)
Wed, 12 Aug 2020 08:14:15 GMT | Total Writes: 8
Wed, 12 Aug 2020 08:14:15 GMT | dump complete

三.本地备份与恢复脚本

# 1.本地备份脚本
[root@db04 ~]# vim output_Es.sh
#!/bin/bash
read -p '要备份的机器是：'${1}
#要导出的索引名
index_name='
student
'
for index in `echo $index_name`
do
    echo "start input index ${index}"
    elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_alias.json --type=alias &> /dev/null
    elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_analyzer.json --type=analyzer &> /dev/null
    elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_data.json --type=data &> /dev/null
    elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_alias.json --type=alias &> /dev/null
    elasticdump --input=http://${1}:9200/${index} --output=/data/${index}_template.json --type=template &> /dev/null
done

# 2.恢复脚本
[root@db04 ~]# vim input_Es.sh
#!/bin/bash
read -p '要导入的机器是：'${1}
#要导入的索引名
index_name='
student
'
for index in `echo $index_name`
do
    echo "start input index ${index}"
    elasticdump --input=/data/${index}_alias.json --output=http://${1}:9200/${index} --type=alias &> /dev/null
    elasticdump --input=/data/${index}_analyzer.json --output=http://${1}:9200/${index} --type=analyzer &> /dev/null
    elasticdump --input=/data/${index}_data.json --output=http://${1}:9200/${index} --type=data &> /dev/null
    elasticdump --input=/data/${index}_template.json --output=http://${1}:9200/${index} --type=template &> /dev/null
done

测试脚本

# 0.配置环境变量
[root@db04 ~]# vim /etc/profile.d/elasticdump.sh 
export PATH=/root/node-v10.13.0-linux-x64/lib/node_modules/elasticdump/bin:$PATH
[root@db04 ~]# source /etc/profile


# 1.创建备份目录
[root@db04 ~]# mkdir /data

# 2.执行备份脚本
[root@db04 ~]# sh output_Es.sh 
要备份的机器是：10.0.0.54
start input index student

# 3.查看备份目录
[root@db04 ~]# ll /data/
total 16
-rw-r--r-- 1 root root    5 Aug 12 16:59 student_alias.json
-rw-r--r-- 1 root root  101 Aug 12 16:59 student_analyzer.json
-rw-r--r-- 1 root root 1284 Aug 12 16:59 student_data.json
-rw-r--r-- 1 root root    5 Aug 12 16:59 student_template.json

# 模拟误删除操作

# 执行恢复脚本
[root@db04 ~]#  sh input_Es.sh 
要导入的机器是：10.0.0.54
start input index student


## 数据恢复

极客公园 , 版权所有丨如未注明 , 均为原创丨本网站采用BY-NC-SA协议进行授权
转载请注明原文链接：Elasticsearch数据备份与恢复

您必须登录才能发表评论！