Linux下sersync数据实时同步
sersync其实是利用inotify和rsync两种软件技术来实现数据实时同步功能的,inotify是用于监听sersync所在服务器上的文件变化,结合rsync软件来进行数据同步,将数据实时同步给客户端服务器。
工作过程:在同步主服务器上开启sersync,负责监听文件系统的变化,然后调用rsync命令把更新的文件同步到目标服务器上,主服务器上安装sersync软件,目标服务器上安装rsync服务。
1、客户端配置
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
[root@localhost2 ~] # cat /etc/rsyncd.conf
##created by cai at 2018-2-24
uid= rsync
gid= rsync
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd .pid
lock file = /var/run/rsync .lock
log file = /var/log/rsyncd .log
[data]
path = /data/
ignore errors
read only = false
list = false
hosts allow = 192.168.181.128:52000 /24
hosts deny = 0.0.0.0 /32
auth users = rsync_body
secrets file = /etc/rsync .password
[root@localhost2 ~] # ls -ld /data/
drwxrwxrwx. 3 rsync rsync 4096 feb 24 16:58 /data/
[root@localhost2 ~] # cat /etc/rsync.password
rsync_body:admin
[root@localhost2 ~] # ls -ld /etc/rsync.password
-rw-------. 1 root root 17 feb 24 16:26 /etc/rsync .password #600权限
[root@localhost2 ~] # netstat -lntup| grep "rsync"
tcp 0 0 0.0.0.0:873 0.0.0.0:* listen 1497 /rsync
tcp 0 0 :::873 :::* listen 1497 /rsync
|
2、主服务器配置
1
2
3
4
|
[root@localhost1 ~] # cat /etc/rsync.password
admin
[root@localhost1 ~] # ls -ld /etc/rsync.password
-rw------- 1 root root 6 feb 24 03:54 /etc/rsync .password
|
3、安装sersync服务
采用inotify来对文件进行监控,当监控到文件有文件发生改变的时候,就会调用rsync实现触发式实时同步!
安装sersync(注意sersync是工作在rsync的源服务器上,也就是客户端上)
1
2
3
4
5
6
7
8
9
10
11
|
[root@salt-client01 ~] # cd /usr/local/src/
[root@salt-client01 src] # tar -zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz -c /usr/local/
gnu-linux-x86/
gnu-linux-x86 /sersync2
gnu-linux-x86 /confxml .xml
[root@salt-client01 src] # cd /usr/local/
[root@salt-client01 local ] # mv gnu-linux-x86 sersync
[root@salt-client01 local ] # cd sersync/
[root@salt-client01 sersync] # mkdir conf bin log
[root@salt-client01sersync] # mv confxml.xml conf
[root@salt-client01sersync] # mv sersync2 bin/sersync
|
修改配置文件
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
***********************************30行开始******************************
<commonparams params= "-artuz" /> #-artuz为rsync同步时的参数
<authstart= "true" users = "rsync的虚拟用户名(rsync_backup)" passwordfile= "rsync的密码文件" />
<userdefinedport start= "true" port= "873" /><!-- port=874 -->
<timeout start= "false" time = "100" /><!--timeout=100 -->
<sshstart= "false" />
************************************第36行***********************************
<faillogpath= "自己定义的log文件夹(/usr/local/sersync/log)rsync_fail_log.sh"
timetoexecute= "60" /><!--defaultevery 60mins execute once-->
*******************************************************************************
*注:若有多个目录备份可以穿件多个配置文件在启动时的-o参数中添加即可
[root@salt-client01 conf] # diff confxml.xml confxml.xml.bak
24,25c24,25
< <localpath watch = "/data/" > #data就是本地需要同步的文件夹到服务器端的目录
< <remote ip= "192.168.91.166" name= "data" /> #data (server的模块名)是rsync 服务端的文件夹,也就是推送到服务器端的目标文件夹,可以配置多个,
---
> <localpath watch = "/opt/tongbu" >
> <remote ip= "127.0.0.1" name= "tongbu1" />
31c31
< <auth start= "true" users = "rsync_body" passwordfile= "/etc/rsync.password" /> #true 才能生效,rsync_body同步时候虚拟账号,后面是密码文件
---
> <auth start= "false" users = "root" passwordfile= "/etc/rsync.pas" />
33c33
< <timeout start= "true" time = "100" /><!-- timeout=100 --> #true 才能生效
---
> <timeout start= "false" time = "100" /><!-- timeout=100 -->
36c36
< <faillog path= "/usr/local/sersync/log/rsync_fail_log.sh" timetoexecute= "60" /><!--default every 60mins execute once--> #检测rsync进程判断,没有自动启
---
> <faillog path= "/tmp/rsync_fail_log.sh" timetoexecute= "60" /><!--default evepry 60mins execute once-->
|
启动sersync
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
|
[root@salt-client01 src] # echo 'export path=$path:/usr/local/sersync/bin'>>/etc/profile #声明环境变量
[root@salt-client01 src] # source /etc/profile
[root@salt-client01 src] # sersync2 -r -d -o /usr/local/sersync/conf/confxml.xml #启动
set the system param
execute: echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute: echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param
option: -r rsync all the local files to the remote servers before the sersync work
option: -d run as a daemon
option: -o config xml name: /usr/local/sersync/conf/confxml .xml
daemon thread num: 10
parse xml config file
host ip : localhost host port: 8008
daemon start,sersync run behind the console
use rsync password- file :
user is rsync_body
passwordfile is /etc/rsync .password
config xml parse success
please set /etc/rsyncd .conf max connections=0 manually
sersync working thread 12 = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads)
max threads numbers is: 22 = 12(thread pool nums) + 10(sub threads)
please according your cpu ,use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command : cd /data && rsync -artuz -r --delete ./ --timeout=100 rsync_body@192.168.91.166::data --password- file = /etc/rsync .password > /dev/null 2>&1
run the sersync:
watch path is: /data #此时可以看出sersync已经启动成功了
#检测脚本
[root@salt-client01 log] # pwd
/usr/local/sersync/log
[root@salt-client01 log] # vim rsync_fail_log.sh
[root@salt-client01 log] # chmod +x rsync_fail_log.sh
[root@salt-client01 ~] # cat /usr/local/sersync/log/rsync_fail_log.sh
#!/bin/bash
#purpose: check sersync whether it is alive
#author: cai meng zhi
sersync= "/usr/local/sersync/bin/sersync2"
conf_file= "/usr/local/sersync/conf/confxml.xml"
status=$( ps aux | grep 'sersync2' | grep - v 'grep' | wc -l)
if [ $status - eq 0 ];
then
$sersync -d -r -o $conf_file &
else
exit 0;
fi
脚本写好以后,添加到计划任务中去
* /1 * * * * /bin/bash /usr/local/sersync/log/rsync_fail_log .sh > /dev/null 2>&1
测试同步:
客户端新增文件
[root@salt-client01 data] # cp /etc/passwd 192.168.91.156.passwd
[root@salt-client01 data] # ll
total 4
-rw-r--r-- 1 root root 1928 nov 10 18:15 192.168.91.156. passwd
-rw-r--r-- 1 root root 0 nov 10 17:27 3
服务端检测
[root@salt-master data] # cd /data/
[root@salt-master data] # ll
total 8
-rw-r--r-- 1 root root 1928 nov 10 18:15 192.168.91.156. passwd #说明已经同步过来了
-rw-r--r-- 1 root root 0 nov 10 17:27 3
drwxr-xr-x 2 root root 4096 nov 10 17:27 data
客户端测试删除
[root@salt-client01 data] # rm rf 192.168.91.156.passwd
rm : cannot remove `rf': no such file or directory
rm : remove regular file `192.168.91.156. passwd '? y
[root@salt-client01 data] # ll
total 0
-rw-r--r-- 1 root root 0 nov 10 17:27 3
[root@salt-client01 data] #
服务器端:
[root@salt-master data] # ll
total 4
-rw-r--r-- 1 root root 0 nov 10 17:27 3 #说明已经删除掉了
drwxr-xr-x 2 root root 4096 nov 10 17:27 data
|
4、常见错误汇总
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
错误一:
@error: auth failed on module xxxxx
rsync : connection unexpectedly closed(90 bytes read so far)
rsync error: error in rsync protocoldata stream (code 12) at io.c(150)
说明:这是因为密码设置错了,无法登入成功,检查一下 rsync . pwd ,看客服是否匹配。还有服务器端没启动 rsync 服务也会出现这种情况。
错误二:
password file must not beother-accessible
continuing without password file
password:
说明:这是因为rsyncd.pwdrsyncd.sec的权限不对,应该设置为600。如:chmod600 rsyncd. pwd
错误三:
@error: chroot failed
rsync : connection unexpectedly closed(75 bytes read so far)
rsync error: error in rsync protocoldata stream (code 12) at io.c(150)
说明:这是因为你在 rsync .conf中设置的 path 路径不存在,要新建目录才能开启同步
错误四:
rsync : failed to connect to218.107.243.2: no route to host (113)
rsync error: error in socket io (code10) at clientserver.c(104) [receiver=2.6.9]
说明:防火墙问题导致,这个最好先彻底关闭防火墙,排错的基本法就是这样,无论是s还是c,还有ignore errors选项问题也会导致
错误五:
@error: access denied to www fromunknown (192.168.1.123)
rsync : connection unexpectedly closed(0 bytes received so far) [receiver]
rsync error: error in rsync protocoldata stream (code 12) at io.c(359)
说明:此问题很明显,是配置选项hostallow的问题,初学者喜欢一个允许段做成一个配置,然后模块又是同一个,致使导致
错误六:
rsync error: received sigint,sigterm, or sighup (code 20) at rsync .c(244) [generator=2.6.9]
rsync error: received sigusr1 (code19) at main.c(1182) [receiver=2.6.9]
说明:导致此问题多半是服务端服务没有被正常启动,到服务器上去查查服务是否有启动,然后查看下 /var/run/rsync .pid 文件是否存在,最干脆的方法是杀死已经启动了服务,然后再次启动服务或者让脚本加入系统启动服务级别然后 shutdown -r now服务器
错误七:
rsync : read error: connection resetby peer (104)
rsync error: error in rsync protocoldata stream (code 12) at io.c(604) [sender=2.6.9]
说明:原数据目录里没有数据存在
|
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持。
原文链接:https://www.cnblogs.com/caicairui/archive/2018/02/26/8472888.html
本文由主机测评网发布,不代表主机测评网立场,转载联系作者并注明出处:https://zhuji.jb51.net/linux/5287.html