149  
redis做从库时报I/O error trying to sync with MASTER: connection lost错误
作者: wyasw 于 2020年03月29日 发布在分类 / linux / 填过的坑 下,并于 2020年03月29日 编辑
redis

redis做从库时报I/O error trying to sync with MASTER: connection lost错误

因单个程序访问redis流量超出980Mb,单个redis不能满足业务需要,需要做redis主从,让单个程序从redis从只读,主的数据为1.7G,做完从后,发现数据没有同步过来,查看从库日志

 

18546:S 27 Dec 11:31:01.738 * Connecting to MASTER 192.170.5.88:6379

18546:S 27 Dec 11:31:01.738 * MASTER <-> SLAVE sync started

18546:S 27 Dec 11:31:01.738 * Non blocking connect for SYNC fired the event.

18546:S 27 Dec 11:31:01.740 * Master replied to PING, replication can continue...

18546:S 27 Dec 11:31:01.742 * Partial resynchronization not possible (no cached master)

18546:S 27 Dec 11:31:01.924 * Full resync from master: 8a8b019dfa1192f81f404c68075d4c5dbc0467fd:473627650

18546:S 27 Dec 11:32:34.235 * MASTER <-> SLAVE sync: receiving 1824088898 bytes from master

18546:S 27 Dec 11:32:47.160 # I/O error trying to sync with MASTER: connection lost

18546:S 27 Dec 11:32:48.154 * Connecting to MASTER 192.170.5.88:6379

18546:S 27 Dec 11:32:48.154 * MASTER <-> SLAVE sync started

18546:S 27 Dec 11:32:48.155 * Non blocking connect for SYNC fired the event.

18546:S 27 Dec 11:32:48.155 * Master replied to PING, replication can continue...

18546:S 27 Dec 11:32:48.167 * Partial resynchronization not possible (no cached master)

18546:S 27 Dec 11:32:48.428 * Full resync from master: 8a8b019dfa1192f81f404c68075d4c5dbc0467fd:614945359

18546:S 27 Dec 11:34:21.233 * MASTER <-> SLAVE sync: receiving 1823925331 bytes from master

18546:S 27 Dec 11:34:27.113 # I/O error trying to sync with MASTER: connection lost

 

 

##错误I/O error trying to sync with MASTER: connection lost

 

再来查看主库日志,发现在redis主库在接到从库要求重新同步数据的时候先生成一个rdb文件,再通过psync来做部分同步,可以看到问题就出在部分同步这一块,由于client-output-buffer-limit值设置太小,导致导数发送失败。失败后,从库继续发起数据同步的请求,但是每次都失败,redis主库反复地生成rdb文件,虽然是有fork一个子进程,但是对redis主库的吞吐量是有一定影响的。

 

解决问题:

在主库操作

127.0.0.1:6379> config get client-output-buffer-limit

1) "client-output-buffer-limit"

2) "normal 0 0 0 slave 268435456 67108864 60 pubsub 33554432 8388608 60"

可以看到,目前的限制是最大256M60s内不超过64M,从我们上面日志信息可以看到psync的数据明显是大于256M的。我们进行如下设置把很限制调大:

config set client-output-buffer-limit 'slave 1073741824 268435456 60'

调完后再观察,发现从库的的复制状态很快就变成了up

我们上面的调整是加大复制输出缓冲区,还有一个办法就是关闭复制输出缓冲区的限制:

config set client-output-buffer-limit 'slave 0 0 0'

 




 推荐知识

 历史版本

修改日期 修改人 备注
2020-03-29 21:31:23[当前版本] wyasw 创建版本

  目录
    文艺知识分享平台 - 4.3.0 - 文艺范儿