注册 登录  
 加关注
查看详情
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

曾国藩的博客

 
 
 

日志

 
 

MySQL多主复制-MySQL Galera安装部署  

2014-06-04 15:31:29|  分类: SQL/Oracle/Mysql |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
http://www.zrwm.com/?cat=130

MySQL Galera介绍

  • 特性简介

    MySQL/Galera是MySQL/InnoDB的多主集群,有以下特性:
    - 同步复制
    - Active-active的多主拓扑结构
    - 集群任意节点可以读和写
    - 自动身份控制,失败节点自动脱离集群
    - 自动节点接入
    - 真正的基于”行”级别和ID检查的并行复制
    - 无单点故障,易扩展

    galera-cluster

MySQL Galera安装

  • 安装前准备
    • 机器准备
      G221 : 192.168.1.221 (Centos 6.4)
      G222 : 192.168.1.222 (Centos 6.4)
      G223 : 192.168.1.223 (Centos 6.4)
    • 安装依赖
      - 确认安装有gcc和gcc-c++的版本为至少4.4
      # yum install gcc gcc-c++

      - 确认安装有boost-devel的版本至少为1.4.1

      # yum install boost-devel

      - 安装scons check-devel openssl-devel

      # yum install scons check-devel openssl-devel
  • MySQL Galera安装
    • 安装含wsrep Patch的MySQL 5.5.29
      # yum install libaio
      # wget https://launchpad.net/codership-mysql/5.5/5.5.29-23.7.3/+download/mysql-5.5.29_wsrep_23.7.3-linux-x86_64.tar.gz
      # tar zxvf mysql-5.5.29_wsrep_23.7.3-linux-x86_64.tar.gz 
      # # mv mysql-5.5.29_wsrep_23.7.3-linux-x86_64 /usr/local/mysql
      # cd /usr/local/mysql/
      # groupadd mysql
      # useradd -r -g mysql mysql
      # chown -R mysql:mysql .
      # ./scripts/mysql_install_db --no-defaults --datadir=/usr/local/mysql/data --user=mysql
      # chown -R root .
      # chown -R mysql data
    • 安装Galera复制插件
      # wget https://launchpad.net/galera/2.x/23.2.4/+download/galera-23.2.4-src.tar.gz
      # tar zxvf galera-23.2.4-src.tar.gz
      # cd galera-23.2.4-src
      # scons
      # cp garb/garbd /usr/local/mysql/bin/
      # cp libgalera_smm.so /usr/local/mysql/lib/plugin/
  • MySQL Galera配置
    • MySQL Galera配置例子:
      # cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysql
      # mkdir -p /var/lib/mysql
      # chown mysql:mysql /var/lib/mysql
      # vi /etc/my.cnf
      # cat /etc/my.cnf
      [client]
      port = 3306
      socket = /var/lib/mysql/mysql.sock
      
      [mysqld_safe]
      log-error = /var/lib/mysql/mysql.log
      pid-file = /var/lib/mysql/mysql.pid
      
      [mysqld]
      wsrep_node_name = node1
      wsrep_provider = /usr/local/mysql/lib/plugin/libgalera_smm.so
      #wsrep_provider_options ='gcache.size=1G;socket.ssl_key=my_key;socket.ssl_cert=my_cert'
      #wsrep_slave_threads=16
      wsrep_sst_method = rsync
      #wsrep_sst_auth=root:
      
      port = 3306
      socket = /var/lib/mysql/mysql.sock
      user = mysql
      basedir = /usr/local/mysql
      datadir = /usr/local/mysql/data
      
      default_storage_engine=InnoDB
      #innodb_buffer_pool_size=1G
      #innodb_log_file_size=256M
      innodb_autoinc_lock_mode=2
      innodb_locks_unsafe_for_binlog=1
      innodb_flush_log_at_trx_commit=0
      innodb_doublewrite=0
      innodb_file_per_table=1
      
      binlog_format=ROW
      log-bin=mysql-bin
      server-id=101
      relay-log=mysql-relay-bin
      #read_only=1
      log-slave-updates=1

      注: 可参考MySQL wsrep参数

  • MySQL Galera启动与关闭
    • 初次启动:
      [root@G221 ~]# /usr/local/mysql/bin/mysqld_safe --wsrep_cluster_address=gcomm:// >/dev/null &

      [root@G221 ~]# service mysql start --wsrep_cluster_address=gcomm://

      查看mysqld监听的端口:

      [root@G221 ~]# netstat -plantu | grep mysqld
      tcp        0      0 0.0.0.0:4567       0.0.0.0:*          LISTEN      3656/mysqld         
      tcp        0      0 0.0.0.0:3306       0.0.0.0:*          LISTEN      3656/mysqld

      注:
      1)”gcomm://”是特殊的地址,仅仅是galera cluster初始化启动时候使用,再次启动的时候需要使用具体的IP地址.
      2) 端口4567是wsrep使用的默认端口.该端口的防火墙设置规则应该和3306的一样.

    • 关闭:
      [root@G221 ~]# /usr/local/mysql/bin/mysqladmin -uroot -p shutdown

MySQL Galera新节点

  • 添加新节点
    add-node
    • 节点接入
      添加新节点的时候,新接入的节点叫Joiner,给joiner提供复制的节点叫Donor.新的节点接入需要:
      1) 安装带wsrep patch的MySQL版本
      2) 安装Galera复制插件
      3) 配置好新节点的MySQL(参考Donnor的my.cnf)
      4) 配置或启动的gcomm://的地址是需要使用donnor的IP.

      接入节点G222:

      [root@G222 data]# /usr/local/mysql/bin/mysqld_safe --wsrep_cluster_address="gcomm://192.168.1.221:4567,192.168.1.223:4567"    >/dev/null &

      接入节点G223:

      [root@G223 data]# service mysql start --wsrep_cluster_address="gcomm://192.168.1.221:4567,192.168.1.222:4567"
    • 修改节点的wsrep_cluster_address修改wsrep_cluster_address有两种方式:1)使用新的wsrep_cluster_address重启节点:
      [root@G221 data]# service mysql restart --wsrep_cluster_address="gcomm://192.168.1.222:4567,192.168.1.223:4567"
      Shutting down MySQL.... SUCCESS! 
      Starting MySQL....... SUCCESS!

      2)直接修改MySQL全局变量

      mysql> SHOW VARIABLES LIKE 'wsrep_cluster_address';
      +-----------------------+----------------------------+
      | Variable_name         | Value                      |
      +-----------------------+----------------------------+
      | wsrep_cluster_address | gcomm://192.168.1.222:4567 |
      +-----------------------+----------------------------+
      1 row in set (0.00 sec)
      
      mysql> set global wsrep_cluster_address="gcomm://192.168.1.222:4567,192.168.1.223:4567";
      Query OK, 0 rows affected (2.20 sec)
      
      mysql> SHOW VARIABLES LIKE 'wsrep_cluster_address';
      +-----------------------+-------------------------------------------------------+
      | Variable_name         | Value                                                 |
      +-----------------------+-------------------------------------------------------+
      | wsrep_cluster_address | gcomm://192.168.1.222:4567,192.168.1.223:4567 |
      +-----------------------+-------------------------------------------------------+
      1 row in set (0.00 sec)

MySQL Galera监控

  • 查看相关变量

    查看MySQL版本:

    mysql> SHOW GLOBAL VARIABLES LIKE 'version';
    +---------------+------------+
    | Variable_name | Value      |
    +---------------+------------+
    | version       | 5.5.29-log |
    +---------------+------------+
    1 row in set (0.00 sec)

    查看wsrep版本:

    mysql> SHOW GLOBAL STATUS LIKE 'wsrep_provider_version';
    +------------------------+------------+
    | Variable_name          | Value      |
    +------------------------+------------+
    | wsrep_provider_version | 2.4(rXXXX) |
    +------------------------+------------+
    1 row in set (0.00 sec)

    查看wsrep有关的所有变量:

    mysql> SHOW VARIABLES LIKE 'wsrep%' \G
    *************************** 1. row ***************************
    Variable_name: wsrep_OSU_method
            Value: TOI
    *************************** 2. row ***************************
    Variable_name: wsrep_auto_increment_control
            Value: ON
    *************************** 3. row ***************************
    Variable_name: wsrep_causal_reads
            Value: OFF
    *************************** 4. row ***************************
    Variable_name: wsrep_certify_nonPK
            Value: ON
    *************************** 5. row ***************************
    Variable_name: wsrep_cluster_address
            Value: gcomm://192.168.1.222:4567,192.168.1.223:4567
    *************************** 6. row ***************************
    Variable_name: wsrep_cluster_name
            Value: my_wsrep_cluster
    *************************** 7. row ***************************
    Variable_name: wsrep_convert_LOCK_to_trx
            Value: OFF
    *************************** 8. row ***************************
    Variable_name: wsrep_data_home_dir
            Value: /usr/local/mysql/data/
    *************************** 9. row ***************************
    Variable_name: wsrep_dbug_option
            Value: 
    *************************** 10. row ***************************
    Variable_name: wsrep_debug
            Value: OFF
    *************************** 11. row ***************************
    Variable_name: wsrep_drupal_282555_workaround
            Value: OFF
    *************************** 12. row ***************************
    Variable_name: wsrep_forced_binlog_format
            Value: NONE
    *************************** 13. row ***************************
    Variable_name: wsrep_log_conflicts
            Value: OFF
    *************************** 14. row ***************************
    Variable_name: wsrep_max_ws_rows
            Value: 131072
    *************************** 15. row ***************************
    Variable_name: wsrep_max_ws_size
            Value: 1073741824
    *************************** 16. row ***************************
    Variable_name: wsrep_mysql_replication_bundle
            Value: 0
    *************************** 17. row ***************************
    Variable_name: wsrep_node_address
            Value: 
    *************************** 18. row ***************************
    Variable_name: wsrep_node_incoming_address
            Value: AUTO
    *************************** 19. row ***************************
    Variable_name: wsrep_node_name
            Value: node1
    *************************** 20. row ***************************
    Variable_name: wsrep_notify_cmd
            Value: 
    *************************** 21. row ***************************
    Variable_name: wsrep_on
            Value: ON
    *************************** 22. row ***************************
    Variable_name: wsrep_provider
            Value: /usr/local/mysql/lib/plugin/libgalera_smm.so
    *************************** 23. row ***************************
    Variable_name: wsrep_provider_options
            Value: base_host = 192.168.1.221; base_port = 4567; cert.log_conflicts = no; evs.causal_keepalive_period = PT1S; evs.debug_log_mask = 0x1; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.info_log_mask = 0; evs.install_timeout = PT15S; evs.join_retrans_period = PT1S; evs.keepalive_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.use_aggregate = true; evs.user_send_window = 2; evs.version = 0; evs.view_forget_timeout = PT5M; gcache.dir = /usr/local/mysql/data/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /usr/local/mysql/data//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = NO; gmcast.listen_addr = tcp://0.0.0.0:4567; gmcast.mcast_addr = ; gmcast.mcast_ttl = 1; gmcast.peer_timeout = PT3S; gmcast.time_wait = PT5S; gmcast.version = 0; ist.recv_addr = 192.168.1.221; pc.checksum = true; pc.ignore_quorum = false; pc.ignore_sb = false; pc.linger = PT20S; pc.npvo = false; pc.version = 0; pc.weight = 1; protonet.backend = asio; protonet.version = 0; replicator.causal_read_timeout = PT30S; replicator.commit_order = 3
    *************************** 24. row ***************************
    Variable_name: wsrep_recover
            Value: OFF
    *************************** 25. row ***************************
    Variable_name: wsrep_replicate_myisam
            Value: OFF
    *************************** 26. row ***************************
    Variable_name: wsrep_retry_autocommit
            Value: 1
    *************************** 27. row ***************************
    Variable_name: wsrep_slave_threads
            Value: 2
    *************************** 28. row ***************************
    Variable_name: wsrep_sst_auth
            Value: 
    *************************** 29. row ***************************
    Variable_name: wsrep_sst_donor
            Value: 
    *************************** 30. row ***************************
    Variable_name: wsrep_sst_donor_rejects_queries
            Value: OFF
    *************************** 31. row ***************************
    Variable_name: wsrep_sst_method
            Value: rsync
    *************************** 32. row ***************************
    Variable_name: wsrep_sst_receive_address
            Value: AUTO
    *************************** 33. row ***************************
    Variable_name: wsrep_start_position
            Value: 80cdd13d-8cf2-11e2-0800-e0817023b754:0
    33 rows in set (0.00 sec)
  • 状态监控

    查看Galera集群状态:

    mysql> show status like 'wsrep%';
    +----------------------------+----------------------------------------------------------+
    | Variable_name              | Value                                                    |
    +----------------------------+----------------------------------------------------------+
    | wsrep_local_state_uuid     | 80cdd13d-8cf2-11e2-0800-e0817023b754                     |
    | wsrep_protocol_version     | 4                                                        |
    | wsrep_last_committed       | 3                                                        |
    | wsrep_replicated           | 3                                                        |
    | wsrep_replicated_bytes     | 522                                                      |
    | wsrep_received             | 6                                                        |
    | wsrep_received_bytes       | 1134                                                     |
    | wsrep_local_commits        | 1                                                        |
    | wsrep_local_cert_failures  | 0                                                        |
    | wsrep_local_bf_aborts      | 0                                                        |
    | wsrep_local_replays        | 0                                                        |
    | wsrep_local_send_queue     | 0                                                        |
    | wsrep_local_send_queue_avg | 0.000000                                                 |
    | wsrep_local_recv_queue     | 0                                                        |
    | wsrep_local_recv_queue_avg | 0.000000                                                 |
    | wsrep_flow_control_paused  | 0.000000                                                 |
    | wsrep_flow_control_sent    | 0                                                        |
    | wsrep_flow_control_recv    | 0                                                        |
    | wsrep_cert_deps_distance   | 1.000000                                                 |
    | wsrep_apply_oooe           | 0.000000                                                 |
    | wsrep_apply_oool           | 0.000000                                                 |
    | wsrep_apply_window         | 1.000000                                                 |
    | wsrep_commit_oooe          | 0.000000                                                 |
    | wsrep_commit_oool          | 0.000000                                                 |
    | wsrep_commit_window        | 1.000000                                                 |
    | wsrep_local_state          | 4                                                        |
    | wsrep_local_state_comment  | Synced                                                   |
    | wsrep_cert_index_size      | 5                                                        |
    | wsrep_causal_reads         | 0                                                        |
    | wsrep_incoming_addresses   | 192.168.1.221:3306,192.168.1.222:3306,192.168.1.223:3306 |
    | wsrep_cluster_conf_id      | 13                                                       |
    | wsrep_cluster_size         | 3                                                        |
    | wsrep_cluster_state_uuid   | 80cdd13d-8cf2-11e2-0800-e0817023b754                     |
    | wsrep_cluster_status       | Primary                                                  |
    | wsrep_connected            | ON                                                       |
    | wsrep_local_index          | 0                                                        |
    | wsrep_provider_name        | Galera                                                   |
    | wsrep_provider_vendor      | Codership Oy <info@codership.com>                        |
    | wsrep_provider_version     | 2.4(rXXXX)                                               |
    | wsrep_ready                | ON                                                       |
    +----------------------------+----------------------------------------------------------+
    40 rows in set (0.00 sec)
  • 监控状态说明
    • 集群完整性检查:
      • wsrep_cluster_state_uuid:在集群所有节点的值应该是相同的,有不同值的节点,说明其没有连接入集群.
      • wsrep_cluster_conf_id:正常情况下所有节点上该值是一样的.如果值不同,说明该节点被临时”分区”了.当节点之间网络连接恢复的时候应该会恢复一样的值.
      • wsrep_cluster_size:如果这个值跟预期的节点数一致,则所有的集群节点已经连接.
      • wsrep_cluster_status:集群组成的状态.如果不为”Primary”,说明出现”分区”或是”split-brain”状况.
    • 节点状态检查:
      • wsrep_ready: 该值为ON,则说明可以接受SQL负载.如果为Off,则需要检查wsrep_connected.
      • wsrep_connected: 如果该值为Off,且wsrep_ready的值也为Off,则说明该节点没有连接到集群.(可能是wsrep_cluster_address或wsrep_cluster_name等配置错造成的.具体错误需要查看错误日志)
      • wsrep_local_state_comment:如果wsrep_connected为On,但wsrep_ready为OFF,则可以从该项查看原因.
    • 复制健康检查:
      • wsrep_flow_control_paused:表示复制停止了多长时间.即表明集群因为Slave延迟而慢的程度.值为0~1,越靠近0越好,值为1表示复制完全停止.可优化wsrep_slave_threads的值来改善.
      • wsrep_cert_deps_distance:有多少事务可以并行应用处理.wsrep_slave_threads设置的值不应该高出该值太多.
      • wsrep_flow_control_sent:表示该节点已经停止复制了多少次.
      • wsrep_local_recv_queue_avg:表示slave事务队列的平均长度.slave瓶颈的预兆.

      最慢的节点的wsrep_flow_control_sent和wsrep_local_recv_queue_avg这两个值最高.这两个值较低的话,相对更好.

    • 检测慢网络问题:
      • wsrep_local_send_queue_avg:网络瓶颈的预兆.如果这个值比较高的话,可能存在网络瓶
    • 冲突或死锁的数目:
      • wsrep_last_committed:最后提交的事务数目
      • wsrep_local_cert_failures和wsrep_local_bf_aborts:回滚,检测到的冲突数目

MySQL Galera其它特性

  • Galera arbitrator

    garbd可以做为一个无数据的Galera节点运行,该节点可以用一种优化的方式帮助检测和处理网络分裂(network splits).
    一个好的garbd节点可以防止”split-brain”状况发生.

  • SSL支持

    Galera支持SSL通讯,这对增加数据安全性很重要(特别是云部署).可以通过参数socket.ssl_cert和socket.ssl_key来定义.

  • SST方式选择

    SST允许新接入的节点使用定制的方法来获取最初的数据.SST方式有mysqldump(默认,比较慢),rsync,和xtrabackup(很快).
    可通过wsrep_sst_method来定义.Production环境中建议使用rsync或是xtrabackup方式.

MySQL Galera负载均衡

  评论这张
 
阅读(1223)| 评论(0)
推荐 转载

历史上的今天

在LOFTER的更多文章

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2018