Master-replica: automated failover

Step 1: Configuring a failover mode

First, set the replication.failover option to election:

replication:
  failover: election

Step 2: Defining a replica set topology

Define a replica set topology inside the groups section. The iproto.listen option specifies an address used to listen for incoming requests and allows replicas to communicate with each other.

groups:
  group001:
    replicasets:
      replicaset001:
        instances:
          instance001:
            iproto:
              listen:
              - uri: '127.0.0.1:3301'
          instance002:
            iproto:
              listen:
              - uri: '127.0.0.1:3302'
          instance003:
            iproto:
              listen:
              - uri: '127.0.0.1:3303'

Step 3: Creating a user for replication

In the credentials section, create the replicator user with the replication role:

credentials:
  users:
    replicator:
      password: 'topsecret'
      roles: [replication]

Step 4: Specifying advertise URIs

Set iproto.advertise.peer to advertise the current instance to other replica set members:

iproto:
  advertise:
    peer:
      login: replicator

Resulting configuration

The resulting replica set configuration should look as follows:

credentials:
  users:
    replicator:
      password: 'topsecret'
      roles: [replication]

iproto:
  advertise:
    peer:
      login: replicator

replication:
  failover: election

groups:
  group001:
    replicasets:
      replicaset001:
        instances:
          instance001:
            iproto:
              listen:
              - uri: '127.0.0.1:3301'
          instance002:
            iproto:
              listen:
              - uri: '127.0.0.1:3302'
          instance003:
            iproto:
              listen:
              - uri: '127.0.0.1:3303'

Starting instances

1. After configuring a replica set, execute the tt start command from the tt environment directory:

   $ tt start auto_leader
      • Starting an instance [auto_leader:instance001]...
      • Starting an instance [auto_leader:instance002]...
      • Starting an instance [auto_leader:instance003]...
   

2. Check that instances are in the RUNNING status using the tt status command:

   $ tt status auto_leader
   INSTANCE                 STATUS   PID   MODE  CONFIG  BOX      UPSTREAM
   auto_leader:instance001  RUNNING  9170  RO    ready   running  --
   auto_leader:instance002  RUNNING  9171  RO    ready   running  --
   auto_leader:instance003  RUNNING  9172  RW    ready   running  --
   

Checking a replica set status

1. Connect to instance001 using tt connect:

   $ tt connect auto_leader:instance001
      • Connecting to the instance...
      • Connected to auto_leader:instance001
   

2. Check the instance state in regard to leader election using box.info.election. The output below shows that instance001 is a follower while instance002 is a replica set leader.

   auto_leader:instance001> box.info.election
   ---
   - leader_idle: 0.77491499999815
     leader_name: instance002
     state: follower
     vote: 0
     term: 2
     leader: 1
   ...
   

3. Check that instance001 is in read-only mode using box.info.ro:

   auto_leader:instance001> box.info.ro
   ---
   - true
   ...
   

4. Execute box.info.replication to check a replica set status. Make sure that upstream.status and downstream.status are follow for instance002 and instance003.

   auto_leader:instance001> box.info.replication
   ---
   - 1:
       id: 1
       uuid: 4cfa6e3c-625e-b027-00a7-29b2f2182f23
       lsn: 9
       upstream:
         status: follow
         idle: 0.8257709999998
         peer: replicator@127.0.0.1:3302
         lag: 0.00012326240539551
       name: instance002
       downstream:
         status: follow
         idle: 0.81174199999805
         vclock: {1: 9}
         lag: 0
     2:
       id: 2
       uuid: 9bb111c2-3ff5-36a7-00f4-2b9a573ea660
       lsn: 0
       name: instance001
     3:
       id: 3
       uuid: 9a3a1b9b-8a18-baf6-00b3-a6e5e11fd8b6
       lsn: 0
       upstream:
         status: follow
         idle: 0.83125499999733
         peer: replicator@127.0.0.1:3303
         lag: 0.00010204315185547
       name: instance003
       downstream:
         status: follow
         idle: 0.83213399999659
         vclock: {1: 9}
         lag: 0
   ...
   

   To see the diagrams that illustrate how the upstream and downstream connections look, refer to Monitoring a replica set.

Adding data

To check that replicas (instance001 and instance003) get all updates from the master (instance002), follow the steps below:

1. Connect to instance002 using tt connect:

   $ tt connect auto_leader:instance002
      • Connecting to the instance...
      • Connected to auto_leader:instance002
   

2. Create a space and add data as described in CRUD operation examples.

3. Use the select operation on instance001 and instance003 to make sure data is replicated.

4. Check that the 1 component of box.info.vclock values are the same on all instances:

   • instance001:

     auto_leader:instance001> box.info.vclock
     ---
     - {0: 1, 1: 32}
     ...
     

   • instance002:

     auto_leader:instance002> box.info.vclock
     ---
     - {0: 1, 1: 32}
     ...
     

   • instance003:

     auto_leader:instance003> box.info.vclock
     ---
     - {0: 1, 1: 32}
     ...
     

Примечание

Note that a vclock value might include the 0 component that is related to local space operations and might differ for different instances in a replica set.