elasticsearch,

How to manually change the Elasticsearch's allocation of shards

Jul 13, 2021 · 2 mins read · Post a comment

While updating the mapping on some ES clusters or doing a full reindex which are expensive operations, you may face a data node failure. Due to the node failure, there is a possibility some shards to become unassigned, but the situation will get complicated if you haven’t configured a replica shard. It will throw a RED cluster state, so you need to make sure that you have at least one replica.

Prerequisites

  • Full access of the Elasticsearch Domain Service

Checking the unassigned shards in the ES Cluster

Step 1. To check which shards are unassigned, run:

curl -XGET 'ES_Endpoint/_cat/shards?h=index,shard,prirep,state,unassigned.reason' | grep UNASSIGNED

Step 2. Identify the root cause of the cluster’s unassigned shards.

curl -XGET 'ES_Endpoint/_cluster/allocation/explain'

If you find the following error:

cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster

The meaning of the error is that this is the primary shard and there is no replica shard, so the primary shard cannot be reallocated because of data loss. One solution is to reindex those indexes or return them from backup, but the easiest and safest method is to manually reallocate the shards to one of the active data nodes and increase the number of replicas.

Manually change the allocation of shards

Step 1. Execute the next API call to manually reallocate the needed shard:

curl -XPOST 'ES_Endpoint/_cluster/reroute'
{
  "commands": [
    {
      "move": {
        "index": "index-name", "shard": 0,
        "from_node": "node1", "to_node": "node2"
      }
    }
  ]
}

Step 2. After executing the command the shard should be initialized on the new node. So you will have to change the number of replica to at least one.

curl -XPUT 'ES_Endpoint/<indexname>/_settings'
{
  "index" : {
    "number_of_replicas" : 1
  }
}

Conclusion

This tutorial will help you to reallocate the unassigned shards in your ES Cluster and avoid any data loss. Feel free to leave a comment below and if you find this tutorial useful, follow our official channel on Telegram.

elasticsearch