All geek questions in one place

Remote nodetool node stuck during uninstall

I have a small cluster that is almost empty. Typically, nodetool removenode ends an order of 10 seconds. However, currently I have a node removal in a process that takes 10 seconds and does not seem to progress. An additional node deletion request was rejected because deletion is already in progress. How can I fix this problem? For reference, here is the output on nodetool status :

 Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack DL 192.168.12.207 152.14 KB 256 32.2% 683d8351-c625-4d7f-99cc-61f6b73b0c56 rack1 UN 192.168.12.205 215.21 KB 256 37.2% b66d5fff-ef1d-4fbf-a49a-43709df99a0c rack1 UN 192.168.12.208 148.09 KB 256 30.6% 39b54771-59b8-49f7-8db8-9cf4523d6c8d rack1

In addition, cassandra does not work on host 207 (outgoing node), but it works on two other hosts.

EDIT: There seems to be at least one token that is stuck waiting for replication:

 $ nodetool removenode status RemovalStatus: Removing token (-9037887679483580088). Waiting for replication confirmation from [/192.168.12.205].
+6
source share
2 answers

I don't know which version of Cassandra is the one with the problem. But if nodetool removeenode does not work, according to the Apache Cassandra Wiki , you should try the following:

Removenode

Removing a node that does not physically exist is performed in two steps:

  bin/nodetool removenode <UUID> bin/nodetool removenode force

The first command will be blocked forever if the computer connected to this UUID is physically removed (or no longer starts Cassandra). Just hit Ctrl-C after a second or two before running the second command. Obviously, it is better to write off a node first, if possible, or you may lose some of your data.

The "bin / nodetool status" command shows the UUID of your nodes.

Hope this helps.

+7
source

You are probably experiencing a known bug, CASSANDRA-6542

0
source

Source: https://habr.com/ru/post/975562/

More articles:


All Articles