Failure during delete of zk node for process group can cause issues on leader switch

[anvinjain]

In ZookeeperBasedBackendAssociationStore, method deAssociateBackendWithProcessGroup:
delete of process group zk node happens. If delete in ZK goes through but there is a n/w issue which leads to exception being thrown by ZK, in-memory data structures will not get updated. Subsequent de-associate calls for that backend will always throw exception because node does not exist when deleting, so in-memory DS will remain in-consistent.

Possible fix is to wrap the delete and update of in-memory DS in a ZK transaction (along with check of existence of node after delete has happened) and corresponding rollback behaviour if any issue occurs in the transaction. Evaluate this fix though, not set in stone