We have a situation on our search head cluster nodes and one of the peer node KVstore is filling up, so not sure we can clean manually or is there any process to backup and clean? What are the limitations if we clean all the data?
You need to identify which KVStore collection is taking lot of space and decide if that is required or not . Don't just remove all KVStore collections in your SHC environment unless you understand the impact.
I would start with Checking the KVStore collection size in DMC and find out which collections are taking more size and go from there.
If you need to remove one collection individually in SHC, you can use | outputlookup
to remove it.
If you decide to backup your kvstore and restore it later you can refer below documentation .