I use Splunk 4.2 and have 2 node A and B on Cluster and logs has compression in zip files
i don't want duplicate logs from B,
how can I do for this problem? or it is bug of 4.2
I'd need more details on what's involved in your Cluster switch over. Is the log directory moved from being available only on node A, to only being available on node B.
If this is the case, it's not a bug, as after the switch to node B all these files would be new to SplunkForwarder on B. All of the data that keeps track of what logs have been indexed would be stuck over on node A.
Would it be possible to have the log rotated as part of the cluster switch? Then have Splunk only look at the active file? Again, difficult to give specifics without more knowledge of your cluster configuration.
Thank you for your answer,
I keep splunk_db and all logs on san box.
In my test case, I try to use this cluster to forward .log(not compress) files and the result is work well.
Splunk forwarder has been install on each node.
San box will move to available on B after switched.
in the same case, no problem with normal log (file isnot compression)
As long as Splunk starts before the application(s) that generate the logs to be indexed you could add:
followTail = true
to the source definitions in inputs.conf. This would get Splunk to only look at data added after Splunk has started to monitor the logs. The danger is that if the application starts before Splunk has started to monitor the logs you may miss log entries.
that is a good suggestion for me, i will try and update on this post in soon.
but, I have some question about this.
why it only have problem with compression file on cluster?
Thank you again, for this answer.
i think, followTail = [0|1] is a good tip.
in this case. i can try to use ignoreOlderThan param on B for help to read log file by ignore the old files.
but, i still want to know the answer of my main question.
i wish to see SplunkForwarder working completely with compression files same as normal log file.
so, if you have any comments for my main question please let me know.
ignoreOlderThan=2d is a good solution for short term workaround. We can set the parameter on both node and apply on input.conf. Anyway, we are waiting for long term solution from Splunk Support team.