Splunk Search

manually copying log files... overwrite or add new?

glennh
Engager

Hi folks,

I'm trialling Splunk and while I'm waiting for my support folks to install the Splunk Forwarder on my Tomcat servers I have installed a forwarder on my dev pc monitoring a directory and manually copied my tomcat logs to my local monitored directory.

When I take a new copy of the tomcat logs should I just overwrite the current logs? Will this cause the original logs to be reindexed and sho duplicate events in Splunk?

thanks

Glenn

Tags (3)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

If you are writing data to the same index, you are essentially duplicating data. Splunk guards against this by looking at the first 256 bytes of data. If the crcSalt we come up with matches one we've previously calculated, we'll skip indexing that file. If you want Splunk to reindex files from the forwarder and your just in a trial at the moment, you may want to think about cleaning out the indexed data with the the command 'splunk clean eventdata' from $SPLUNK_HOME/bin/.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

If you are writing data to the same index, you are essentially duplicating data. Splunk guards against this by looking at the first 256 bytes of data. If the crcSalt we come up with matches one we've previously calculated, we'll skip indexing that file. If you want Splunk to reindex files from the forwarder and your just in a trial at the moment, you may want to think about cleaning out the indexed data with the the command 'splunk clean eventdata' from $SPLUNK_HOME/bin/.

Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...