Splunk Search

manually copying log files... overwrite or add new?

glennh
Engager

Hi folks,

I'm trialling Splunk and while I'm waiting for my support folks to install the Splunk Forwarder on my Tomcat servers I have installed a forwarder on my dev pc monitoring a directory and manually copied my tomcat logs to my local monitored directory.

When I take a new copy of the tomcat logs should I just overwrite the current logs? Will this cause the original logs to be reindexed and sho duplicate events in Splunk?

thanks

Glenn

Tags (3)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

If you are writing data to the same index, you are essentially duplicating data. Splunk guards against this by looking at the first 256 bytes of data. If the crcSalt we come up with matches one we've previously calculated, we'll skip indexing that file. If you want Splunk to reindex files from the forwarder and your just in a trial at the moment, you may want to think about cleaning out the indexed data with the the command 'splunk clean eventdata' from $SPLUNK_HOME/bin/.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

If you are writing data to the same index, you are essentially duplicating data. Splunk guards against this by looking at the first 256 bytes of data. If the crcSalt we come up with matches one we've previously calculated, we'll skip indexing that file. If you want Splunk to reindex files from the forwarder and your just in a trial at the moment, you may want to think about cleaning out the indexed data with the the command 'splunk clean eventdata' from $SPLUNK_HOME/bin/.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...