Getting Data In

growing file

splunk_gs
Explorer

I have a growing file on a remote location. Can i use the forwarder to monitor this file without having to reupload it everytime it grows? i know initially it will upload the main file but overtime when it grows i just want it to add the changes not the entire file. How do i go about setting this up?

Thanks

Tags (2)
0 Karma
1 Solution

sdaniels
Splunk Employee
Splunk Employee

Yes, you can install a forwarder on that remote server and set up file monitoring. Splunk will handle log rotation without re-indexing data it has already taken in and remember where it was for a given file so that it only takes in the new data. An exception would be archive files like .tar and .gz. Check the docs.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Monitorfilesanddirectories

View solution in original post

0 Karma

sdaniels
Splunk Employee
Splunk Employee

Yes, you can install a forwarder on that remote server and set up file monitoring. Splunk will handle log rotation without re-indexing data it has already taken in and remember where it was for a given file so that it only takes in the new data. An exception would be archive files like .tar and .gz. Check the docs.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Monitorfilesanddirectories

View solution in original post

0 Karma

sdaniels
Splunk Employee
Splunk Employee

You'll want to set a sourcetype and index. There are other settings that you can look at with details in the inputs.conf spec. Splunk will remember where it is in a file to continuously monitor it and not duplicate data.

http://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf

0 Karma

splunk_gs
Explorer

are there any specific options i would need to select to do this? like tail forwarding?

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.