I have a file on a server that i want to index. I have an app that watches this file and indexes it.
The file contains an output of server metric data and is essentially in the format of "key = value","key = value" and so on.
This file gets overwritten with new data every 60s.
Recently I noticed that the some metrics were dropping off, i checked the file and it is complete. but when i check the source in Splunk i can clearly see that it has been truncated.
Nothing else on the server has changed. I have checked the logs on the universal forwarder, but I could not see anything that stood out.
How long are your events? If your events are many lines long, or a single line is very long, you may need to increase some limits. Here is a great entry that covers this:
Size limit for an event