Reindex and create new event for file when modtime changes, regardless of content

whar_garbl — Tue, 04 Mar 2025 19:23:46 GMT

I have a file I'm monitoring that changes several times a day. It is likely that sometimes the file contents will be the same as a previous iteration, but not guaranteed (the file name name does not change). The file is in text format and is a few dozen lines long.

I want to process the file every time the modtime changes, even if the content is 100% the same, and I want to create a single event with the contents each time.

props.conf:

[my_sourcetype]

DATETIME_CONFIG = current

BREAK_ONLY_AFTER = nevereverbreak

[source::/path/to/file-to-be-read]

CHECK_METHOD = modtime

sourcetype = my_sourcetype

inputs.conf:

[monitor:///path/to/file-to-be-read]

disabled = 0

sourcetype = my_sourcetype

crcSalt = some_random_value_to_try_to_make_it_always_read

If I update file-to-be-read manually by adding new lines to the end, it gets read in immediately and I get an event just like I want. But when the automated process creates the file (with an updated modtime), Splunk seems not to be interested in it. Perms are correct and splunkd.log reflects that the modtime is different and it's re-reading the file... but it doesn't create a new event.

I'm sure I'm missing something obvious, but I'd appreciate any advice. Cheers.

Re: Reindex and create new event for file when modtime changes, regardless of content

livehybrid — Tue, 04 Mar 2025 23:03:47 GMT

Hi @whar_garbl

I think what you have done with "CHECK_METHOD" in props.conf should work.

[source::<yoursource>] CHECK_METHOD = modtime

However, you may also need to set the crcSalt in inputs.conf

[monitor://<path>] crcSalt = <SOURCE>

Here are a few other useful links which might also help!

https://community.splunk.com/t5/Getting-Data-In/Ingesting-file-data/td-p/81645

https://community.splunk.com/t5/Knowledge-Management/Modtime-is-newer-than-stored-will-reread-file-with-9-x-x/td-p/677930 << Beware of this possible bug

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

topic Re: Reindex and create new event for file when modtime changes, regardless of content in Getting Data In

Reindex and create new event for file when modtime changes, regardless of content

Re: Reindex and create new event for file when modtime changes, regardless of content