fully reindexing a file every time the datestamp changes

[mjones414]

I've a few different automated pulls of data into directories of files I want splunk to index. These files get completely overwritten every night at least, but sometimes more often than that depending on different operational conditions out of my control. I need splunk to reindex these files every time the datestamp changes and that doesn't appear to be working. current props configurations:

[source:: /data/ridiculi/all_group/ridiculi.*]
CHECK_METHOD = modtime

[ridiculi:group]
DATETIME_CONFIG = CURRENT
FIELD_DELIMITER = ":"
FIELD_NAMES = gid,status,gidnumber,members
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Operating System
description = Ridiculi
disabled = false
pulldown_type = true

inputs.conf:

[monitor:///data/ridiculi/all_group/ridiculi.wanker]
disabled = false
sourcetype = ridiculi:group
index=ridiculi

[pdaigle_splunk · Accepted solution]

This is a log file rotation setup where you need to use the crcSalt bit configuration and/or the initCrcLength attribute:

https://docs.splunk.com/Documentation/Splunk/7.2.5/Data/Howlogfilerotationishandled

The winning combination was CHECK_METHOD combined with setting crcSalt to something like REINDEX_ALWAYS. Because the file has similar or almost the same data, more than likely the CRC Checksum value and size is the same and Splunk will skip the log file even if the time and date of the file has changed.

[mjones414]

Thank you for posting this Paul!

[pdaigle_splunk]

You're welcome! Just sharing in case others in the community run across the same issue. 🙂

[mjones414]

Just a bump on this to see if there were any more ideas? About to open a case with Splunk support -- as it seems whats here should be sufficient.

[adonio]

the answer is between the lines ... where is your props.conf that has the:
[source:: /data/ridiculi/all_group/ridiculi.*]
CHECK_METHOD = modtime
it supposed to be on the instance that collects the data

[mjones414]

In my situation that is true. the /data directory is monitored by the splunk search head and the props.conf is also on the splunk search head. There is no forwarder involved in this particular data input.

[mjones414]

Just a bump on this to see if there were any more ideas? About to open a case with Splunk support -- as it seems whats here should be sufficient.

[mjones414]

Permissions on all files are 644. permissions on directory are 2644. Filesystem is NFSv3.

[harsmarvania57]

Are you running Universal Forwarder to read /data/ridiculi/all_group/ridiculi.wanker file ? If yes then below props.conf will not work on Universal Forwarder.

[ridiculi:group]
DATETIME_CONFIG = CURRENT
FIELD_DELIMITER = ":"
FIELD_NAMES = gid,status,gidnumber,members
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Operating System
description = Ridiculi
disabled = false
pulldown_type = true

You need to configure above props.conf configuration on first Splunk Enterprise instance from Universal Forwarder because parsing happens on full splunk instance not on UF.

[mjones414]

I'm actually running this on the splunk distributed search head in an app context. The props.conf should be ina distribution bundle that goes to the indexers. The splunk distributed search head does output to an output queue of indexers. Is there something else missing from that config that I need?

[harsmarvania57]

On which instance you are monitoring /data/ridiculi/all_group/ridiculi.wanker logfile ? Search Head or Universal Forwarder ?

[adonio]

where is your props.conf? iirc the top portion has to be on the forwarder:

[source:: /data/ridiculi/all_group/ridiculi.*]
CHECK_METHOD = modtime

the rest will be on the indexer

[mjones414]

Its in etc/apps/ridiculi/local/props.conf and inputs.conf respectively on the distributed search head. the /data path is an autofs mount point that the splunk search head can read (other files are being indexed over /data both from this search head and from indexers as required.)

[somesoni2]

Could you also post first few lines from the file?
Also, The props.conf with [source:..., did you place it in the forwarder (same host as where your inputs.conf lives)?

[mjones414]

sure.

I'm actually running this on the splunk distributed search head in an app context. The props.conf should be ina distribution bundle that goes to the indexers. The splunk distributed search head does output to an output queue of indexers. Is there something else missing from that config that I need?

First few lines from a file:
Admins:NISG:123123:jim,bob,joe
Users:NISG:456456:alpha,whiskey,tango