Why are my headers are getting indexed as events every 1 hour?

[k_harini]

Hi,

I'm facing a strange issue. Header rows are getting extracted as events every 1 hour. I have files flowing into monitoring path with scheduled shell script (every 15 mins).
I have done these changes in prop.conf file

CHECK_FOR_HEADER = TRUE
HEADER_FIELD_LINE_NUMBER = 1
FIELD_NAMES = "Consumer ID","Delivery code","Recipient Status","Event date"
PREAMBLE_REGEX = ^Consumer.*

Around 17 files are there with same headers. only from 1 file this issue is happening
despite all these headers are getting indexed as events.. Please help to resolve this issue.

[krish3]

Is the shell scripts rotating the files or appending or rewriting?

[k_harini]

Its adding the new files . I'm flushing the data with different script. but presently its not adding any files. even then i get header rows every 1 hour

[krish3]

I believe it is being added by scripts every one hour to a new file which is picked up for forwarder please let me know more details on script schedule & log rotation. Thanks,

[k_harini]

I removed that file and checked. Headers are still getting indexed. please help

[scannon4]

Did you ever get this fixed? I am having a similar issue.

[somesoni2]

There should something different about that 1 file which is failing. Check for additional spaces, line break and/or update the PREAMBLE_REGEX to handle any additional spaces.

[k_harini]

There is nothing different, no spaces. Its all same as in other files. Its the first file..

[DalJeanis]

If there is nothing different, then rename the file to something else, --- something later lexicographically -- and then see if it happens to the NEXT first file.

If that fixes the problem, something was bugged about the way splunk was handling that particular named file. (some pointer or sticky note it was using to remember something)

On the other hand, more likely, the renamed file will still be bugged. In that case, edit the file to remove the header, and copy another header record from a file that worked. It was probably an invisible/non-displayable character of some sort. You can do a hex dump to see what it was.