Getting Data In

Why are my headers are getting indexed as events every 1 hour?

k_harini
Communicator

Hi,

I'm facing a strange issue. Header rows are getting extracted as events every 1 hour. I have files flowing into monitoring path with scheduled shell script (every 15 mins).
I have done these changes in prop.conf file

CHECK_FOR_HEADER = TRUE
HEADER_FIELD_LINE_NUMBER = 1
FIELD_NAMES = "Consumer ID","Delivery code","Recipient Status","Event date"
PREAMBLE_REGEX = ^Consumer.*

Around 17 files are there with same headers. only from 1 file this issue is happening
despite all these headers are getting indexed as events.. Please help to resolve this issue.

0 Karma

krish3
Contributor

Is the shell scripts rotating the files or appending or rewriting?

0 Karma

k_harini
Communicator

Its adding the new files . I'm flushing the data with different script. but presently its not adding any files. even then i get header rows every 1 hour

0 Karma

krish3
Contributor

I believe it is being added by scripts every one hour to a new file which is picked up for forwarder please let me know more details on script schedule & log rotation. Thanks,

0 Karma

k_harini
Communicator

I removed that file and checked. Headers are still getting indexed. please help

0 Karma

scannon4
SplunkTrust
SplunkTrust

Did you ever get this fixed? I am having a similar issue.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

There should something different about that 1 file which is failing. Check for additional spaces, line break and/or update the PREAMBLE_REGEX to handle any additional spaces.

0 Karma

k_harini
Communicator

There is nothing different, no spaces. Its all same as in other files. Its the first file..

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

If there is nothing different, then rename the file to something else, --- something later lexicographically -- and then see if it happens to the NEXT first file.

If that fixes the problem, something was bugged about the way splunk was handling that particular named file. (some pointer or sticky note it was using to remember something)

On the other hand, more likely, the renamed file will still be bugged. In that case, edit the file to remove the header, and copy another header record from a file that worked. It was probably an invisible/non-displayable character of some sort. You can do a hex dump to see what it was.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...