All Apps and Add-ons

Why am I getting error "Ran out of data while looking for end of header" configuring the Splunk Add-on for Bro IDS?

adamblock2
Path Finder

We are currently running version 2.4 of Bro, and I have been having difficulty properly configuring the Bro add-on.

According to the documentation, versions 2.1 and 2.2 are supported. I am curious if this could be part of our problem.

The following bro logs are currently being written to "/syslog_hot/splunk/bro" on our syslog server.

-rw-------. 1 root root   3621901 Mar  1 12:36 notice.log
-rw-------. 1 root root    140147 Mar  1 12:30 other.log
-rw-------. 1 root root 654588548 Mar  1 12:36 ssl.log

I added the following to inputs.conf file:

[monitor://syslog_hot/splunk/bro]
    index=bro
    sourcetype=bro
    blacklist = \.(gz*|\d+|txt)$

When I added the "sourcetype=bro" statement to the inputs.conf file as per the documentation, I started receiving the following error:

03-01-2016 12:19:07.616 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header
03-01-2016 12:19:07.633 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header
03-01-2016 12:19:42.778 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header
03-01-2016 12:19:42.796 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header
03-01-2016 12:20:03.077 -0500 ERROR TailingProcessor - Ran out of data while looking for end of header

As soon as I removed the "sourcetype=bro" statement, logs started being forwarded to the indexers. However, they appeared with the following sourcetypes:
ssl.log - "ssl-3"
notice.log - "notice-2"
other.log - "syslog"

Assistance with this would be appreciated.

Thank you.

0 Karma

jorritf
Path Finder

Perhaps it has something to do with other (hidden) files that are also present in the directory you monitor? When I look at my own installations there are various Bro related files like .state, .status .rotated. etc that may get scanned for lines starting with "#fields", so INDEXED_EXTRACTIONS = TSV returns proper header names.

You may try to change the monitor stanza to something like [monitor:///syslog_hot/splunk/bro/*.log], or also blacklist any files in that directory that don't start with "#fields"

0 Karma

zabbasi_splunk
Splunk Employee
Splunk Employee

Make sure your logs are written as type.log or something.type.log because the Splunk platform uses the second part of the name to more specifically source type the log. For example, conn.log produces the bro_conn sourcetype.

0 Karma

adamblock2
Path Finder

I included a listing of the file names in my original question (notice.log, ssl.log, other.log).

0 Karma

ehaddad_splunk
Splunk Employee
Splunk Employee

I don't think it has to do with the version of Bro - not yet at least.
I see you need to add an extra '/'. Not sure if this causing the issue but very possible. I would try to change things to:
[monitor:///syslog_hot/splunk/bro]

0 Karma

adamblock2
Path Finder

I added the additional '/', but I continue receiving the " ERROR TailingProcessor - Ran out of data while looking for end of header" messages.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...