All Apps and Add-ons
Highlighted

Why is the Splunk Add-on for Bro IDS on a Splunk 6.4.0 indexer not automatically extracting fields ?

Engager

Hi,

I have the a Linux box running Bro 2.4 and the Splunk Universal forwarder (6.4.0) configured to monitor my bro logs and forward to an indexer running Splunk 6.4.0 with the Splunk Add-on for Bro IDS installed.
Splunk is setting the sourcetype correctly (brodhcp, brofiles ect..), however, the automatic field extraction is not working.

Is there anything I am missing?

Highlighted

Re: Why is the Splunk Add-on for Bro IDS on a Splunk 6.4.0 indexer not automatically extracting fields ?

Explorer

I'm running into the same issue with EPEL bro-2.4.1-3.el7.x86_64 on CentOS 7 and Splunk 6.5.1. Sourcetype is set correctly, but the dynamic field extraction process isn't working:

props.conf:

[bro]
SHOULD_LINEMERGE = false
TRUNCATE = 0
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %s.%6N
TRANSFORMS-BroAutoType = BroAutoType, TrashComments
INDEXED_EXTRACTIONS = TSV
FIELD_HEADER_REGEX = ^#fields\t(.*)
FIELD_DELIMITER = \t
FIELD_QUOTE = \t

Sample conn.log header:

#separator \x09
#set_separator  ,
#empty_field    (empty)
#unset_field    -
#path   conn
#open   2017-02-07-16-24-39
#fields ts  uid id.orig_h   id.orig_p   id.resp_h   id.resp_p   proto   service duration    orig_bytes  resp_bytes  conn_state  local_orig  local_resp  missed_bytes    history orig_pkts   orig_ip_bytes   resp_pkts   resp_ip_bytes   tunnel_parents
#types  time    string  addr    port    addr    port    enum    string  interval    count   count   string  bool    bool    count   string  count   count   count   count   set[string]
0 Karma
Highlighted

Re: Why is the Splunk Add-on for Bro IDS on a Splunk 6.4.0 indexer not automatically extracting fields ?

Explorer

Also confirmed the same issue with Bro 2.5 installed using the CentOS 7 RPM from: https://www.bro.org/download/packages.html

0 Karma
Highlighted

Re: Why is the Splunk Add-on for Bro IDS on a Splunk 6.4.0 indexer not automatically extracting fields ?

Explorer

Resolved by including the props.conf and transforms.conf file on the forwarder along with inputs.conf, which are required to perform INDEXED_EXTRACTIONS
per the document Extract fields from files with structured data.