All Apps and Add-ons

json field extraction not working

mcbradford
Contributor

I have suricata logs being forwarded from suricata to my syslog-ng server. I am using the ta-suricata and have the props.conf on my indexer and my search head. I cannot get the fields extracted. I wonder if the syslog forwarding is breaking things? If so, how do I fix this?

first few lines of props.conf that should be handling the field extraction:

[suricata]
INDEXED_EXTRACTIONS = NONE
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

sample of event:

Feb  1 13:28:18 TUL1ZBSECSNSR1 eve.json {"timestamp":"2018-02-01T13:28:16.676494-0600","flow_id":1582901243164018,"in_iface":"eth1","event_type":"tls","vlan":8,"src_ip":"10.41.12.176","src_port":49166,"dest_ip":"10.9.64.115","dest_port":443,"proto":"TCP","tls":{"subject":"C=US, O=my company, OU=Enterprise Technology Support, CN=blah.mycompany.net\/ST=GA\/L=Columbus","issuerdn":"C=US, O=mycompany, OU=Enterprise Technology Support, CN=blah.mycompany.net\/ST=GA\/L=Columbus"}}

The only thing that is extracting is where the key value pair has the =, so for example, o=mycompany

I really need the fields like in_iface, event_type to be extracted as well.

0 Karma
1 Solution

mcbradford
Contributor

I figured out a work around:

[suricata]
SEDCMD-strip_prefix = s/^[^{]+//g
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

I added the line:

SEDCMD-strip_prefix = s/^[^{]+//g

And now everything is working perfectly.

View solution in original post

mcbradford
Contributor

I figured out a work around:

[suricata]
SEDCMD-strip_prefix = s/^[^{]+//g
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

I added the line:

SEDCMD-strip_prefix = s/^[^{]+//g

And now everything is working perfectly.

atellez_splunk
Splunk Employee
Splunk Employee

The props do not support syslog extraction, they are designed to read directly from /var/log/suricata/eve.json
For syslog you would need to re-write the props to extract each field.

0 Karma
Get Updates on the Splunk Community!

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...