All Apps and Add-ons

json field extraction not working

mcbradford
Contributor

I have suricata logs being forwarded from suricata to my syslog-ng server. I am using the ta-suricata and have the props.conf on my indexer and my search head. I cannot get the fields extracted. I wonder if the syslog forwarding is breaking things? If so, how do I fix this?

first few lines of props.conf that should be handling the field extraction:

[suricata]
INDEXED_EXTRACTIONS = NONE
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

sample of event:

Feb  1 13:28:18 TUL1ZBSECSNSR1 eve.json {"timestamp":"2018-02-01T13:28:16.676494-0600","flow_id":1582901243164018,"in_iface":"eth1","event_type":"tls","vlan":8,"src_ip":"10.41.12.176","src_port":49166,"dest_ip":"10.9.64.115","dest_port":443,"proto":"TCP","tls":{"subject":"C=US, O=my company, OU=Enterprise Technology Support, CN=blah.mycompany.net\/ST=GA\/L=Columbus","issuerdn":"C=US, O=mycompany, OU=Enterprise Technology Support, CN=blah.mycompany.net\/ST=GA\/L=Columbus"}}

The only thing that is extracting is where the key value pair has the =, so for example, o=mycompany

I really need the fields like in_iface, event_type to be extracted as well.

0 Karma
1 Solution

mcbradford
Contributor

I figured out a work around:

[suricata]
SEDCMD-strip_prefix = s/^[^{]+//g
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

I added the line:

SEDCMD-strip_prefix = s/^[^{]+//g

And now everything is working perfectly.

View solution in original post

mcbradford
Contributor

I figured out a work around:

[suricata]
SEDCMD-strip_prefix = s/^[^{]+//g
SHOULD_LINEMERGE = false
TIME_PREFIX=timestamp":
BREAK_ONLY_BEFORE = ^{
KV_MODE = json

I added the line:

SEDCMD-strip_prefix = s/^[^{]+//g

And now everything is working perfectly.

atellez_splunk
Splunk Employee
Splunk Employee

The props do not support syslog extraction, they are designed to read directly from /var/log/suricata/eve.json
For syslog you would need to re-write the props to extract each field.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...