Getting Data In

Use RegEx to set sourcetype on UF where events are in a JSON format

Path Finder

Scenario:

  1. two different source types being sent to UF (snort and firewall) from the same IP/source.
  2. format of data is JSON (see below example)

    {
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }{
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }

I've tried using regexes like below to separate them.

props.conf

[udp:514]
TRANSFORMS-set_sourcetype = snort,firewall

transforms.conf

[snort]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

[firewall]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

Looking for the best method to assign the sourcetypes. I do not have control over the data structure or how it is sent to the UF.

0 Karma

SplunkTrust
SplunkTrust

A UF doesn't parse the data, so this functionality must be done on the indexer, or an HF. Your title for the question specifically states doing it on a UF, so the answer is, you can't. But you can do it on the indexers or an HF.

0 Karma

SplunkTrust
SplunkTrust

Why do you want to separate sourcetypes?
JSON is extracted fields appropriately .

If you search , you can use type=snort or type=firewall .

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes and swag!