Getting Data In

Use RegEx to set sourcetype on UF where events are in a JSON format

Path Finder

Scenario:

  1. two different source types being sent to UF (snort and firewall) from the same IP/source.
  2. format of data is JSON (see below example)

    {
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }{
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }

I've tried using regexes like below to separate them.

props.conf

[udp:514]
TRANSFORMS-set_sourcetype = snort,firewall

transforms.conf

[snort]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

[firewall]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

Looking for the best method to assign the sourcetypes. I do not have control over the data structure or how it is sent to the UF.

0 Karma

SplunkTrust
SplunkTrust

A UF doesn't parse the data, so this functionality must be done on the indexer, or an HF. Your title for the question specifically states doing it on a UF, so the answer is, you can't. But you can do it on the indexers or an HF.

0 Karma

SplunkTrust
SplunkTrust

Why do you want to separate sourcetypes?
JSON is extracted fields appropriately .

If you search , you can use type=snort or type=firewall .

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!