Getting Data In

Use RegEx to set sourcetype on UF where events are in a JSON format

donaldwayne1975
Path Finder

Scenario:

  1. two different source types being sent to UF (snort and firewall) from the same IP/source.
  2. format of data is JSON (see below example)

    {
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }{
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }

I've tried using regexes like below to separate them.

props.conf

[udp:514]
TRANSFORMS-set_sourcetype = snort,firewall

transforms.conf

[snort]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

[firewall]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

Looking for the best method to assign the sourcetypes. I do not have control over the data structure or how it is sent to the UF.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

A UF doesn't parse the data, so this functionality must be done on the indexer, or an HF. Your title for the question specifically states doing it on a UF, so the answer is, you can't. But you can do it on the indexers or an HF.

0 Karma

to4kawa
Ultra Champion

Why do you want to separate sourcetypes?
JSON is extracted fields appropriately .

If you search , you can use type=snort or type=firewall .

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...