Getting Data In

Use RegEx to set sourcetype on UF where events are in a JSON format

donaldwayne1975
Path Finder

Scenario:

  1. two different source types being sent to UF (snort and firewall) from the same IP/source.
  2. format of data is JSON (see below example)

    {
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }{
    "client": {"ip": "...",...
    ...
    "type": "snort"
    }

    {
    "fw": {"iptables": "...",...
    "client": {"ip": "...",...
    ...
    "type": "firewall"
    }

I've tried using regexes like below to separate them.

props.conf

[udp:514]
TRANSFORMS-set_sourcetype = snort,firewall

transforms.conf

[snort]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

[firewall]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

Looking for the best method to assign the sourcetypes. I do not have control over the data structure or how it is sent to the UF.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

A UF doesn't parse the data, so this functionality must be done on the indexer, or an HF. Your title for the question specifically states doing it on a UF, so the answer is, you can't. But you can do it on the indexers or an HF.

0 Karma

to4kawa
Ultra Champion

Why do you want to separate sourcetypes?
JSON is extracted fields appropriately .

If you search , you can use type=snort or type=firewall .

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...