Use RegEx to set sourcetype on UF where events are in a JSON format

[donaldwayne1975]

Scenario:

1. two different source types being sent to UF (snort and firewall) from the same IP/source.

2. format of data is JSON (see below example)

   {
   "client": {"ip": "...",...
   ...
   "type": "snort"
   }

   {
   "fw": {"iptables": "...",...
   "client": {"ip": "...",...
   ...
   "type": "firewall"
   }{
   "client": {"ip": "...",...
   ...
   "type": "snort"
   }

   {
   "fw": {"iptables": "...",...
   "client": {"ip": "...",...
   ...
   "type": "firewall"
   }

I've tried using regexes like below to separate them.

props.conf

[udp:514]
TRANSFORMS-set_sourcetype = snort,firewall

transforms.conf

[snort]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

[firewall]
REGEX = ^(.*?type\"\:)(?P<type>("([^}]|"")*"))
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::$type

Looking for the best method to assign the sourcetypes. I do not have control over the data structure or how it is sent to the UF.

[cpetterborg]

A UF doesn't parse the data, so this functionality must be done on the indexer, or an HF. Your title for the question specifically states doing it on a UF, so the answer is, you can't. But you can do it on the indexers or an HF.

[to4kawa]

Why do you want to separate sourcetypes?
JSON is extracted fields appropriately .

If you search , you can use type=snort or type=firewall .