topic Re: Inline/Uses Transform Field Extraction in Splunk Search

Inline/Uses Transform Field Extraction

SplunkDash — Thu, 02 Dec 2021 17:50:20 GMT

Hello,

How would I implement inline or Uses Transform Field extraction (please see screenshot below) for following event (please see sample event below). Any help will be highly appreciated, thank you so much.

Screenshot (inline field extraction option)

One Sample Event

{"log":"\u001b[0m\u001b[0m05:14:09,516 INFO [stdout] (default task-4193) 2021-12-02 05:14:09,516 INFO [tltest.logging.TltestEventWriter] \u003cMODTRANSAUDTRL\u003e\u003cEVENTID\u003e1210VIEW\u003c/EVENTID\u003e\u003cEVENTTYPE\u003eDATA_INTERACTION\u003c/EVENTTYPE\u003e\u003cSRCADDR\u003e192.131.8.1\u003c/SRCADDR\u003e\u003cRETURNCODE\u003e00\u003c/RETURNCODE\u003e\u003cSESSIONID\u003etfYU4-AEPnEzZg\u003c/SESSIONID\u003e\u003cSYSTEM\u003eTLCATS\u003c/SYSTEM\u003e\u003cTIMESTAMP\u003e20211202051409\u003c/TIMESTAMP\u003e\u003cUSERID\u003eAX3BLNB\u003c/USERID\u003e\u003cUSERTYPE\u003eAdmin\u003c/USERTYPE\u003e\u003cVARDATA\u003eCASE NUMBER, CASE NAME;052014011348000,BANTAM LLC\u003c/VARDATA\u003e\u003c/MODTRANSAUDTRL\u003e\n","stream":"stdout","time":"2021-12-02T05:14:09.517228451Z"}

Re: Inline/Uses Transform Field Extraction

richgalloway — Thu, 02 Dec 2021 18:44:46 GMT

What is it you wish to extract from the event or transform it to?

Re: Inline/Uses Transform Field Extraction

SplunkDash — Thu, 02 Dec 2021 20:57:18 GMT

Thank you so much for you response, appreciate it. Actually I would like to extract fields from there, list of the fields are:

TIMESTAMP, USERTYPE, USERID, SYSTEM, EVENTTYPE, EVENTID, SRCADDR, SESSIONID, TAXPERIOD, RETURNCODE, TAXFILERTIN, VARDATA.

Re: Inline/Uses Transform Field Extraction

richgalloway — Fri, 03 Dec 2021 17:00:42 GMT

Is the sample event exactly how Splunk receives it or has it interpreted by another program. It looks like XML except the < and > characters are encoded.

Re: Inline/Uses Transform Field Extraction

SplunkDash — Fri, 03 Dec 2021 17:07:37 GMT

Hello,

Thank you for your response, truly appreciate it. Yes, this is raw event...... exactly how Splunk receives it. Thank you.

Re: Inline/Uses Transform Field Extraction

richgalloway — Fri, 03 Dec 2021 17:34:28 GMT

Assuming the elements are always in the same order then this regex should do it.

\\u003cEVENTID\\u003e(?<EVENTID>[^\\]+).*?\\u003cEVENTTYPE\\u003e(?<EVENTTYPE>[^\\]+).*?\\u003cSRCADDR\\u003e(?<SRCADDR>[^\\]+).*?\\u003cRETURNCODE\\u003e(?<RETURNCODE>[^\\]+).*?\\u003cSESSIONID\\u003e(?<SESSIONID>[^\\]+).*?\\u003cSYSTEM\\u003e(?<SYSTEM>[^\\]+).*?\\u003cTIMESTAMP\\u003e(?<TIMESTAMP>[^\\]+).*?\\u003cUSERID\\u003e(?<USERID>[^\\]+).*?\\u003cUSERTYPE\\u003e(?<USERTYPE>[^\\]+).*?\\u003cVARDATA\\u003e(?<VARDATA>[^\\]+)

Re: Inline/Uses Transform Field Extraction

SplunkDash — Fri, 03 Dec 2021 17:42:24 GMT

Hello, thank you so much. appreciated and working as expecting if events are in the same order and no missing values are there. But only problem is that there are missing values in some of the events, causing issue with this extraction. Like some events have TRASACTIONCODE field with same structure after RETURNCODE some events are not. Are there any ways we can address those issues?

Re: Inline/Uses Transform Field Extraction

richgalloway — Fri, 03 Dec 2021 18:07:09 GMT

In that case, you would need a separate regex/transform for each field you wish to extract.