topic Re: Does Splunk recognize LEEF formatted? in Splunk Search

Does Splunk recognize LEEF formatted?

steveirogers — Mon, 06 Mar 2017 20:13:27 GMT

I am trying to import "LEEF" formatted data (from an IBM mainframe) into Splunk, but none of the name / value pairs are recognized. There is question in Splunk community from 2011 regarding this same issue which was not answered. Should I just use the manual field extraction for this type of data or is this a known log format which Splunk can handle?

See sample log event below:
"LEEF:1.0|IBM|RACF|2.2.1|80 27.0|devTimeFormat=yyyy-MM-dd'T'HH:mm:ss.SSSZ devTime=2017-02-27T14:01:47.630-0500 usrName=U020005 name=LISA DODARO usrPriv= usrGroups= ICTXname= ICTXreg= job=JB0 27 Feb 2017 14:01:46.26 U0200051 intent= allow= class=MXADMIN prof= res= vol= dsn= sens= own= box= terminal= poe= logstr=CSQH RESLEVEL CHECK PERFORMED AGAINST PROFILE(MQGP.RESLEVEL), CLASS(MXADMIN), ACCESS EQUATES TO (NONE) auth= desc=Success reason= appl= sum=RACF GENERAL success for U020005: logstr=CSQH RESLEVEL CHECK PERFORMED AGAINST PROFILE(MQGP.RESLEVEL), CLASS(MXADMIN), ACCESS EQUATES TO (NONE) cmd="

Re: Does Splunk recognize LEEF formatted?

woodcock — Mon, 06 Mar 2017 21:44:54 GMT

Like this...

In props.conf:

[YourSourcetypeHere]
TRANSFORMS-index_time_field_extractions = LEEF_KVP
#REPORT-search_time_field_extractions = LEEF_KVP

In transforms.conf:

[LEEF_KVP]
REGEX = (\w+)=([^=]+)(?:\s+|$)
FORMAT = $1::$2
MV_ADD = true

Re: Does Splunk recognize LEEF formatted?

steveirogers — Tue, 07 Mar 2017 04:35:26 GMT

Thanks very much for you prompt response. I will try adding those configurations.

Re: Does Splunk recognize LEEF formatted?

Dan — Mon, 13 Mar 2017 19:50:33 GMT

MV_ADD = [true|false]
* NOTE: This attribute is only valid for search-time field extractions.

Thus, you need to use REPORT- not TRANSFORMS-

Re: Does Splunk recognize LEEF formatted?

woodcock — Mon, 13 Mar 2017 19:53:25 GMT

Good point.

Re: Does Splunk recognize LEEF formatted?

steveirogers — Tue, 14 Mar 2017 04:25:22 GMT

Thanks Dan. Problem solved. Woodcock, thank you also for taking the time to responsd.

Best regards,
Steve Rogers

Re: Does Splunk recognize LEEF formatted?

woodcock — Tue, 14 Mar 2017 15:45:22 GMT

What was your final solution? Post it here and Accept it (or maybe you used mine, so click Accept on that one).

Re: Does Splunk recognize LEEF formatted?

steveirogers — Tue, 14 Mar 2017 16:18:54 GMT

I used the solution provided by Dan [Splunk]. Thanks again for your assistance.

Re: Does Splunk recognize LEEF formatted?

woodcock — Tue, 14 Mar 2017 23:14:04 GMT

Please do post the actual solution so that others can learn. That's the point.

Re: Does Splunk recognize LEEF formatted?

steveirogers — Tue, 29 Sep 2020 13:14:49 GMT

Working solution:
in props.conf:
[LEEF_csv]
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false
pulldown_type = true
REPORT-leef = LEEF_KVP

in transforms.conf:
[LEEF_KVP]
REGEX = (\w+)=([^=]+)(?:\s+|$)
FORMAT = $1::$2
MV_ADD = true

Re: Does Splunk recognize LEEF formatted?

steveirogers — Wed, 15 Mar 2017 00:01:21 GMT

Sorry about that. I thought everyone could see the code posted by Dan.