topic Re: Normalizing IBM log collector data into Splunk fields in Splunk Search

Normalizing IBM log collector data into Splunk fields

johnward4 — Tue, 29 Sep 2020 15:33:45 GMT

I'm trying to monitor log data that is displayed below, and extract the fields into ones that can be used in Splunk

Nov 6 07:51:03 S10125BA QAUDJRN: [JS@0 event="JS-Actions that affect jobs" event_type="S-Start" sev="1" actual_type="JS-S" job_type="Subsystem monitor" job_sub_type="No subtype" chg_job="ALLSYL100" chg_user="QSYS" chg_job_no="866512" effective_user="QSYS" jobd_name="" jobd_library="" jobq_name="" jobq_library="" outq_name="*DEV" outq_library="" printer_device="PRT01" library_list="QSYS QSYS2 QHLPSYS QUSRSYS QGPL QTEMP" eff_group_prf="" supplemental="" jrn_seq="9863803" timestamp="20161106075103429000" job_name="ALLSYL100" user_name="QSYS" job_number="866512" eff_user="QSYS" logical_partition="001" admin_user="yes"]

The log should begin with JS@O event= The fields I'm Most interested in are :

JS@O event
event_type
actual_type
job_type
effective_user
timestamp
job_name
job_number
admin_user

I've tried using the Splunk field extractor but have had no luck pulling out the fields I need. Please help

Re: Normalizing IBM log collector data into Splunk fields

gcusello — Wed, 30 Aug 2017 10:19:15 GMT

Hi johnward4,
when you have in your logs a pair "field=value" Splunk automatically extract fieds and shows them in Relevant Fields, so you don't need to create a custom extraction using regex.
Probably in your case the first field has a wrong name because there is a space between "JS@O" and "event" and because "@" is a special character that cannot be used in field names, but anyway it should be read as "event".
If you don't have automatic extraction, you could use regexes like the following:

JS@0 event\=\"(?<event>[^\"]*)\"

Bye.
Giuseppe

Re: Normalizing IBM log collector data into Splunk fields

johnward4 — Wed, 30 Aug 2017 18:47:18 GMT

I'm trying to test the regex extraction as Splunk is not auto extracting the fields. I need to first regex the timestamp that begin the log in Nov 6 07:51:03 format.

Then cut and start the next extracted field beginning after [ and extract and create fields for each field=value pair. Ending at the closing ].