topic Confuguring events for AS400 logs in Getting Data In

Confuguring events for AS400 logs

yasinmoha — Tue, 09 Aug 2016 06:11:06 GMT

Hi...

I have a AS400 syslog file. for which I am want to configure splunk to pick up the events at every 2 lines.
Please advise which is the best way to do it

Sample log

5761SS1 V6R1M0 080215                                   History Log                                                   Page  0001
MSGID    SEV MSG TYPE

Start of the event

CPF1124  00  INFO         Job 252582/QTMHHTTP/QIWIRELESS started on 07/04/16 at 00:00:21 in subsystem QHTTPSVR in QHTTPSVR. Job ent
                  QIWIRELESS QTMHHTTP   252582 QWTPIIPP     0000 07/04/16 00:00:22.002029 QTMHHTTP

End of the Event

CPF1164  00  COMPLETION   Job 252582/QTMHHTTP/QIWIRELESS ended on 07/04/16 at 00:00:22; .034 seconds used; end code 0
                  QIWIRELESS QTMHHTTP   252582 QWTMCEOJ     0000 07/04/16 00:00:22.701533 QTMHHTTP
CPF1124  00  INFO         Job 252583/QPM400/Q1PDR started on 07/04/16 at 00:00:23 in subsystem QSYSWRK in QSYS. Job entered system
                  Q1PDR      QPM400     252583 QWTPIIPP     0000 07/04/16 00:00:23.286025 QPM400

Re: Confuguring events for AS400 logs

yasinmoha — Tue, 09 Aug 2016 08:10:54 GMT

I thought of adding the regex .\n. in /opt/splunk/etc/system/local/props.conf but could not locate the file.

Re: Confuguring events for AS400 logs

inventsekar — Tue, 09 Aug 2016 08:48:09 GMT

the question is bit not clear. may we know, some more info please..
1. from this above sample log msg, do you want to query this event ah?
Start of the event

CPF1124 00 INFO Job 252582/QTMHHTTP/QIWIRELESS started on 07/04/16 at 00:00:21 in subsystem QHTTPSVR in QHTTPSVR. Job ent
QIWIRELESS QTMHHTTP 252582 QWTPIIPP 0000 07/04/16 00:00:22.002029 QTMHHTTP <<<<
End of the Event
2. the above sample log is a single event or multiple events?
3. from this sample log, can you tell us, how to find out the "start of the event" and "end of the event"..

Re: Confuguring events for AS400 logs

yasinmoha — Tue, 29 Sep 2020 10:32:45 GMT

I have ingested this log in splunk but at some places it take 1 line as event and at some places it takes 2 or 3 lines as the event.
An event spans over 2 lines so I am trying to create a single event for every two lines. In the UI I gave the regex .\n. but it picks up 257 lines.
Also I tried to edit $SPLUNK_HOME/etc/system/local/props.conf but could not locate this file. thinking of adding the

SHOULD_LINEMERGE= true
MUST_BREAK_AFTER = .\n.
MAX_EVENTS = 2

Event 1
5761SS1 V6R1M0 080215 History Log Page 0001
MSGID SEV MSG TYPE
Event 2
CPF1124 00 INFO Job 252582/QTMHHTTP/QIWIRELESS started on 07/04/16 at 00:00:21 in subsystem QHTTPSVR in QHTTPSVR. Job ent
QIWIRELESS QTMHHTTP 252582 QWTPIIPP 0000 07/04/16 00:00:22.002029 QTMHHTTP
Event 3
CPF1164 00 COMPLETION Job 252582/QTMHHTTP/QIWIRELESS ended on 07/04/16 at 00:00:22; .034 seconds used; end code 0
QIWIRELESS QTMHHTTP 252582 QWTMCEOJ 0000 07/04/16 00:00:22.701533 QTMHHTTP
Event 4
CPF1124 00 INFO Job 252583/QPM400/Q1PDR started on 07/04/16 at 00:00:23 in subsystem QSYSWRK in QSYS. Job entered system
Q1PDR QPM400 252583 QWTPIIPP 0000 07/04/16 00:00:23.286025 QPM400

Re: Confuguring events for AS400 logs

yasinmoha — Tue, 09 Aug 2016 12:42:08 GMT

Hi...

I am trying to create a single event for every 2 lines of the log. When I ingest to splunk it some times take 1 or 2 or 3 lines as a single event. like below

Re: Confuguring events for AS400 logs

yasinmoha — Fri, 12 Aug 2016 20:09:26 GMT

Made a few changes under the advanced tab while ingesting the data

SHOULD_LINEMERGE=True
BREAK_ONLY_BEFORE=^\S
MAX_EVENTS=2

And then the event breaking got configured properly.