Knowledge Management

How do I change the event boundaries of a syslog file from the mainframe

usernamejpblais
Engager

Hi! I created a new sourcetype (syslog_sic) because I have a syslog file coming from the mainframe with multiple line event that I want to break at each timestamp. My timestamp defenition is "2019099 00:24:48.71" meanning 2019=year 099=number of day in the year. When the data get indexed, it reconnized the time but not the date. The event break is set to breaking at each timestamp but instead it is breaking at each line.

0 Karma
1 Solution

rmjharris
Path Finder

In props.conf

Simplest
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N

Better but with specific regex based on the small sample you provided.
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \w{5}\s\w{7}\s\w{4}\s
LINE_BREAKER = ([\r\n]+)(?=\w{5}\s\w{7}\s\w{4}\s\d{7})
SHOULD_LINEMERGE = false

View solution in original post

CurtisGannaway
Loves-to-Learn

Hi @usernamejpblais,

If you need to get mainframe data (security, database, CICS, FTP, TCPIP, master console messages and much more), please see dgtechllc.com/meas. Our Mainframe Event Acquisition System (MEAS) product will allow you to monitor, filter and forward - in real time - any/all events from the mainframe that you would like to see in Splunk. It take roughly 1/2 day to install and no IPL necessary. Let me know if this solution could help you out. 

 

Thanks!

0 Karma

rmjharris
Path Finder

In props.conf

Simplest
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N

Better but with specific regex based on the small sample you provided.
[syslog_sic]
TIME_FORMAT = %Y%j %H:%M:%S.%N
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \w{5}\s\w{7}\s\w{4}\s
LINE_BREAKER = ([\r\n]+)(?=\w{5}\s\w{7}\s\w{4}\s\d{7})
SHOULD_LINEMERGE = false

usernamejpblais
Engager

Super!!!

Thanks mjharris!

0 Karma

koshyk
Super Champion

please provide atleast 4-5 lines to see how the sample data looks like

0 Karma

usernamejpblais
Engager

Hello Koshyk!

Thanks for you're help!

H158N 4020000 H158 2019099 00:24:47.97 STC67273 00000080 XCOMM0780E Txpi 227: Socket received
H158S Last error: 167
H158N 4020000 H158 2019099 00:24:47.97 STC67273 00000080 XCOMM0805I TCP/IP CONNECTION END
H158N 0002000 H158 2019099 00:24:48.11 STC64107 00000090 PGTV1710E TCPERR 00050000 on READ
H158S CONNECTION CLOSED PREMATURELY
H158M 0000000 H158 2019099 00:24:48.33 STC66246 00000090 CECA0143I The subscription heartbeat
H158S 779
H158D 779 00000090 DATASRC=IMS SUBSTATE=REPLICATE
H158D 779 00000090 PE=Active/Standby LATENCYSTATE=No
H158E 779 00000090 COMMITS=0 ABSBOOKMARK=2019-04-
H158N FDE0000 H158 2019099 00:24:48.71 STC66280 00000281 HWSP1415E TCP/IP SOCKET FUNCTION
H158S , M=SDRC, ID=DELDUMMY,IPv4=10.250.1

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...