Event Breaking for PROPS Configuration File

[SplunkDash]

Hello,

Please let me know how I would break the events, write TIME_PREFIX and TIME_FORMAT for my PROPS Conf.  file  for this sample source data events:

TIME_PREFIX=

TIME_FORMAT=

LINE_BREAKER=

BREAK_ONLY_BEFORE=

Sample data has 5 events.  I marked the text  as RED to indicate beginning of each events and time as GREEN

Thank you so much, greatly appreciated!

---------------------------Sample Data Starts-------------------

TCC     A TCU00002I 22.59.00 MFE REPORT LAST 5.0 MINUTES                                                     2021-06-14 00:00:09.420

TCC     A Server            TSID  I PKTS  O PKTS |Server            TSID  I PKTS  O PKTS                     2021-06-14 00:00:09.421

TCC     A VP2SMTBAPPICE10   VQME     607     623 |VP2SMTBAPPICE11   VQMF   629   661 _                       2021-06-14 00:00:09.422

TCC     A VP2SMTBAPPICE12   VQMG     603     605 |LAPKSC            UZ77     6     6                         2021-06-14 00:00:09.423

TCC     A VP2SMTBAPPICCE2   VPQJ     586     595 |VP2SMTBAPPICCE4   VPQK   600   618                         2021-06-14 00:00:09.424

TCC     A VP2SMTBAPPICCE5   VPQM       7       7 |VP2SMTBAPPICCE6   VPQN    11    11                         2021-06-14 00:00:09.425

TCC     A VP2SMTBAPPICCE7   VPQO      15      15 |VP2SMTBAPPCLS02   VXBK    13    13 _                       2021-06-14 00:00:09.426

TCC     A VP2SMTBAPPCLS03   VXBL      20      20 |VP2SMTBAPPCLS04   VXBM    11    11                         2021-06-14 00:00:09.427

TCC     A VP2SMEMAPPICCE1   VXBA     520     528 |VP2SMEMAPPICCE2   VXBB   548   560                         2021-06-14 00:00:09.428

TCC     A VP2SMEMAPPICCE3   VXBC     523     530 |VP2SMEMAPPICCE5   VXBE    28    28                         2021-06-14 00:00:09.429

TCC     A VP2SMEMAPPICCE6   VXBF      40      40 |VP2SMEMAPPICCE8   VXBH    25    28 _                       2021-06-14 00:00:09.430

TCC     A VD2SMEMAPPCLS02   VXBO      35      35 |VD2SMEMAPPCLS03   VXBP    49    49                         2021-06-14 00:00:09.431

TCC     A VD2SMEMAPPCLS04   VXBQ      40      40 |VP2SMEMAPPICE10   VQMB   526   537                         2021-06-14 00:00:09.432

TCC     A VP2SMEMAPPICE11   VQMC     602     609 |VP2SMEMAPPICE12   VQMD   486   486                         2021-06-14 00:00:09.433

TCC     A VP2SMTBAPPICE13   VQMH     565     572 |VP2SMEMAPPICCE4   VXBD   591   597 _                       2021-06-14 00:00:09.434

TCC     A VP2SMTBAPPCLS01   VXBJ      12      12 |VP2SMTBAPPICCE1   VPQI   565   580                         2021-06-14 00:00:09.435

TCC     A VP2SMTBAPPICCE4   VPQL     551     561 |VP2SMEMAPPICCE7   VXBG    40    40                         2021-06-14 00:00:09.436

TCC     A VD2SMEMAPPCLS01   VXBN      42      42 |VP2SMEMAPPICCE9   VQMA   528   535                         2021-06-14 00:00:09.437

TCC     A VP2SMTBAPPICCE8   VPQP       2       2 |                                                           2021-06-14 00:00:09.438

TCC     A                                                                                                    2021-06-14 00:00:09.439

TCC     A PID POOL PIDS IN USE: 1312 OUT OF 3001                                                             2021-06-14 00:00:09.440

TCC     A END OF MFE REPORT+

TCC     A CVZB0001I 22.59.00 LAST FALLBACK COPY OF CP KEYPOINTS ON SYMBOLIC                                  2021-06-14 00:00:09.442

TCC     A MODULE: 010A DEVICE: 710A+                                                                         2021-06-14 00:00:09.443

TCC     A TCPF0001I 22.59.00 TCP KEYPOINTED+                                                                 2021-06-14 00:00:09.444

TCC     A OCC10000I 22.59.02 RMT HOST-A CCMOD DSBL ERSS AT+                                                  2021-06-14 00:00:11.445

TCC     A OCC10013I 22.59.02 *MEH1PRD* COMMAND CODE(S) DISABLED BY RMT HOST+                                 2021-06-14 00:00:11.446

TCC     A COMMAND CODE DISPLAY                                                                               2021-06-14 00:00:11.447

 ------------------------Sample Data Ends---------------------------

[richgalloway]

What is the rule which defines when a new event starts?  I ask because there is no discernable pattern to the red text in the sample events.

[SplunkDash]

Thank you so much, appreciated.

In regards to rule to starts a new event, TCU00002I 22.59.00 ...this "9 alphanumeric characters" plus "1 space" plus  "8" digits (including  2 ".") is the indication of event starts. That means "TCU00002I 22.59.00" indicates the starting of event.

[richgalloway]

"9 alphanumeric characters" is \w{9}

"plus 1 space"  is \s

"plus  8 digits (including  2 ".")" is [\d\.]{8}

Put it all together to get this LINE_BREAKER

LINE_BREAKER = ([\r\n]+)TCC\s+A\s\w{9}\s[\d\.]{8}

[SplunkDash]

Thank you so much, appreciated your support. Event breaking working as expected...cool.  Issue with TIME_PREFIX and TIME_FORMAT, TIME is not extracted  and giving  error message. Here is my full PROPS configuration file. Any help will be highly appreciated, thank you again!

[ <SOURCETYPE NAME> ]

SHOULD_LINEMERGE=false

LINE_BREAKER=([\r\n]+)\w{3}\s+A\s\w{9}\s[\d\.]{8}

NO_BINARY_CHECK=true

CHARSET=UTF-8

TIME_PRIFIX=\w{3}\s+A\s\w{9}\s

TIME_FORMAT=%m.%d.%y

MAX_TIMESTAMP_LOOKAHEAD=30

[richgalloway]

What is the error message?

"TIME_PRIFIX" should be "TIME_PREFIX"

The TIME_FORMAT setting is incorrect.  None of the sample data shown has a timestamp in m.d.y format.  Try TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N

[SplunkDash]

Please see the following PROPS Conf. It is working as expected....does it make sense to you. Thank you ...greatly appreciated.....

[ <SOURCETYPE NAME> ]

SHOULD_LINEMERGE=false

LINE_BREAKER=([\r\n]+)\w{3}\s+A\s\w{9}\s[\d\.]{8}

CHARSET=UTF-8

TIME_PREFIX=\s+

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3F

MAX_TIMESTAMP_LOOKAHEAD=250

[richgalloway]

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.

[SplunkDash]

Ok I tried with this as last time take as a time format, but still getting error. Thank you so much, appreciated.

[ <SOURCETYPE NAME> ]

SHOULD_LINEMERGE=false

LINE_BREAKER=([\r\n]+)\w{3}\s{5}A\s\w{9}\s[\d\.]{8}

NO_BINARY_CHECK=true

CHARSET=UTF-8

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N

MAX_TIMESTAMP_LOOKAHEAD=200

TIME_PREFIX=\s+

[richgalloway · Accepted solution]

The TIME_FORMAT value needs to exactly match the format in the event or Splunk won't find it. 

I found the green text. Try these settings

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\w{3}\s{5}A\s\w{9}\s[\d\.]{8}
TIME_FORMAT=%H.%M.%S
MAX_TIMESTAMP_LOOKAHEAD=200
TIME_PREFIX=TCC\s+A\s\w{9}\s

[SplunkDash]

Yes, working as expected considering Green text as Time, thank you so much....

Please let me know if following PROPS configuration makes sense to you ... if I consider  2021-06-14 00:00:09.422  as Time Format. I am getting result as expected!!!

[ <SOURCETYPE NAME> ]

SHOULD_LINEMERGE=false

LINE_BREAKER=([\r\n]+)\w{3}\s+A\s\w{9}\s[\d\.]{8}

CHARSET=UTF-8

TIME_PREFIX=\s+

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3F

MAX_TIMESTAMP_LOOKAHEAD=250

[SplunkDash]

Thank you so much again. I know what you mean. But, 22.59.00......represents time....I marked it as Green in the sample events provided. Thank you, appreciated!!!