BREAK_ONLY_BEFORE not working

[tmarlette]

I have a clustered system that I am using, and I'm attempting to break events at the search head level, and it seems right though the events aren't breaking appropriately. I am attempting to break the event at the "Job <myJob>" portion of the messages below. also, for whatever reason, the 'code box' on here replaces the symbol "<" with an ampersand, and a "lt" just as a heads up.

This is an example of the log messages:
Job <myJob>, User <myuser>, Project <default>, Status <RUN>, Queue <normal>,
Command <myCommand>
Mon Apr 27 07:33:03: Submitted from host <myHost>, CWD <$HOME
>;
Mon Apr 27 07:33:04: Started on <myHost>, Execution Home </myHome>, Execution CWD <myCwd>;
Tue Apr 28 09:57:59: Resource usage collected.
The CPU time used is 1345 seconds.
MEM: 114 Mbytes; SWAP: 35.1 Gbytes; NTHREAD: 54
PGID: 9430; PIDs: 9430 9435 9437 9450 9501
PGID: 9461; PIDs: 9461

 MEMORY USAGE:
 MAX MEM: 4.2 Gbytes;  AVG MEM: 194 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem] 
------------------------------------------------------------------------------

Job <myJob>, User <myUser>, Project <default>, Status <RUN>, Queue <normal>, 
                     Interactive mode, Command </myCommand >
Tue Apr 28 01:42:38: Submitted from host <myHost>, CWD </myCwd;
Tue Apr 28 01:42:38: Started on <myHost>;
Tue Apr 28 09:58:00: Resource usage collected.
                     The CPU time used is 192 seconds.
                     MEM: 11 Mbytes;  SWAP: 1 Gbytes;  NTHREAD: 15
                     PGID: 20416;  PIDs: 20416 
                     PGID: 20425;  PIDs: 20425 20427 20442 


 MEMORY USAGE:
 MAX MEM: 11 Mbytes;  AVG MEM: 10 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem] 
------------------------------------------------------------------------------

Job <myJob>, User <myUser>, Project <default>, Status <RUN>, Queue <normal>, I
                     nteractive mode, Command </myCommand>
Tue Apr 28 02:47:25: Submitted from host <myHost>, CWD <myCwd
                     >;
Tue Apr 28 02:47:25: Started on <myHost>;
Tue Apr 28 09:58:26: Resource usage collected.
                     The CPU time used is 84 seconds.
                     MEM: 8 Mbytes;  SWAP: 928 Mbytes;  NTHREAD: 14
                     PGID: 25895;  PIDs: 25895 
                     PGID: 25898;  PIDs: 25898 25900 25915 


 MEMORY USAGE:
 MAX MEM: 8 Mbytes;  AVG MEM: 7 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem] 

and this is the my stanza in props.conf on the search head for this sourcetype:

[mySourcetype]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = \Job\s.\d+.

When i use regexr.com for this extraction it matches where I need to break the event, however it doesn't seem to be working here. Am I doing something wrong?

[tmarlette · Accepted solution]

This is due to the "BREAK_ONLY_BEFORE" being at the search head level, and not the indexing level. Thank you Rich!

in order for "BREAK_ONLY_BEFORE" to work successfully, it MUST be at the indexing tier.

[richgalloway]

Is <MyJob> literal text or a field? What kind of field?

Have you tried putting parens around the regex to create a matching group?

[tmarlette]

This is literal. In the log entry this is an integer which ends up looking like. The brackets are in the log entry as well.

Job <89238764>

I have not tried putting parens around it. I can give it a shot.

[richgalloway]

Your regex should match, except there's no need to escape the 'J'.

[tmarlette]

I agree. I don't know if I should be putting this at the indexing tier as well though. Do you know if this needs to be there as well?

[richgalloway]

Yes, you should be setting the props.conf file on your indexer(s).

[tmarlette]

I believe that would be the problem then. When i'm able to get that setting into my indexers, I will let know the results. Currently it only resides in the search heads.

Thank you!