BREAK_ONLY_BEFORE not working

tmarlette — Mon, 04 May 2015 18:00:41 GMT

I have a clustered system that I am using, and I'm attempting to break events at the search head level, and it seems right though the events aren't breaking appropriately. I am attempting to break the event at the "Job <myJob>" portion of the messages below. also, for whatever reason, the 'code box' on here replaces the symbol "<" with an ampersand, and a "lt" just as a heads up.

This is an example of the log messages:
Job <myJob>, User <myuser>, Project <default>, Status <RUN>, Queue <normal>,
Command <myCommand>
Mon Apr 27 07:33:03: Submitted from host <myHost>, CWD <$HOME
>;
Mon Apr 27 07:33:04: Started on <myHost>, Execution Home </myHome>, Execution CWD <myCwd>;
Tue Apr 28 09:57:59: Resource usage collected.
The CPU time used is 1345 seconds.
MEM: 114 Mbytes; SWAP: 35.1 Gbytes; NTHREAD: 54
PGID: 9430; PIDs: 9430 9435 9437 9450 9501
PGID: 9461; PIDs: 9461

 MEMORY USAGE:
 MAX MEM: 4.2 Gbytes;  AVG MEM: 194 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem] 
------------------------------------------------------------------------------

Job &lt;myJob&gt;, User &lt;myUser&gt;, Project &lt;default&gt;, Status &lt;RUN&gt;, Queue &lt;normal&gt;, 
                     Interactive mode, Command &lt;/myCommand &gt;
Tue Apr 28 01:42:38: Submitted from host &lt;myHost&gt;, CWD &lt;/myCwd;
Tue Apr 28 01:42:38: Started on &lt;myHost&gt;;
Tue Apr 28 09:58:00: Resource usage collected.
                     The CPU time used is 192 seconds.
                     MEM: 11 Mbytes;  SWAP: 1 Gbytes;  NTHREAD: 15
                     PGID: 20416;  PIDs: 20416 
                     PGID: 20425;  PIDs: 20425 20427 20442 


 MEMORY USAGE:
 MAX MEM: 11 Mbytes;  AVG MEM: 10 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem] 
------------------------------------------------------------------------------

Job &lt;myJob&gt;, User &lt;myUser&gt;, Project &lt;default&gt;, Status &lt;RUN&gt;, Queue &lt;normal&gt;, I
                     nteractive mode, Command &lt;/myCommand&gt;
Tue Apr 28 02:47:25: Submitted from host &lt;myHost&gt;, CWD &lt;myCwd
                     &gt;;
Tue Apr 28 02:47:25: Started on &lt;myHost&gt;;
Tue Apr 28 09:58:26: Resource usage collected.
                     The CPU time used is 84 seconds.
                     MEM: 8 Mbytes;  SWAP: 928 Mbytes;  NTHREAD: 14
                     PGID: 25895;  PIDs: 25895 
                     PGID: 25898;  PIDs: 25898 25900 25915 


 MEMORY USAGE:
 MAX MEM: 8 Mbytes;  AVG MEM: 7 Mbytes

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -   0.9       -     -    -     -     -      -      -  
 loadStop    -     -     -   0.9       -     -    -     -     -      -      -  

 RESOURCE REQUIREMENT DETAILS:
 Combined: select[type == local] order[ut:mem]
 Effective: select[type == local] order[ut:mem]

and this is the my stanza in props.conf on the search head for this sourcetype:

[mySourcetype]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = \Job\s.\d+.

When i use regexr.com for this extraction it matches where I need to break the event, however it doesn't seem to be working here. Am I doing something wrong?

Re: BREAK_ONLY_BEFORE not working

richgalloway — Mon, 04 May 2015 20:06:52 GMT

Is <MyJob> literal text or a field? What kind of field?

Have you tried putting parens around the regex to create a matching group?

Re: BREAK_ONLY_BEFORE not working

tmarlette — Mon, 04 May 2015 20:10:35 GMT

This is literal. In the log entry this is an integer which ends up looking like. The brackets are in the log entry as well.

Job <89238764>

I have not tried putting parens around it. I can give it a shot.

Re: BREAK_ONLY_BEFORE not working

richgalloway — Mon, 04 May 2015 20:13:48 GMT

Your regex should match, except there's no need to escape the 'J'.

Re: BREAK_ONLY_BEFORE not working

tmarlette — Mon, 04 May 2015 20:17:41 GMT

I agree. I don't know if I should be putting this at the indexing tier as well though. Do you know if this needs to be there as well?

Re: BREAK_ONLY_BEFORE not working

richgalloway — Mon, 04 May 2015 20:19:02 GMT

Yes, you should be setting the props.conf file on your indexer(s).

Re: BREAK_ONLY_BEFORE not working

tmarlette — Mon, 04 May 2015 22:58:53 GMT

I believe that would be the problem then. When i'm able to get that setting into my indexers, I will let know the results. Currently it only resides in the search heads.

Thank you!

Re: BREAK_ONLY_BEFORE not working

tmarlette — Mon, 28 Sep 2020 19:50:44 GMT

This is due to the "BREAK_ONLY_BEFORE" being at the search head level, and not the indexing level. Thank you Rich!

in order for "BREAK_ONLY_BEFORE" to work successfully, it MUST be at the indexing tier.

topic Re: BREAK_ONLY_BEFORE not working in Getting Data In

BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working

Re: BREAK_ONLY_BEFORE not working