topic Re: Break events based on a string in Splunk Search

Break events based on a string

Nadhiyaa — Thu, 21 Mar 2019 13:11:36 GMT

hi ,

Below is my single event indexing into splunk.I want to break the events into single events .It should break and comes into the next row after the string "SNMPv2- SMI::enterprises" .i got stuck writing regex on this . Kindly help.

SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29825" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29865" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22018.41032.29826" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22018.41032.29863" = "0" SNMPv2-SMI

Re: Break events based on a string

lakshman239 — Thu, 21 Mar 2019 13:20:32 GMT

In your props.conf, can you try with MUST_BREAK_AFTER=([\r\n]+)(?=SNMPv2- SMI::enterprises)

Also, send your props.conf

Re: Break events based on a string

DMohn — Thu, 21 Mar 2019 15:10:25 GMT

Add the following line to your appropriate props.conf stanza:

[your_sourcetype]
LINE_BREAKER = (?:SNMPv2-SMI::enterprises)(.)

It is always encouraged to use the LINE_BREAKER stanza where possible. THis will take (at least) one RegEx capturing group, which marks the end of one event and will be discarded. So in your case, as the end of your events is represented by the string "SNMPv2-SMI::enterprises" this needs to come before the end-of-event-group and hence is a non-capturing group. The single charakter thereafter is being discarded, and the next event starts right after.

Re: Break events based on a string

Nadhiyaa — Tue, 29 Sep 2020 23:48:12 GMT

Hi ,@DMohn @lakshman239
Below props.conf worked when i added the data manually .but not working when specified in the heavy forwarder .Tried specified in both indexer and search head as well. Please help what could be the issue .

[sourcetype]
BREAK_ONLY_BEFORE = ([\s]+)(?=SNMPv2-SMI::enterprises)
DATETIME_CONFIG =
LINE_BREAKER = ([\s]+)(?=SNMPv2-SMI::enterprises)

Re: Break events based on a string

woodcock — Sun, 24 Mar 2019 13:03:18 GMT

Google Splunk Magic 6 or Splunk Magic 8 and read up on how bad it is to let Splunk merge events and guess at timestamps. You need EXACTLY these settings in props.conf:

[yourSourcetypeHere]
SHOULD_LINEMERGE = false
#LINE_BREAKER = ([\n\r\s]*)SNMPv2-SMI::enterprises\.
LINE_BREAKER = ([\n\r\s]*SNMPv2-SMI::enterprises\.)
#EVENT_BREAKER = ([\n\r\s]*)SNMPv2-SMI::enterprises\.
EVENT_BREAKER = ([\n\r\s]*SNMPv2-SMI::enterprises\.)
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = CURRENT

You can save license by stripping off the SNMPv2-SMI::enterprises. or you can keep it.

Re: Break events based on a string

woodcock — Sun, 24 Mar 2019 13:04:20 GMT

This is a very poor configuration, if it even works. Do not use this.

Re: Break events based on a string

DMohn — Mon, 25 Mar 2019 07:14:02 GMT

Won't this configuration cut the SNMPv2-SMI::enterprises part out of the event? I don't know if this is intended. Thats why I included this in a non-capturing group in my example.

But apart from that you are correct, if data comes in via UF, you should always inclund EVENT_BREAKER config!

Re: Break events based on a string

Nadhiyaa — Mon, 25 Mar 2019 09:36:48 GMT

@woodcock

above props.conf works when we manually add the data .
As we are getting the snmp data via our heavy forwarder somehow its not taking the props.conf and its not working. Specified the props in search head and indexers as well but none worked .

Re: Break events based on a string

woodcock — Mon, 25 Mar 2019 10:14:04 GMT

It must be deployed to the HFs, not the Indexers.

Re: Break events based on a string

woodcock — Mon, 25 Mar 2019 10:19:55 GMT

Swap for the one that is commented out.

Re: Break events based on a string

pgadhari — Tue, 26 Mar 2019 05:30:59 GMT

yes, we have deployed it on the HF, but still it is not working ? Can you further advise on this issue ?

Re: Break events based on a string

pgadhari — Tue, 26 Mar 2019 11:29:12 GMT

@DMohn - If you see the logs properly, the event starts with "SNMPv2-SMI::enterprises", and ends with third "space", in between there are 2 spaces.

e.g :

SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29825" = "0"
so ideally, we need to break at the last "Space" of that event i.e. third space. I have attached the image for the event and marked the space with red where line break should occur.

https://ibb.co/cCqSsXg

How do we write the capture regex for that ?

Strange behaviour is when we export sample raw logs and upload from local manually, the suggested regex works. But when we put that on HF, it does not works. We are restarting splunk after the change on HF.