Break events based on a string

[Nadhiyaa]

hi ,

Below is my single event indexing into splunk.I want to break the events into single events .It should break and comes into the next row after the string "SNMPv2- SMI::enterprises" .i got stuck writing regex on this . Kindly help.

SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29825" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29865" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22018.41032.29826" = "0" SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22018.41032.29863" = "0" SNMPv2-SMI

[woodcock]

Google Splunk Magic 6 or Splunk Magic 8 and read up on how bad it is to let Splunk merge events and guess at timestamps. You need EXACTLY these settings in props.conf:

[yourSourcetypeHere]
SHOULD_LINEMERGE = false
#LINE_BREAKER = ([\n\r\s]*)SNMPv2-SMI::enterprises\.
LINE_BREAKER = ([\n\r\s]*SNMPv2-SMI::enterprises\.)
#EVENT_BREAKER = ([\n\r\s]*)SNMPv2-SMI::enterprises\.
EVENT_BREAKER = ([\n\r\s]*SNMPv2-SMI::enterprises\.)
EVENT_BREAKER_ENABLE = true
DATETIME_CONFIG = CURRENT

You can save license by stripping off the SNMPv2-SMI::enterprises. or you can keep it.

[Nadhiyaa]

@woodcock

above props.conf works when we manually add the data .
As we are getting the snmp data via our heavy forwarder somehow its not taking the props.conf and its not working. Specified the props in search head and indexers as well but none worked .

[woodcock]

It must be deployed to the HFs, not the Indexers.

[pgadhari]

yes, we have deployed it on the HF, but still it is not working ? Can you further advise on this issue ?

[DMohn]

Won't this configuration cut the SNMPv2-SMI::enterprises part out of the event? I don't know if this is intended. Thats why I included this in a non-capturing group in my example.

But apart from that you are correct, if data comes in via UF, you should always inclund EVENT_BREAKER config!

[woodcock]

Swap for the one that is commented out.

[DMohn]

Add the following line to your appropriate props.conf stanza:

[your_sourcetype]
LINE_BREAKER = (?:SNMPv2-SMI::enterprises)(.)

It is always encouraged to use the LINE_BREAKER stanza where possible. THis will take (at least) one RegEx capturing group, which marks the end of one event and will be discarded. So in your case, as the end of your events is represented by the string "SNMPv2-SMI::enterprises" this needs to come before the end-of-event-group and hence is a non-capturing group. The single charakter thereafter is being discarded, and the next event starts right after.

[pgadhari]

@DMohn - If you see the logs properly, the event starts with "SNMPv2-SMI::enterprises", and ends with third "space", in between there are 2 spaces.

e.g :

SNMPv2-SMI::enterprises."2011.2.281.2.1.20008.41032.1044.1.1.26.33555088.41025.22017.41032.29825" = "0"
so ideally, we need to break at the last "Space" of that event i.e. third space. I have attached the image for the event and marked the space with red where line break should occur.

https://ibb.co/cCqSsXg

How do we write the capture regex for that ?

Strange behaviour is when we export sample raw logs and upload from local manually, the suggested regex works. But when we put that on HF, it does not works. We are restarting splunk after the change on HF.

[Nadhiyaa]

Hi ,@DMohn @lakshman239
Below props.conf worked when i added the data manually .but not working when specified in the heavy forwarder .Tried specified in both indexer and search head as well. Please help what could be the issue .

[sourcetype]
BREAK_ONLY_BEFORE = ([\s]+)(?=SNMPv2-SMI::enterprises)
DATETIME_CONFIG =
LINE_BREAKER = ([\s]+)(?=SNMPv2-SMI::enterprises)

[woodcock]

This is a very poor configuration, if it even works. Do not use this.

[lakshman239]

In your props.conf, can you try with MUST_BREAK_AFTER=([\r\n]+)(?=SNMPv2- SMI::enterprises)

Also, send your props.conf