Knowledge Management

How to extract the given log into one log using props.conf?

murali18
Engager

I am trying to extract the below file into single log, but it got breaks into two or more files in splunk

Sample file :

PING 20.152.32.XXX (20.152.32.XXX) 56(84) bytes of data.
64 bytes from 20.152.32.XXX: icmp_seq=1 ttl=248 time=67.9 ms
64 bytes from 20.152.32.XXX: icmp_seq=2 ttl=248 time=68.2 ms
64 bytes from 20.152.32.XXX: icmp_seq=3 ttl=248 time=68.1 ms
64 bytes from 20.152.32.XXX: icmp_seq=4 ttl=248 time=68.2 ms

--- 20.152.32.XXX ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 67.926/68.153/68.276/0.134 ms

What need to changed in the props.conf

[lala_pop]
BREAK_ONLY_BEFORE = PING\s+\d+.\d+.\d+.\d+
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true

Appreciate you help. Thanks

Tags (1)
0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

If the content of log file is overwritten every time then you don't need to BREAK the events. Just set SHOULD_LINEMERGE to true and check.

0 Karma

PavelP
Motivator

try to use LINE_BREAKER instead of BREAK_ONLY_BEFORE:

[lala_pop]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)PING\s+\d+\.\d+\.\d+\.\d+
NO_BINARY_CHECK=true

the difference between BREAK_ONLY_BEFORE and LINE_BREAKER:

BREAK_ONLY_BEFORE = <regular expression>
* When set, Splunk software creates a new event only if it encounters a new
  line that matches the regular expression.
* Default: empty string

:

LINE_BREAKER = <regular expression>
* Specifies a regex that determines how the raw text stream is broken into
  initial events, before line merging takes place. (See the SHOULD_LINEMERGE
  setting, below.)
* The regex must contain a capturing group -- a pair of parentheses which
  defines an identified subcomponent of the match.
* Wherever the regex matches, Splunk software considers the start of the first
  capturing group to be the end of the previous event, and considers the end
  of the first capturing group to be the start of the next event.
* The contents of the first capturing group are discarded, and are not
  present in any event. You are telling Splunk software that this text comes
  between lines.
* NOTE: You get a significant boost to processing speed when you use
  LINE_BREAKER to delimit multi-line events (as opposed to using
  SHOULD_LINEMERGE to reassemble individual lines into multi-line events).
  * When using LINE_BREAKER to delimit events, SHOULD_LINEMERGE should be set
    to false, to ensure no further combination of delimited events occurs.
  * Using LINE_BREAKER to delimit events is discussed in more detail in the
    documentation. Search the documentation for "configure event line breaking"
    for details.
* Default: ([\r\n]+) (Data is broken into an event for each line,
  delimited by any number of carriage return or newline characters.)
0 Karma

murali18
Engager

Hi,
Thanks for the reply.
There is only single log in the destination file and is overwriting every time.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...