Splunk Search

Why is my data not parsing correctly?

mhouse3
Path Finder

I am trying to make sure I know how to configure an environment to ingest weblogs that are correctly parsed and I am running into trouble in that I am only getting 1 single event. I have used feedback provided to similar questions to build out my configurations. Note that the original intent of this exercise was to see what would the different effect be with two different props.conf.

My weblog source is this on both forwarders:

  '<photo id="123" title="Birthday" format="jpg">
       <owner id="1111">Jason</owner>
    <CreationDate>2009-11-06T02:22:37.063</CreationDate>
       <comments>
           <comment ownerid="112">Good pic!</comment>
           <comment ownerif="223">Happy birthday</comment>
       <comments>
   </photo>


  <photo id="123" title="Birthday" format="jpg">
       <owner id="1111">Jason</owner>
    <CreationDate>2009-11-06T02:22:37.063</CreationDate>
       <comments>
           <comment ownerid="112">Good pic!</comment>
           <comment ownerif="223">Happy birthday</comment>
       <comments>
   </photo>


  <photo id="123" title="Birthday" format="jpg">
       <owner id="1111">Jason</owner>
    <CreationDate>2009-11-06T02:22:37.063</CreationDate>
       <comments>
           <comment ownerid="112">Good pic!</comment>
           <comment ownerif="223">Happy birthday</comment>
       <comments>
   </photo>'

My inputs.conf on FW1is this:
'[monitor:///home/labuser/xmldata/]
index=web
sourcetype=xml
disabled=false'

My inputs.conf on FW2 is this so that I could figure out which props.conf works:
'[monitor:///home/labuser/xmldata/]
index=web2
sourcetype=xml2
disabled=false'

My props.conf on FW1 is this:
'KV_MODE = xml
LINE_BREAKER = ()
MUST_BREAK_AFTER = \
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0
TIME_PREFIX = \
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N'

My props.conf on FW2 is this:
'KV_MODE = xml
LINE_BREAKER = ([\r\n]+)()
MUST_BREAK_AFTER = \
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0
TIME_PREFIX = \
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N'

All the data in both web index and web2 index looks identical in Splunk that both index=web1 or index=web2 produce identical results in that I only get a single event back instead of multiple events. What am I doing wrong?

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

For LINE_BREAKER, regex captured group should contain everything between end of previous event and beginning of current event. Try this:

[xml]
KV_MODE = xml
LINE_BREAKER = \<\/photo\>([\r\n\s]+)\<photo
NO_BINARY_CHECK = 1
TRUNCATE = 0
TIME_PREFIX = \<CreationDate\>
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
0 Karma

mhouse3
Path Finder

@manjunathmeti I just realized that my questions got cut off.

I was actually using:
LINE_BREAKER = ()

and later tried using this:
LINE_BREAKER = ([\r\n]+)()

In both cases all the data came in as a single event. These should have worked right?

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...