Getting Data In

Parsing XML and props.conf help

jtashker
New Member

Let me first preface this by saying that I am a total Splunk newbie and this is very similar to a lot of XML parsing questions, however I have read through ~20 topics and am still unable to get my XML log to parse properly.

Here is a sample of my XML file:

<?xml version="1.0" encoding="utf-8"?>
<Customers>
    <customer>
        <customerName>SampleCustomer1</customerName>
        <siteName>Main</siteName>
        <lastUpdated>8/10/2015 8:09:51 AM</lastUpdated>
        <alarmedState>False</alarmedState>
        <updateFrequency>45</updateFrequency>
        <pointsNotCommunicating>
            <pointNotCommunicating>SamplePoint1@2015-07-02 17:07:00</pointNotCommunicating>
            <totalNotCommunicating>1</totalNotCommunicating>
        </pointsNotCommunicating>
        <ClientIsUp>true</ClientIsUp>
    </customer>
    <customer>
        <customerName>SampleCustomer2</customerName>
        <siteName>Main</siteName>
        <lastUpdated>8/10/2015 8:27:57 AM</lastUpdated>
        <alarmedState>False</alarmedState>
        <updateFrequency>30</updateFrequency>
        <pointsNotCommunicating>
            <totalNotCommunicating>0</totalNotCommunicating>
        </pointsNotCommunicating>
        <ClientIsUp>true</ClientIsUp>
    </customer>

</Customers>

First: What I did was create a new file called props.conf in my splunk directory under .../etc/system/local because there wasn't a props.conf file there. Here is the entire contents of my props.conf file:

[source::C:\\Program Files (x86)\\SkySparkServer\\serverStatus.xml]
sourcetype: SkySparkLog

[SkySparkLog]
KV_MODE=xml
TIME_PREFIX = <lastUpdated>
TIME_FORMAT = %m\/%d\/%Y %I:%M:%S
SHOULD_LINEMERGE = true
MUST_NOT_BREAK_BEFORE = \</pointsNotCommunicating\>
MUST_BREAK_AFTER = \</customer\>
TRUNCATE = 0

Is this the proper method and if so, am I doing something wrong in my props.conf file?

Thanks for the help.

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Try this under your sourcetype definition in props.conf on Indexer/Heavy forwarder.

 [SkySparkLog]
KV_MODE=xml
TRUNCATE = 0
TIME_PREFIX = \<lastUpdated\>
SHOULD_LINEMERGE = true
TIME_FORMAT = %m/%d/%Y %I:%M:%S
LINE_BREAKER = (<customer>)
MUST_BREAK_AFTER = \</customer\>
SEDCMD-acrRemover = s/\x0D//g
SEDCMD-blfRemover = s/\x0A//g
SEDCMD-abremoveheader = s/\<\?xml.*\s*\<Customers\>\s*//g
SEDCMD-bremovefooter = s/\<\/Customers\>//
SEDCMD-caddroot = s/(.*)/<customer>\1/

View solution in original post

somesoni2
Revered Legend

Try this under your sourcetype definition in props.conf on Indexer/Heavy forwarder.

 [SkySparkLog]
KV_MODE=xml
TRUNCATE = 0
TIME_PREFIX = \<lastUpdated\>
SHOULD_LINEMERGE = true
TIME_FORMAT = %m/%d/%Y %I:%M:%S
LINE_BREAKER = (<customer>)
MUST_BREAK_AFTER = \</customer\>
SEDCMD-acrRemover = s/\x0D//g
SEDCMD-blfRemover = s/\x0A//g
SEDCMD-abremoveheader = s/\<\?xml.*\s*\<Customers\>\s*//g
SEDCMD-bremovefooter = s/\<\/Customers\>//
SEDCMD-caddroot = s/(.*)/<customer>\1/

jtashker
New Member

This works, thank you. Can you explain what the SEDCMD fields are doing?

0 Karma

somesoni2
Revered Legend

Its just removing the unwanted strings from you xml. Thexml header, the root Customers tag and newline and tabs, so youd get individual customer tag with proper xml syntax.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I believe you should change the following lines.

TIME_FORMAT = %m\/%d\/%Y %I:%M:%S %p
MUST_BREAK_AFTER = \</Customers\>
---
If this reply helps you, Karma would be appreciated.
0 Karma

jtashker
New Member

Rich, thank you for your reply. I will change TIME_FORMAT to match, but I don't think I want to break after /Customers.

I really want each customer to be one entry. Am I thinking about this the wrong way?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I see where breaking where you want makes sense.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...