topic Parsing XML and props.conf help in Getting Data In

Parsing XML and props.conf help

jtashker — Mon, 10 Aug 2015 14:48:57 GMT

Let me first preface this by saying that I am a total Splunk newbie and this is very similar to a lot of XML parsing questions, however I have read through ~20 topics and am still unable to get my XML log to parse properly.

Here is a sample of my XML file:

<?xml version="1.0" encoding="utf-8"?>
<Customers>
    <customer>
        <customerName>SampleCustomer1</customerName>
        <siteName>Main</siteName>
        <lastUpdated>8/10/2015 8:09:51 AM</lastUpdated>
        <alarmedState>False</alarmedState>
        <updateFrequency>45</updateFrequency>
        <pointsNotCommunicating>
            <pointNotCommunicating>SamplePoint1@2015-07-02 17:07:00</pointNotCommunicating>
            <totalNotCommunicating>1</totalNotCommunicating>
        </pointsNotCommunicating>
        <ClientIsUp>true</ClientIsUp>
    </customer>
    <customer>
        <customerName>SampleCustomer2</customerName>
        <siteName>Main</siteName>
        <lastUpdated>8/10/2015 8:27:57 AM</lastUpdated>
        <alarmedState>False</alarmedState>
        <updateFrequency>30</updateFrequency>
        <pointsNotCommunicating>
            <totalNotCommunicating>0</totalNotCommunicating>
        </pointsNotCommunicating>
        <ClientIsUp>true</ClientIsUp>
    </customer>

</Customers>

First: What I did was create a new file called props.conf in my splunk directory under .../etc/system/local because there wasn't a props.conf file there. Here is the entire contents of my props.conf file:

[source::C:\\Program Files (x86)\\SkySparkServer\\serverStatus.xml]
sourcetype: SkySparkLog

[SkySparkLog]
KV_MODE=xml
TIME_PREFIX = <lastUpdated>
TIME_FORMAT = %m\/%d\/%Y %I:%M:%S
SHOULD_LINEMERGE = true
MUST_NOT_BREAK_BEFORE = \</pointsNotCommunicating\>
MUST_BREAK_AFTER = \</customer\>
TRUNCATE = 0

Is this the proper method and if so, am I doing something wrong in my props.conf file?

Thanks for the help.

Re: Parsing XML and props.conf help

richgalloway — Mon, 10 Aug 2015 15:55:09 GMT

I believe you should change the following lines.

TIME_FORMAT = %m\/%d\/%Y %I:%M:%S %p
MUST_BREAK_AFTER = \</Customers\>

Re: Parsing XML and props.conf help

jtashker — Mon, 10 Aug 2015 16:16:24 GMT

Rich, thank you for your reply. I will change TIME_FORMAT to match, but I don't think I want to break after /Customers.

I really want each customer to be one entry. Am I thinking about this the wrong way?

Re: Parsing XML and props.conf help

richgalloway — Mon, 10 Aug 2015 16:50:40 GMT

I see where breaking where you want makes sense.

Re: Parsing XML and props.conf help

somesoni2 — Mon, 10 Aug 2015 17:08:21 GMT

Try this under your sourcetype definition in props.conf on Indexer/Heavy forwarder.

 [SkySparkLog]
KV_MODE=xml
TRUNCATE = 0
TIME_PREFIX = \<lastUpdated\>
SHOULD_LINEMERGE = true
TIME_FORMAT = %m/%d/%Y %I:%M:%S
LINE_BREAKER = (<customer>)
MUST_BREAK_AFTER = \</customer\>
SEDCMD-acrRemover = s/\x0D//g
SEDCMD-blfRemover = s/\x0A//g
SEDCMD-abremoveheader = s/\<\?xml.*\s*\<Customers\>\s*//g
SEDCMD-bremovefooter = s/\<\/Customers\>//
SEDCMD-caddroot = s/(.*)/<customer>\1/

Re: Parsing XML and props.conf help

jtashker — Mon, 10 Aug 2015 17:25:35 GMT

This works, thank you. Can you explain what the SEDCMD fields are doing?

Re: Parsing XML and props.conf help

somesoni2 — Mon, 10 Aug 2015 17:45:57 GMT

Its just removing the unwanted strings from you xml. Thexml header, the root Customers tag and newline and tabs, so youd get individual customer tag with proper xml syntax.