How to configure Splunk DB Connect to index a SQL table with XML so each event is one row of data in the database, not per line of XML?

[ezajac]

I've got a SQL Table that contains XML Blobs that I want to index in Splunk. The XML Blob contains carriage returns. When indexing, Splunk is creating a new row for each line in the XML Blob. I'm sure this can be worked around by modifying one of the .conf files in the Splunk DB Connect App. I would like one entry in Splunk for each row of data in the database (not one entry per line of XML in all the rows in the database).

Desired:
Entry 1:
"date id=123 xml_results=contentn body /body n /content "

Entry 2:
"date id=124 xml_results= content n body /body n /content "

What is happening now:
Entry 1:
"date id=123 xml_results= content "

Entry 2:
body /body

Entry 3:
/content

Entry 4:
"date id=124 xml= content "

[jcoates_splunk]

Can the dbquery command show the output properly? If so, you should continue trying to solve this in Splunk... but if not, I would look for an SQL way to do it, because your JDBC driver is probably the problem and we're not going to be able to get around that. This looks like a good starting point: https://www.simple-talk.com/sql/database-administration/converting-string-data-to-xml-and-xml-to-str...

[maciep]

Are you specifying a custom sourcetype in your database input? And what format are you using? key-value, multi-line key-value, csv, etc?

[ezajac]

I am creating a custom sourcetype and using key-value.

[maciep]

would you be willing to try using the default sourcetype that dbconnect would use for that format? And if that doesn't work, also try the multi-line kv format? You may also want to put dbconnect on your indexer if it isn't already. If either of those work and you still want to use a custom sourcetype, that should be doable

There are configs for those default sourcetypes in props.conf for dbconnect that tell splunk how to break up the events. If you're using your own sourcetype, it's probably just guessing. Once you find what works, then you can copy their settings for that sourcetype and drop it into your own props for you custom sourcetype.

I hope that makes sense...and is somewhat correct. Let us know how it goes.