Deployment Architecture

DB Connect: How to index quoted string values from a database with a dbmon tail?

falkyre
Explorer

I have some data in a database that contains quoted values like the following:

REPT ALM ONT "ONT-1-1-16-3-11:MJ,LEVELLO,SA,7-26,11-58-9: \"1490 Optical signal level too low\""

When I run a dbquery, the value comes out with no issues. However, when using a tail on the database, the data gets indexed as a key value pair and drops everything after first " and shows up in Splunk as REPT ALM ONT \/ (should be a forward slash as it looks like it's trying to escape the first double quote).

Is there a setting that I can use to get all of the data indexed as the line shows up in the database so it shows up in Splunk? Can I set KV_MODE=auto in the props.conf or transforms.conf for dbx?

Thanks,

Tags (2)
0 Karma
1 Solution

pmdba
Builder

I would use a view to select your data instead of selecting it directly from the table. The view should use a "replace" function (or the equivalent for your particular database) to substitute some other character for the quotes, or to insert an escape that Splunk will recognize.

View solution in original post

pmdba
Builder

I would use a view to select your data instead of selecting it directly from the table. The view should use a "replace" function (or the equivalent for your particular database) to substitute some other character for the quotes, or to insert an escape that Splunk will recognize.

View solution in original post

falkyre
Explorer

Did some more digging and the data is being indexed properly in Splunk (ie I can see it in the Events view) but what's happening is it's getting escaped out when the default table view is done. I followed your suggestion and created a new extraction based on the data that is showing up (using the extract fields functionality which is much nicer in Splunk 6.2). Doing that got me to where I needed to be.

Thanks.

0 Karma

falkyre
Explorer

Unfortunately your suggestion won't work as I am using the tail functionality to index the database tables on the fly. If I do a dbquery, I get the values correctly but when I'm using the tail function, I am indexing the data with key-value pairs and that's when it "breaks". I want to know how to get the tail to index the field with all of the quotes in it. The actual key value pair is as follows:

SUMMARY=REPT ALM ONT "ONT-1-1-16-3-11:MJ,LEVELLO,SA,7-26,11-58-9: "1490 Optical signal level too low""

0 Karma

pmdba
Builder

A view works just as well as a table for dbtail. My point was that with a view you could insert whatever escape character(s) Splunk needs into your data and possibly preserve the quotes that way. Another possibility is to define a new key/term within Splunk using the "extract fields" functionality.

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.