All Apps and Add-ons

Formatting log fields containing = in the values

dharkness9549
New Member

I am writing guidance for developers to use when writing logs for internal applications. We log to syslog and push to splunk. One field I would like to include in our logs involves dynamically generated sql queries. An example field is:

query="SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5"

The thought was that by double quoting the entire value I would get a key-value pair of:

key = query
value = SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5

While it DOES extract this key-value pair, it also auto-extracts:

key = t1_foreign_key
value = t2.key

and

key=WHERE_t1_field1
value=5

Is there a way to make it ignore the "=" sign inside of the values? I had assumed that placing the whole string inside quotes would "protect" it, but it seems that was a bad assumption. Are there other characters that I need to watch out for (most of the queries are much more complex than this example)?

Thank You

0 Karma

dharkness9549
New Member

For clarification, the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

Additionally, while my example involves a sql query, the question is broadly "How should I format field values that contain the = character, such that Splunk won't try to auto extract from the middle of the value?"

0 Karma

neelamssantosh
Contributor

in sql we have a |trim() That can help you,
http://www.sqlite.org/lang_corefunc.html.
Hope it can help you.

0 Karma

dharkness9549
New Member

Unfortunately, I don't think this will help. The question is about the structuring of custom logs, and is applicable to any field value. The example that I gave is specific to logging a sql query, but really I just want to know about ANY value where an "=" character is present.

Just to be clear the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...