I am writing guidance for developers to use when writing logs for internal applications. We log to syslog and push to splunk. One field I would like to include in our logs involves dynamically generated sql queries. An example field is:
query="SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5"
The thought was that by double quoting the entire value I would get a key-value pair of:
key = query
value = SELECT t1.field1, t2.field2, t2.field3 FROM table1 t1 JOIN table2 t2 ON t1.foreign_key = t2.key WHERE t1.field1 = 5
While it DOES extract this key-value pair, it also auto-extracts:
key = t1_foreign_key
value = t2.key
and
key=WHERE_t1_field1
value=5
Is there a way to make it ignore the "=" sign inside of the values? I had assumed that placing the whole string inside quotes would "protect" it, but it seems that was a bad assumption. Are there other characters that I need to watch out for (most of the queries are much more complex than this example)?
Thank You
For clarification, the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"
Additionally, while my example involves a sql query, the question is broadly "How should I format field values that contain the = character, such that Splunk won't try to auto extract from the middle of the value?"
in sql we have a |trim() That can help you,
http://www.sqlite.org/lang_corefunc.html.
Hope it can help you.
Unfortunately, I don't think this will help. The question is about the structuring of custom logs, and is applicable to any field value. The example that I gave is specific to logging a sql query, but really I just want to know about ANY value where an "=" character is present.
Just to be clear the question is about "best practice" for formatting log data BEFORE it even reaches splunk. I.e., How should I structure my log messages to make it "splunk friendly"