Default Log format for splunk

[mclane41]

I see some post about rules for splunk logs.

But I don't find a list of rules. My applications logs a  lot of lines for splunk (100GB/day) and we prefere use the default integration in splunk (without transformation, extraction...) in order to save time during indexing.

I propose to my developeurs to logs with these constraints.

Where can I find all constraints, or the better constraints ...

Please log like that :

[%m-%d-%Y %H:%M:%S.%Q]key1=value1,key2=value2,...

keys : not begin with number or '_'

values : no spaces or commas else between quote

[PickleRick]

Typically most of extraction is taking place in search time so the most important thing about field extraction is that the format is consistent and can be easily configured (so you don't have cases like escaped characters).

From indexing performance point of view it's most important that the format is consistent across the whole sourcetype, the data breaks easily into separate events and that the timestamp is well-defined and hopefully placed at the beginning of the event.

If you have all this and your sourcetype has the so-called great eight properly configured, you're good to go.

From the practical point of view regarding parsing the data - avoid any nesting - like "real" data as somehow-formatted string within a json structure or the other way around - json structure with a syslog header, any escaped strings within strings and so on - it makes writing extractions and searches a painful experience.

[ITWhisperer]

Your constraints look reasonable. You appear to have an easy-to-find timestamp, which presumably will help split your log into separate events, and your field definition appears robust. I suggest you go with what you have.