Logging Best Practises, non key=value pair

[pjhawar]

We generally follow a pattern of logging in a key=value pattern.

I am curious if we should totally avoid logs that are not in that format. Is it not recommended to have logs like:

log.info("Flushing kafka buffer before callback.");

[PickleRick]

As with almost every such general question - it depends. Generally, you want to have a standardized and consistent log format so that you can easily extract field values.

It might also be a bit easier to remember that you need to search for something like "module=callback action=flush" instead of the particular wording of this exact message. But if you need to review the logs manually however, you might want to have a verbose log because it's easier human-readable.

The best of both worlds would be to have both - structured form and a free-form message but that increases storage usage (and in Splunk's case license consumption)

[isoutamo]

Hi

when I am writing logging guidelines/policies I always recommend to use standard format at 1st part of log. Then you should have some system specific part which also should standardize as much as possible inside same systems. That way it’s much easier and even possible to create continuous audit trail over customer transactions/sessions. That is much more important than are there some known kv pairs written in log events. Of course if/when you are using log files by dashboards, reports and alerts you could/should decrease the space of events as much as possible to save license costs. But if there are many people who are not familiar with your logs then it’s much easier that those contains also keywords to help to understand those.

r. Ismo

[ITWhisperer]

If the logged events follow a defined pattern, not necessarily key=value, they can be ingested and use regular expressions (regex) to extract the data into fields. For example:

log.info\(\"(?<loginfo>[^\"]+)\"\)\;