Getting Data In

Logging for Splunk - Best Practices or Tips?

shaneharter
New Member

Just getting started with Splunk. I'm looking to get better instrumentation and visibility into our systems. In some cases this could be for debugging, in other cases for audit-trail type uses.

Since I'm going to (re)write all of our logging code in these systems, does anybody have any suggestions?

Certainly one thing I'm looking to accomplish is "joining" events across log files as they flow through our system. Aside from that, my use cases seem pretty basic. But since i'm in this position where I can design my logs just for splunk I thought people who've been doing this for a while could have some tips that might make my life easier later on.

Thanks!
Shane

0 Karma
1 Solution

ziegfried
Influencer

This is a pretty good starting point: http://dev.splunk.com/view/logging-with-splunk/SP-CAAADP5

What I can add from my experience is that printing the following things in every event is really valuable:

  • The severity of the event (ie. DEBUG/INFO/WARN/ERROR)
  • A thread identifier (Some sequential or random string that identifies the current thread)
  • The source of the log event (ie. class, function or filename)

View solution in original post

ziegfried
Influencer

This is a pretty good starting point: http://dev.splunk.com/view/logging-with-splunk/SP-CAAADP5

What I can add from my experience is that printing the following things in every event is really valuable:

  • The severity of the event (ie. DEBUG/INFO/WARN/ERROR)
  • A thread identifier (Some sequential or random string that identifies the current thread)
  • The source of the log event (ie. class, function or filename)

Drainy
Champion

The identifier is a great point but missed by some and when combined with transaction you have a great tool for troubleshooting and debugging process/thread/event specific execution

0 Karma

mtevenan_splunk
Splunk Employee
Splunk Employee

The logging content referred to above is now here: http://dev.splunk.com/view/logging/SP-CAAAFCK

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...