Splunk Search

Evaluating if splunk is for me.

Path Finder

Hi Everyone,

I'm trying to find a log solution and here is what I would like to achieve.

  • I have 50 systems with weekly messages aggregation of under 500MB a week.
  • I also have jboss applications running on the same 50 nodes that I'd like to capture their error.log's (but not server.log).
  • I also want to filter what actually gets sent to splunk as I'm only interested in the first line of the stacktraces.

I can filter these out using egrep for a date format - which brings my 100M log down to 4M. Does splunk have any capability to do filtering before it actually brings something in to index? Sometimes our logs can get out of control and I can write 2-5GB of error.logs within a couple hours - most of which I'm not interested in, and wouldn't want in splunk, which would cause me to go over the 500MB free threshold.

Anyone have any thoughts? How do other people handle similar types of problems?

[dmurphy@jboss11 ~]$ egrep [0-9]{4}-[0-9]{2}-[0-9]{2} error.log.1  
2012-01-22 13:02:36,548 [http-] ERROR [org.apache.commons.beanutils.PropertyUtils] Method invocation failed.
2012-01-22 13:04:08,114 [http-] ERROR [org.apache.commons.beanutils.PropertyUtils] Method invocation failed.
Tags (1)


The short answer is YES. Everything that you are looking to do can be done with splunk. I won't go into the details because you are better off reading the documentation and playing with splunk yourself, but it's not hard at all to configure splunk for your requirements. My recommendation is to download splunk, and go through the tutorials available in the documentation. Then read the sections that deal with installing and administration of splunk. And of course, once you have more detailed questions, with regards to configuration, ask them here.

0 Karma


For what it's worth, I'd also think about doing a support contract for a short while. Then you get some expert help when something particularly tricky shows up.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...