Getting Data In

What do Splunk Ninjas think are the top three daily Splunk tasks in a large distributed environment?

brian1_tate
Path Finder

Hello all,

I am trying to build a workflow for our new Splunk product and want to know what top three regular daily tasks you may do in Splunk Enterprise. This includes anything in regards to ES administration as well and maintenance tasks.

If anyone has suggestions, I would certainly appreciate your feedback. This new environment has 5 indexers in a cluster, three search heads in a cluster and several heavy forwarders with a ton of data sent via forwarders. Comments anyone?

0 Karma

gjanders
SplunkTrust
SplunkTrust

So, most of what I do is set up alerts to advise me about potential issues, for ES this has included:

  1. Sources no longer sending data ( | tstats max(_indextime) where ... or similar is very useful here)
  2. Users exceeding there quota (disk quota, search quota), often this is harmless but sometimes I review the limits to see if they are appropriate.
  3. Check for badly written/long running searches via the monitoring console (this one is more a manual task for now).

There would be many other potential things on a daily basis, one thing I do try to do is review the error logs...

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/

brian1_tate
Path Finder

That's not relevant here. All I'm looking for is general procedural steps for those that use Splunk Enterprise with ES as a SIEM. I have those for ArcSight but after working with a larger Splunk environment, frankly the largest I have ever encountered - one would have them or develop them. So in this case, I am developing them based upon my own experiences and based upon those that others may use in day to day operations.

If you just say, I use Splunk - best wishes in justification of your career...

0 Karma

a212830
Champion

It's a joke. Lighten up Francis.

0 Karma

klaxdal
Contributor
0 Karma

Richfez
SplunkTrust
SplunkTrust

Y'all need to put more smileys in your posts to help us old geezers know you were joking. Geez. 🙂

Plus I am cleaning up a duplicate comment and am slightly rearranging the remaining, since they were responses to one another.

0 Karma

a212830
Champion

Answer questions on why shouldn't we use ELK/open-source...

0 Karma

aaraneta_splunk
Splunk Employee
Splunk Employee

@brian1_tate - When you say "ES" are you referring to the app "Splunk Enterprise Security"? Because that is usually what "ES" refers to. Please clarify, thank you!

0 Karma

brian1_tate
Path Finder

That's correct but in general, I wanted to get people's feedback on what daily/weekly tasks they perform as I am transitioning from ArcSight to Splunk.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!