Getting Data In

How do you manage inputs.conf?

alanzchan
Path Finder

We have more than 100 applications in our deployment. Sometimes a log path/name is changed or added on the server side and I have to update inputs.conf to reflect the new change. Over the years, each change was just appended in the inputs.conf and logs that we once monitored but no longer do are now clogging up the inputs.conf.

Of course, I can log into one of the servers in each serverclass and see which logs we are actually monitoring, but as you can imagine that would be a hefty job. Is there a smarter way to clean up inputs.conf? Any suggestions on how to manage it?

0 Karma

gcusello
SplunkTrust
SplunkTrust

alanzchan,
I created a TA for each app input.
Then I created an App on my Search Head that reads all the inputs.conf on the Deployment Server (there's a TA also on Deployment Server) and then shows all the inputs in a dedicated dashboard.
In this way I can search if an input is already configured.
Bye.
Giuseppe

0 Karma

diogofgm
SplunkTrust
SplunkTrust

you can try to use REST to get that info. Try this search

| rest splunk_server=local /services/admin/monitor
| fields title index sourcetype eai:acl.app host
| join sourcetype [| metadata type=sourcetypes index=_* index=* | eval diff = tostring(now() - recentTime, "duration") | fields - type]
| convert ctime(recentTime)

This targets the local server (you can change that to target other server) to return all monitor inputs, joins it with info from metadata which lists the source types and the last time you got data from each one. This can be modified to do the join by source.

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

solarboyz1
Builder

We create a app for each serverClass, using a standard naming convention we created:

Acme_CHI_hfw_inputs/local/inputs.conf

All inputs are maintained in this server class.
We do the same for indexes, and well as baseline settings:

Acme_CHI_idx_indexes/local/indexes.conf
Acme_ALL_idx_base/local/limits.conf, distsearch.conf, etc..

By using a standardized naming convention, we know what settings are applied to which servers.
By seperating these settings from the original Add-On, we have one location to update.

We also use the ConfigQuest app to report on settings deployed, and serverClass members.

Mileage may vary.

0 Karma

alanzchan
Path Finder

We also have a central configuration for each application. Since we have been just appending log inputs into the configuration file, I'm more interested in how I can find out which input stanza in inputs.conf is actually being used.

0 Karma

solarboyz1
Builder

So, you can use your metrics logs to see the per-source thruput

index=_*  component=Metrics group=per_source_thruput

And match that to your inputs/serverClass?

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...