topic Re: Remove default attribute in Getting Data In

Remove default attribute

nbowman — Tue, 29 Sep 2020 08:17:43 GMT

I have an environment where I want to use apps like Splunk for Nix, but have the logs go to different indexes.

Splunk_TA_nix/default/inputs.conf:

[monitor:///var/log]
whitelist=(\.log|log$|messages|secure|auth|mesg$|cron$|acpid$|\.out)
blacklist=(lastlog|anaconda\.syslog)
index=os
disabled = 1

I don't want the default inputs.conf to have index=os. I want to set the index in another app and be able to upgrade the app without messing with the default inputs.conf of Splunk for Nix each time. For example...

serverclass.conf:

[serverClass:TEST1]
whitelist.0 = 1.1.1.1
[serverClass:TEST1:app:TEST1-IndexConfig]

[serverClass:TEST2]
whitelist.0 = 2.2.2.2
[serverClass:TEST2:app:TEST2-IndexConfig]

TEST1-IndexConfig default inputs.conf

[default]
index=test1

TEST2-IndexConfig default inputs.conf

[default]
index=test2

Am I going to be stuck commenting out all the "index=" in the defaults each time I want to upgrade the app? Or can I specify in the local confs to ignore the default conf attribute and respect the [default] in my other app?

Re: Remove default attribute

richgalloway — Wed, 30 Dec 2015 17:41:17 GMT

Any changes you make to an app's configuration should be done in local for that very reason. Your settings in local override those shipped with the app in default.

Re: Remove default attribute

nbowman — Wed, 30 Dec 2015 17:44:38 GMT

I agree with this. However, I need to remove the index= attribute, not modify. And use another app to apply the [default] index=. Does that make sense?

Re: Remove default attribute

richgalloway — Wed, 30 Dec 2015 17:54:25 GMT

I doubt one app can change another app's settings. Instead of removing the index attribute can you just set it to the same value as in the other app?

Re: Remove default attribute

nbowman — Wed, 30 Dec 2015 18:06:23 GMT

I wish it were that easy. In my case, when a Universal Forwarder checks in to my deployment server, it gets an inputs.conf with it's [default] index=. All data from that client, unless otherwise specified, goes there.

That way, I can give login's to sysadmins who represent each "div" and they won't have unnecessary access to another division's data.

Re: Remove default attribute

jplumsdaine22 — Wed, 30 Dec 2015 18:22:47 GMT

I see your problem. Your best bet is to probably create a separate app for each division. Whenever there's an update you will have to unpack the updated tar to all your apps, but you can have a separate local config for each one then.

Alternatively on the indexer you could rewrite the index with a transforms.conf/props.conf combo based on the host name. Theres a few examples in Splunk answers already. (eg https://answers.splunk.com/answers/135315/different-index-based-on-hostname.html)

Also, you could set search filters for your users to prevent them from accessing data their not supposed to. Have a look at http://docs.splunk.com/Documentation/Splunk/6.3.1511/Security/Addandeditroles#Search_filter_format

Re: Remove default attribute

nbowman — Tue, 29 Sep 2020 08:17:45 GMT

I considered this route, however, I want to maintain the flexibility of sending data to specific indexes. For example, if I have a nix box that has the Splunk_TA_nix app sending to index=div02 for the sysadmins; I might also want to send other data to index=finance for the finance people.

I'm getting the feeling that this might be a case of wanting my cake and eating it too lol

Re: Remove default attribute

jplumsdaine22 — Thu, 31 Dec 2015 09:49:18 GMT

You want to send the same data to multiple indexes?

You know that costs double right?

Re: Remove default attribute

nbowman — Mon, 04 Jan 2016 15:15:27 GMT

It's not the same data. I want the default index to be set depending on the subnet that is used to check into the deployment server. That index holds the generic logs, like /var/log/secure and syslog, etc. And in one-off cases, like /var/log/finance.log would go to index=finance.

Re: Remove default attribute

jplumsdaine22 — Mon, 04 Jan 2016 15:34:14 GMT

I'm a bit confused. This is what I think you're saying.

You have two servers in different subnets.
You want the /var/log/secure to go to a different index based on the subnet the host is in
You do not have any way to distinguish their servers except their subnet (ie hostnames do not relate to status)
You may have other log files in /var/log on the servers that you wan to send to a different index

Is that correct?

Re: Remove default attribute

jplumsdaine22 — Tue, 05 Jan 2016 09:22:05 GMT

Also have a read of this: http://docs.splunk.com/Documentation/Splunk/6.3.2/Forwarding/Routeandfilterdatad

Re: Remove default attribute

nbowman — Tue, 05 Jan 2016 16:20:16 GMT

Yes, that is an accurate summary.

Also, I have looked at Splunk's routing and filtering of data that you linked. It doesn't solve my problem because very often in my environment, sysadmins will install a Splunk Universal Forwarder on their systems but won't allow me access to them for purposes of configuration. So, I can't control hostnames. All I can do is point them to my deployment server. Filtering on sourcetype doesn't work either, because the sourcetypes will be the same.