Getting Data In

How to limit the scope of fields.conf stanzas to only apply configurations to data for certain indexes or sourcetypes?

gesman
Communicator

I wanted to auto convert data within these logs: field_a="value1|value2|value3", field_b="value_x|value_y" to multivalues at search time.
The perfectly working solution seems to be to add this to ./etc/system/local/fields.conf:

[field_a]
TOKENIZER = ([^\|]+)

[field_b]
TOKENIZER = ([^\|]+)

The only problem is that above approach seems to have global effect across all indexes and all sourcetypes.
Is there a way to limit the scope of stanzas within fields.conf?

I just want it to apply to specific index or to specific sourcetype.

0 Karma
1 Solution

woodcock
Esteemed Legend

Yes: do not put it in ./etc/system/local; put it in /.etc/apps/MyApp/default/fields.conf or /.etc/apps/MyApp/local/fields.conf where MyApp is where your inputs.conf file exists for this input.

View solution in original post

woodcock
Esteemed Legend

Yes: do not put it in ./etc/system/local; put it in /.etc/apps/MyApp/default/fields.conf or /.etc/apps/MyApp/local/fields.conf where MyApp is where your inputs.conf file exists for this input.

acharlieh
Influencer

It is true that good practice says to put configuration into apps for reasons of making it easier to distribute, and control scope at search time. That said fields.conf is a search time configuration. It has nothing to do with where inputs.conf resides (being an input time configuration file). Putting fields.conf in MyApp makes it such that it only applies to searches being run in the context of MyApp (assuming that these stanzas are not being exported to the system context). It doesn't limit the scope of these stanzas to a specific sourcetype or index.

woodcock
Esteemed Legend

That was a typo on my part; I meant fields.conf.

0 Karma

gesman
Communicator

I'd guess so too, but doc mentions only ./system/:
http://docs.splunk.com/Documentation/Splunk/6.2.3/Admin/fieldsconf

Is it documented anywhere regarding /apps/ as a possible location as well?

0 Karma

Arun_N_007
Communicator

Restriction can be done in App level, User level and System level. Where it differs in seach time and index time.

I dont thtink we can restrict field extraction to index level or sourcetype level.

It would be better you do this in search time using split
like
|eval field_a=split("value1|value2|value3","|")

0 Karma

acharlieh
Influencer

The Configuration File Precedence discusses how Splunk configuration files at different points of execution combine to make the configuration that's running on the system both during interactive and non-interactive times. (This goes into depth as to how configuration from /apps/ comes into play, and applies to all config files). System is just the easiest to understand.

Additionally as you get into editing configuration files across apps btool becomes insanely useful for figuring out how all this fits together.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...