topic Re: Should we add metadata...? in Splunk Search

Should we add metadata...?

a212830 — Fri, 06 May 2016 15:35:44 GMT

Hi,

I read this blog, which was great. I noticed that the app being mentioned - https://splunkbase.splunk.com/app/2975/#/overview - creates a custom metadata field. In the past, I've avoided doing this, based upon Splunks recommendations. Is this still the case, or is it now ok to create our own metadata fields? I'd love to add them, but if it's going to slow down the system significantly, it's not worth the hassle. At the same time, the can provide a lot of additional meaningful information.

Re: Should we add metadata...?

somesoni2 — Fri, 06 May 2016 20:48:45 GMT

The drawback of adding metdata field (at index time) is still valid (additional space of indexers as these will be stored in tsidx file as well and slower indexing performance).

Re: Should we add metadata...?

a212830 — Fri, 06 May 2016 22:05:13 GMT

How significant is this? I find it curious that this blog entry proposes adding metadata, but in the next breath it's not really something they recommend - especially for something like syslog, which is typically a large feed. Mixed signals...

Re: Should we add metadata...?

jkat54 — Sat, 07 May 2016 00:02:43 GMT

The "damage" done depends on the size of the FieldName and its value. Obviously reallyLongFieldNameasddgghkfkrjxnekznck=reallyLongValuedudjdjdjdjfjd. Would have more impact than a=1

Re: Should we add metadata...?

sloshburch — Mon, 13 Jun 2016 20:25:58 GMT

Which is the blog entry that you're referring to?

Re: Should we add metadata...?

ryanoconnor — Tue, 29 Sep 2020 09:57:32 GMT

Splunk still recommends against creating index-time extractions unless absolutely necessary for two reasons:

Negative performance impact on your indexers
This type of extraction is less flexible. If you end up wanting to alter it it would require re-indexing all of your data.

http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Indextimeversussearchtime

In addition to this, I just dug through the app and the app does not appear to be using splunk_syslog_server in the overview dashboard inside the app. Have you tried just installing this and skipping that step and seeing if the app works?