Should we add metadata...?

[a212830]

Hi,

I read this blog, which was great. I noticed that the app being mentioned - https://splunkbase.splunk.com/app/2975/#/overview - creates a custom metadata field. In the past, I've avoided doing this, based upon Splunks recommendations. Is this still the case, or is it now ok to create our own metadata fields? I'd love to add them, but if it's going to slow down the system significantly, it's not worth the hassle. At the same time, the can provide a lot of additional meaningful information.

[ryanoconnor]

Splunk still recommends against creating index-time extractions unless absolutely necessary for two reasons:

1. Negative performance impact on your indexers
2. This type of extraction is less flexible. If you end up wanting to alter it it would require re-indexing all of your data.

http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Indextimeversussearchtime

In addition to this, I just dug through the app and the app does not appear to be using splunk_syslog_server in the overview dashboard inside the app. Have you tried just installing this and skipping that step and seeing if the app works?

[sloshburch]

Which is the blog entry that you're referring to?

[somesoni2]

The drawback of adding metdata field (at index time) is still valid (additional space of indexers as these will be stored in tsidx file as well and slower indexing performance).

[a212830]

How significant is this? I find it curious that this blog entry proposes adding metadata, but in the next breath it's not really something they recommend - especially for something like syslog, which is typically a large feed. Mixed signals...

[jkat54]

The "damage" done depends on the size of the FieldName and its value. Obviously reallyLongFieldNameasddgghkfkrjxnekznck=reallyLongValuedudjdjdjdjfjd. Would have more impact than a=1