What are some of the best practices for field extractions?

[a212830]

Hi,

There is some debate in our group regarding best practices for field extractions. We have a feed that has well defined key-value fields. We also have field extractions setup on the SH, for a number of these fields. Is there a really a need for the field extractions, since key-value pairs will get picked up automatically? Pros/cons? We use CIM/ES extensively.

[sloshburch]

It depends on the sourcetype definition. If the sourcetype is handling the extractions natively then you are slowing things down by adding more search time extractions.

Sounds like we might need an example...

[a212830]

It's pretty straight-forward:

field1=value1 field2=value2 field3=value3

The sourcetype is configured with KVMODE=auto. We also have an app on the search-head, which also does extractions against this sourcetype, using transforms. IMO, the app isn't needed, unless there's some need for it with CIM/ES, which I'm really just getting familiar with.

[sloshburch]

Yea...def straightforward. My guess is your hunch: a case of over engineering. While they may have considered it benign, it def would produce redundant processing and marginally slow down the Search Head processing.

If I were you, here's what I would do to validate:

• Export a sample of the data to my local sandbox and index it there showing that the name/value pairs are available out of the box
• Comment out the related reference to the transforms that exists in props (but not the transforms itself as it might be used by other sourcetypes) and see if it still works
• Change KV_MODE from auto to auto_escaped. See props.conf.spec

To be safe, you might as well share what the transforms is. You mentioned KV_MODE which is props. But let's be sure about what the purpose of the related transforms is.

Also, it could be the case that in your environment, someone erroneously edited the default KV_MODE thereby obligating any sourcetype to need such over-the-top extra config.