Getting Data In

Create calculated field during the indexing

rayar
Contributor

We are considering to calculate specific filed (list)  during the indexing 

the calculation will be -   | eval list=if(match(dhost,"\.[\w]{2,3}\.[\w]{2}:?[\d]?"),"mozilla","iana")

1. What is the performance impact  ?

2. how it should be done ?

Labels (1)
0 Karma

somesoni2
Revered Legend

As far as I know, there is no calculated indexed-time field extraction possible in Splunk. All indexed time field extractions are either static OR using regex on _raw OR metadata fields (host/source/sourcetype/_time) etc. 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rayar,

the choose to use indexed fields (or not) depends on the volume of indexed data and searches:

  • if you have a large volume of logs to index, it isn't a good ideas because you overload your indexers and it's better to perform this extraction at search time when you already filtered yourdata in a search,
  • if you have many searches using those fields, it could be useful.

In other words, you have to decide if anticipate a work at index time or to use it at search time.

If you have intermediate Heavy Forwarders, they could be used to extract fields without overloading Indexers.

About the way to do this, you can see at https://docs.splunk.com/Documentation/Splunk/8.2.5/Data/Configureindex-timefieldextraction

I usually don't extract fields at index time!

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Enhance Your Splunk App Development: New Tools & Support

UCC FrameworkAdd-on Builder has been around for quite some time. It helps build Splunk apps faster, but it ...

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...