Getting Data In

How do I configure custom index-time field extraction?

GolemXIV
New Member

Hello,

i want to extract a field on index-time extraction on search head (i know it's not the best idea), but I'm have some strange issues with it.
A new field should be indexed through collect command to summary indexes, but when i collect data this way, i don't see this field extraction. When i use | extract some-number-idx , the fields are visible in verbose mode, so it looks like field extraction is ok. I try to restart Splunk, move confs from app to etc/system/local, but all is useless. What could be the problem? Maybe there are some logs in Splunk to debug this process?

My confs in custom app:

#transforms.conf
[some-number-idx]
REGEX = myfield=(<number_idx>\d+)
FORMAT = number_idx::$1
WRITE_META = true

#props.conf
[stash]
TRANSFORMS-myfield = some-number-idx

#fields.conf
[number_idx]
INDEXED = true
0 Karma

woodcock
Esteemed Legend

You should back ALL the way up. You definitely should not put in any settings for sourcetype statsh, especially index-time. What exactly is your original problem?

0 Karma

GolemXIV
New Member

Thanks for the answer. I make slices (~ 200 million) using | collect in composite indexes, which are verified with an external source for this numeric field. Ideally, I want to index it as a timestamp in order to run in a range of several jobs across this field.
As I understand it, when setting the sourcetype to collect, the license is wasted, so keep stash is the default sourcetype ...

0 Karma
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...