Getting Data In

How do I configure custom index-time field extraction?

GolemXIV
New Member

Hello,

i want to extract a field on index-time extraction on search head (i know it's not the best idea), but I'm have some strange issues with it.
A new field should be indexed through collect command to summary indexes, but when i collect data this way, i don't see this field extraction. When i use | extract some-number-idx , the fields are visible in verbose mode, so it looks like field extraction is ok. I try to restart Splunk, move confs from app to etc/system/local, but all is useless. What could be the problem? Maybe there are some logs in Splunk to debug this process?

My confs in custom app:

#transforms.conf
[some-number-idx]
REGEX = myfield=(<number_idx>\d+)
FORMAT = number_idx::$1
WRITE_META = true

#props.conf
[stash]
TRANSFORMS-myfield = some-number-idx

#fields.conf
[number_idx]
INDEXED = true
0 Karma

woodcock
Esteemed Legend

You should back ALL the way up. You definitely should not put in any settings for sourcetype statsh, especially index-time. What exactly is your original problem?

0 Karma

GolemXIV
New Member

Thanks for the answer. I make slices (~ 200 million) using | collect in composite indexes, which are verified with an external source for this numeric field. Ideally, I want to index it as a timestamp in order to run in a range of several jobs across this field.
As I understand it, when setting the sourcetype to collect, the license is wasted, so keep stash is the default sourcetype ...

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Introducing the 2026 - 2027 SplunkTrust cohort!

The goal of the SplunkTrust™ membership has historically been to acknowledge and recognize those who go above ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...