Getting Data In

How do I configure custom index-time field extraction?

GolemXIV
New Member

Hello,

i want to extract a field on index-time extraction on search head (i know it's not the best idea), but I'm have some strange issues with it.
A new field should be indexed through collect command to summary indexes, but when i collect data this way, i don't see this field extraction. When i use | extract some-number-idx , the fields are visible in verbose mode, so it looks like field extraction is ok. I try to restart Splunk, move confs from app to etc/system/local, but all is useless. What could be the problem? Maybe there are some logs in Splunk to debug this process?

My confs in custom app:

#transforms.conf
[some-number-idx]
REGEX = myfield=(<number_idx>\d+)
FORMAT = number_idx::$1
WRITE_META = true

#props.conf
[stash]
TRANSFORMS-myfield = some-number-idx

#fields.conf
[number_idx]
INDEXED = true
0 Karma

woodcock
Esteemed Legend

You should back ALL the way up. You definitely should not put in any settings for sourcetype statsh, especially index-time. What exactly is your original problem?

0 Karma

GolemXIV
New Member

Thanks for the answer. I make slices (~ 200 million) using | collect in composite indexes, which are verified with an external source for this numeric field. Ideally, I want to index it as a timestamp in order to run in a range of several jobs across this field.
As I understand it, when setting the sourcetype to collect, the license is wasted, so keep stash is the default sourcetype ...

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Catalog Is Now Generally Available on Splunk Cloud Platform

A Unified View of Your Data  Security logs, application events, business data, and historical telemetry often ...

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner   Watch on demand us for a demo-driven ...