Getting Data In

How do I configure custom index-time field extraction?

GolemXIV
New Member

Hello,

i want to extract a field on index-time extraction on search head (i know it's not the best idea), but I'm have some strange issues with it.
A new field should be indexed through collect command to summary indexes, but when i collect data this way, i don't see this field extraction. When i use | extract some-number-idx , the fields are visible in verbose mode, so it looks like field extraction is ok. I try to restart Splunk, move confs from app to etc/system/local, but all is useless. What could be the problem? Maybe there are some logs in Splunk to debug this process?

My confs in custom app:

#transforms.conf
[some-number-idx]
REGEX = myfield=(<number_idx>\d+)
FORMAT = number_idx::$1
WRITE_META = true

#props.conf
[stash]
TRANSFORMS-myfield = some-number-idx

#fields.conf
[number_idx]
INDEXED = true
0 Karma

woodcock
Esteemed Legend

You should back ALL the way up. You definitely should not put in any settings for sourcetype statsh, especially index-time. What exactly is your original problem?

0 Karma

GolemXIV
New Member

Thanks for the answer. I make slices (~ 200 million) using | collect in composite indexes, which are verified with an external source for this numeric field. Ideally, I want to index it as a timestamp in order to run in a range of several jobs across this field.
As I understand it, when setting the sourcetype to collect, the license is wasted, so keep stash is the default sourcetype ...

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...