topic Re: How do I configure custom index-time field extraction? in Getting Data In

How do I configure custom index-time field extraction?

GolemXIV — Mon, 08 Oct 2018 07:02:11 GMT

Hello,

i want to extract a field on index-time extraction on search head (i know it's not the best idea), but I'm have some strange issues with it.
A new field should be indexed through collect command to summary indexes, but when i collect data this way, i don't see this field extraction. When i use | extract some-number-idx , the fields are visible in verbose mode, so it looks like field extraction is ok. I try to restart Splunk, move confs from app to etc/system/local, but all is useless. What could be the problem? Maybe there are some logs in Splunk to debug this process?

My confs in custom app:

#transforms.conf
[some-number-idx]
REGEX = myfield=(<number_idx>\d+)
FORMAT = number_idx::$1
WRITE_META = true

#props.conf
[stash]
TRANSFORMS-myfield = some-number-idx

#fields.conf
[number_idx]
INDEXED = true

Re: How do I configure custom index-time field extraction?

woodcock — Mon, 08 Oct 2018 23:39:32 GMT

You should back ALL the way up. You definitely should not put in any settings for sourcetype statsh, especially index-time. What exactly is your original problem?

Re: How do I configure custom index-time field extraction?

GolemXIV — Tue, 09 Oct 2018 06:06:18 GMT

Thanks for the answer. I make slices (~ 200 million) using | collect in composite indexes, which are verified with an external source for this numeric field. Ideally, I want to index it as a timestamp in order to run in a range of several jobs across this field.
As I understand it, when setting the sourcetype to collect, the license is wasted, so keep stash is the default sourcetype ...