topic Re: not able to extract fields using TRANSFORMS.conf in Splunk Search

not able to extract fields using TRANSFORMS.conf

rbsplunktest — Sun, 15 Nov 2015 12:15:57 GMT

Files at C:\Program Files\Splunk\etc\system\local

transforms.conf

[function_coverage]
REGEX =(fn).(name)(=)\".*?\".(fn_cov)(=)\"(.)\" 
FORMAT = xfncov::$1
WRITE_META = true
REPEAT_MATCH = true

props.conf

[customXML]
KV_MODE = xml
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
category = Structured
disabled = false
pulldown_type = true
BREAK_ONLY_BEFORE = src name="
MAX_EVENTS = 3000
xmlkv maxinputs=10000
TRANSFORMS-funcov = function_coverage

fields.conf

[xfncov]
INDEXED=true

Data in one of my events looks like:

.......
src name="ACRBase.hpp" mtime="1379335836" fn_cov="15" fn_total="19" cd_cov="9" cd_total="16" d_cov="9" d_total="16">
fn name="ACRPublic::C_SmartPtr<T>::addRef() const" fn_cov="1" fn_total="1" cd_cov="2" cd_total="2" d_cov="2" d_total="2">
probe line="46" column="4" kind="function" event="full"/>
probe line="48" kind="decision" event="full"/>
/fn>
fn name="ACRPublic::C_SmartPtr<T>::release() const" fn_cov="0" fn_total="1" cd_cov="2" cd_total="2" d_cov="2" d_total="2">
probe line="58" column="4" kind="function" event="full"/>
probe line="60" kind="decision" event="full"/>
/fn>
............
/src>
src...

I am trying to create index xfncov having fn name="ACRPublic::C_SmartPtr<T>::addRef() const" fn_cov="1"
This regex works in search using ...|regex _raw, but I am not getting any new index using the above transform. I am not able to debug where the problem is.

Can someone help?

Re: not able to extract fields using TRANSFORMS.conf

esix_splunk — Mon, 16 Nov 2015 03:42:31 GMT

Have you created the physical index named "xfncov" on your indexers? Once you do this, assign the index name to the inputs for your sourcetype.

Re: not able to extract fields using TRANSFORMS.conf

rbsplunktest — Mon, 16 Nov 2015 10:22:42 GMT

I am sorry for my language. I am editing the title.
"xfncov" will come into interesting fields automatically and will have values to which the REGEX will match isn't it?
I think this is the way of extracting the fields at the time of indexing. am I right?

Re: not able to extract fields using TRANSFORMS.conf

rbsplunktest — Wed, 18 Nov 2015 12:22:06 GMT

I have corrected my issue of field extraction by correcting my regex.

REGEX = (fn.name=/".*/".fn_cov=/"./")

But the problem is that the extracted field values have only the first match per event, thought there are multiple matches for the REGEX in an event and I want all the matches as values of extracted field.
what should I do for this??

Re: not able to extract fields using TRANSFORMS.conf

richgalloway — Wed, 18 Nov 2015 16:41:58 GMT

Add MV_ADD=1 to your transforms.conf file.

Re: not able to extract fields using TRANSFORMS.conf

rbsplunktest — Tue, 29 Sep 2020 07:55:57 GMT

Thanks a lot. Now multiple fields are getting extracted. But there is one issue.

In one event I have below two texts that should match with the REGEX. These are the last two entries in one event matching the REGEX. But the last one is not getting extracted. Do you see any reason for that?

fn name="ACRPublic::C_SmartPtr<T>::debugGetRefCount() const" fn_cov="1"
fn name="ACRPublic::C_SmartPtr<T>::debugGetRefCountAddr() const" fn_cov="0"

In an another event there is only one entry and it is getting extracted.

Re: not able to extract fields using TRANSFORMS.conf

rbsplunktest — Thu, 19 Nov 2015 04:24:09 GMT

It seems there is a cap on the max number of fields can be extracted per event. I tried increasing possible texts per event. In one event 18 out of 19 possible matches are extracted in another event 23 out of 25 and in one other 23 out of 30.

Re: not able to extract fields using TRANSFORMS.conf

richgalloway — Thu, 19 Nov 2015 12:41:02 GMT

In my experience, regex strings fail to match when the target is slightly different from what was expected. I'm not aware of any limit to the number of fields a regex can extract.

Re: not able to extract fields using TRANSFORMS.conf

rbsplunktest — Mon, 23 Nov 2015 07:06:26 GMT

Thanks Rich Galloway,

I have even tried by just changing the string with only one number i.e if first occurrence have function1 then I have copied function2, function 3 and so on but Field extraction only extract till a point, which is not fixed. So, REGEX should not be an issue.