Splunk Search

multivalue field search time extraction through UI

nawazns5038
Builder

Hi,

I would like to extract field values from UI using the field transformations and field extractions from settings.

I have added the field extractions and referenced it to use a field transform. I follow the same naming conventions for other normal field extractions with transforms and it works well.

  props.conf
 [sourcetype]
 REPORT-IP = REPORT-IP

transforms.conf 
[REPORT-IP]
FORMAT = IP::$1
MV_ADD = 1
REGEX = c=IN IP4 (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
SOURCE_KEY = list

I have verified the regex and it captures well when checked in general, however I don't see the required multi value field in Splunk with the values extracted.

Is anything else missing ?

Thanks

0 Karma

nawazns5038
Builder

I figured out the issue here.

The SOURCE_KEY = list , the field list here is an already search time extracted field. And I am trying to apply one more search time field extraction or a search time multi value field extraction to it and so it could not work.

If I change the SOURCE_KEY = _raw , it works perfect but does not solve my case. As I want the extraction particular only to that field.

I did not change any other parameters in my settings shown in the question except for the source_key.

0 Karma

to4kawa
SplunkTrust
SplunkTrust

try REGEX options.

props.conf

[yourSourcetype]
REPORT-IP = IP

transforms.conf

 [IP]
 FORMAT = IP::$1
 MV_ADD = 1
 REGEX = (?ms)c=IN\sIP4\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
 SOURCE_KEY = list

Sample:

| makeresults
| eval list="zsdfgdsgfgsdfg dfgkh sdfl;askgh a;sdfkghjak gah fkgakfgakgj ha;sdghkjasgh as;kgkahsjkghao[YOWERT    [Y     [OYTUWEH ASDGH IQHIWURHTGIWQUHO    HEOW    ROIouoasdfoutyet uertyuqewr tyouyqwrtuootptypowreqyturwet wqrtu8 sdfgsdfg sdfg w['aE QWE[T QWETJHTJKERHWTERHWKTH EJRTHKJEWRHTUII    auioprtywerryweuortyew c=IN IP4 10.33.22.11ghghfjgfhj ghjgjedkj wrktjyhoe oqw4u5yohwueirthywi hqiuehrtqtp[[qh erhtjqehrtueqhrtuiqwyhtkjewhgiuhdefjkaslhguqirotuyqpuyt quywertuipyp    y] oiq    ytuoqerytu htuqerhuthuqeirhtuy4238[    ]2u5]r    iuwifhdjgfhjrhgtu
 =-+c=IN IP4 10.44.33.22+t334543 34q535asdtesdft"
 | rex field=list max_match=0 "(?ms)c=IN\sIP4\s(?<IP>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
0 Karma

nawazns5038
Builder

Thanks for the regex.

I figured out the issue here.

The SOURCE_KEY = list , the field list here is an already search time extracted field. And I am trying to apply one more search time field extraction or a search time multi value field extraction to it and so it could not work.

If I change the SOURCE_KEY = _raw , it works perfect but does not solve my case. As I want the extraction particular only to that field.

0 Karma

to4kawa
SplunkTrust
SplunkTrust

So there's a problem with the other settings.

0 Karma

michael_bates_1
Path Finder

Please pay attention to the "order of operations" posted earlier.
Fields from a previous step are available in subsequent steps except for calculated fields (they run in parallel), so,

If you configure either an inline field extraction, or an additional Transforms that is executed first, then you would be able to use the field.

E.g
props.conf

[sourcetype]
EXTRACT-list = regex_to_extract_the_list
REPORT-ip = ip_from_list

transforms.conf
[ip_from_list]
MV_ADD = 1
SOURCE_KEY = list
REGEX = c=IN\sIP4\s(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})
FORMAT = IP::$1

0 Karma

michael_bates_1
Path Finder

Hi
The most likely suspect is the field "list" does not exist when the transform attempts to execute.
Search time order of operations are

  1. Inline extraction from props
  2. Transform-based extraction
  3. Auto KV fields
  4. Field-Alias
  5. Calc fields
  6. Lookups
  7. Eventtypes
  8. Tags

Can you please provide some insight into how/when the field "list" used as the SOURCE_KEY is extracted.

Thanks.

nawazns5038
Builder

So you mean we cannot perform search time extractions again on already search time extracted fields. ??

0 Karma

nawazns5038
Builder

I figured out the issue here.

The SOURCE_KEY = list , the field list here is an already search time extracted field. And I am trying to apply one more search time field extraction or a search time multi value field extraction to it and so it could not work.

If I change the SOURCE_KEY = _raw , it works perfect but does not solve my case. As I want the extraction particular only to that field.

I did not change any other parameters in my settings shown in the question except for the source_key.

0 Karma

michael_bates_1
Path Finder

Hi, Are you able to provide a sample of the data you are attempting to extract from.

0 Karma

nawazns5038
Builder
zsdfgdsgfgsdfg dfgkh sdfl;askgh a;sdfkghjak gah fkgakfgakgj ha;sdghkjasgh as;kgkahsjkghao[YOWERT    [Y  [OYTUWEH ASDGH IQHIWURHTGIWQUHO HEOW    ROIouoasdfoutyet uertyuqewr tyouyqwrtuootptypowreqyturwet wqrtu8 sdfgsdfg sdfg w['aE QWE[T QWETJHTJKERHWTERHWKTH EJRTHKJEWRHTUII    auioprtywerryweuortyew c=IN IP4 10.33.22.11ghghfjgfhj ghjgjedkj wrktjyhoe oqw4u5yohwueirthywi hqiuehrtqtp[[qh erhtjqehrtueqhrtuiqwyhtkjewhgiuhdefjkaslhguqirotuyqpuyt quywertuipyp  y] oiq  ytuoqerytu htuqerhuthuqeirhtuy4238[ ]2u5]r  iuwifhdjgfhjrhgtu
=-+c=IN IP4 10.44.33.22+t334543 34q535asdtesdft
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

These two are always different values. I don't know what the effect of this would be. The second one should match the stanza in transforms.conf

 [sourcetype]
 REPORT-IP = REPORT-stanza

  [REPORT-stanza]
 your code
0 Karma

nawazns5038
Builder

I can understand what you mean but it does not effect anything , I use the same format for other field transforms as well in order to avoid confusion and it works well.

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!