topic Re: How to extract two fields separated by delimiter comma ","? in Splunk Search

How to extract two fields separated by delimiter comma ","?

akash_akkis — Wed, 24 Sep 2014 11:48:35 GMT

Here is my output from splunk

    2014.09.19 13:33:37.739,2014.09.19       13:33:37.726,13,dsd45sd4,gdm=DT30&kslsdj=LiveToday,/cpt/mtp/fgkt/,124.235.124.2,gzip,NONE_FOUND,data,

I want

gdm=DT30&kslsdj=LiveToday

and

data

field please help

Re: How to extract two fields separated by delimiter comma ","?

mathiask — Wed, 24 Sep 2014 12:31:57 GMT

Is the value you want is
field1="gdm=DT30&kslsdj=LiveToday"
field2="data" ?

The extraction regex will depend quite a bit whether the position is stable in relation to the separators ","

possible would be something like

[^,]+,[^,]+,[^,]+,(?<field1>[^,]+),[^,]+,[^,]+,[^,]+,[^,]+,(?<field2>[^,]+),

This is not tested it could be that a backslash is required before the comma or that the questionmark has to be repositioned
you can simply test it by using a search on the corresponding data using the command rex like

<search> | rex field=_raw "[^,]+,[^,]+,[^,]+,(?<field1>[^,]+),[^,]+,[^,]+,[^,]+,[^,]+,(?<field2>[^,]+),"

If the fields get extracted correctly you can configure them using the settings > fields > field extractions to extract them automatically on every search on this source or sourcetype or host

you can also use the Interactive field extractor of splunk I find it a good start but it sometimes produces weird expressions so I tend to rework the manually.

Here some documentation sources

http://docs.splunk.com/Splexicon:Fieldextraction
http://docs.splunk.com/Documentation/Splunk/6.1.3/Knowledge/ExtractfieldsinteractivelywithIFX
http://docs.splunk.com/Documentation/Splunk/6.1.3/SearchReference/rex
http://docs.splunk.com/Documentation/Splunk/6.1.3/Knowledge/Managesearch-timefieldextractions

Re: How to extract two fields separated by delimiter comma ","?

tgow — Wed, 24 Sep 2014 13:35:45 GMT

Another method is to use the props/transforms combination that uses the comma as a delimiter. Here is an example:

props.conf

 [sourcetype]
 REPORT-fields = commafields

transforms.conf

 [commafields]
 DELIMS = ","
 FIELDS = field1, field2, field3, field4, uri, field6, field7, field8, field9, date

Here is a link to more information:

http://docs.splunk.com/Documentation/Splunk/6.1.3/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles

Re: How to extract two fields separated by delimiter comma ","?

mathiask — Wed, 24 Sep 2014 13:46:34 GMT

If you don't feel comfortable or have direct access to the conf files you can do this also in the web UI

Simply put the regex into a transform settings > fields > transformations

Then you create a new extraction in settings > fields > field extractions, instead of inline you use tranformation and give the name of the transformation

This setup has the advantage that you can use the same regex/transformation in multiple extractions, therefore if the regex changes you only have to fix it once and not in every extraction.

you can do most of the changes in the web UI which is mostly good enough, but not all of it..