Splunk Search

How to create a extracted filed using regex on existing field

pradjswl
Explorer

By default regex uses _raw field in the field extractor. I dont want to use regex as part of the query but I want a field to be created in the event/app like calculated filed so it always stay as new field rather than specifying in the search query.

Tags (1)
0 Karma

somesoni2
Revered Legend

The regular method of field extraction (using IFX utility OR from Settings-> Fields -> Field extractions) doesn't allow you to extract the fields from another fields, unless you can write a regex off the _raw fields that will extract the value that you need. You would need to use Field Transforms to use another fields (which should be available before the fields transform is run, it can't include auto extracted fields. See this for more information on order of search time field extractions). So, first you need to create a Field transform (Settings-> Fields -> Fields transforms, select SOURCE_KEY as your original field name) and then create a Field extraction which refers to that transform (Settings-> Fields -> Field extractions , type should be 'Uses transform' and provide name of transform).

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi pradjswl,
you should find the correct regex using the rex command:

| rex fiels=your_field "your_regex_with_field_extraction"

when you're sure of your regex go in [Settings -- Fields -- Fields extraction -- New] and then copy your regex in this way:

your_regex_with_field_extraction in your_field

in this way you perform a field extraction from the field you choosed (your_field.)

Bye.
Giuseppe

0 Karma

pradjswl
Explorer

@cusello

I tried [Settings -- Fields -- Fields extraction -- New] and then I copied "((?P[^;]+);(?P[^;)]+).*$ in x_UserAgent" without quote mark. I choose the name as a_xf_UA_OsType1,a_xf_UA_OsVer1 but it didnt work. I tried name as a_xf_UA it also didnt work.

I have correctly added Appto->SourcheTyppe, and chosen Type as Inline. still I dont get any custom filed created when I am running my sample query for that source type.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi pradjswl,
I tried for test to extract from the "source" field the description after "Perfmon:" and runs with the following field extraction

Perfmon:(?<my_field>[^ ]*) in source

Does Your extraction run using the rex command?

(probably there is a visualization problem but I don't see in your regex the field definition)

Bye.
Giuseppe

0 Karma

pradjswl
Explorer

Is there a way you can attach a screenshot from your UI ? possible we are talking two different things. It doestn allow me to share the attachment due to karma points

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

It is certainly possible to still use the Field Extraction Tool and create an extraction that will get your data, but field extractions are done by sourcetype, etc. and look at the whole event rather than a field of the event. This does take more work to get a good regex, but it will work. If the event is complicated enough, you will have to resort to a custom regex, but I suggest doing your own regex anyway, since it will be more exact, easier to read and maintain, and probably far more optimized. When you are in the FET, show the regex, then edit the regex, and put in one that really works well for your data.

If you need help with the regex, you can certainly ask for that help here, or on the Slack Splunk-user-groups channel.

0 Karma

pradjswl
Explorer

@cpetterborg
This is the regex query I have -> ((?P[^;]+);(?P[^;)]+).*$
& I want to use on the existing filed x_UserAgent.

What is "Slack Splunk-user-groups channel" i never heard about it ?

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

You can't do an automatic field extraction from a field, only an event. And the trigger for the event has to be a sourcetype, host or source. So in order to do what you want you have to do your field extraction from the entire event. This means that you imagine that there are no fields and do the field extraction as if the x_UserAgent field isn't extracted.

The Slack splunk-usergroups channel is a channel in Slack (http://slack.com) through which Splunk has set up a great user-driven discussion/support channel. Go to: signup

0 Karma

pradjswl
Explorer

sounds good. I just applied for sign up. Thanks for sharing

0 Karma

pradjswl
Explorer

@cpetterborg
I applied for sign up. I got invitation email but the link is not working.
[SOCIAL NETWORK] bot@stacktodo.com has invited you to join a Slack team
I responded to that email, but its not monitored distro.
Whom do i report this ? I checked on several browser and its not working today n yesterday.
Error "This site cant be reached"

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I don't know. I got that from the people on Slack. I'm not a Splunk employee, so I can't do any more. If I find out something, I'll let you know.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

There is an expiration in the reply email. Make sure you try to get back to the reply quickly enough that it works. That suggestion comes from our Splunk Sale Engineer.

0 Karma

gjanders
SplunkTrust
SplunkTrust

You wan to Extract Fields ? The linked documentation has the answer..., an extracted field will always appear in the search results if you have the appropriate permission to access it (ie. a global field extraction will apply to the particular sourcetype you extracted on in any application in Splunk).

0 Karma

pradjswl
Explorer

@garethatiag this approach uses the regex command in the search query. I am looking for a method to create a custom field which should display the data in filed list even when regex is not specified in the search query. I am sorry If I was not clear enough in my question, as it was bit complicated to explain.

0 Karma

gjanders
SplunkTrust
SplunkTrust

If you read and test what is in that document that saves a field extraction so you do not have to have it as part of future searches....

Alternatively you could create the field extraction via Settings -> Fields -> Field Extractions

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...