topic Re: how can I save rex to IFX? in Splunk Search

how can I save rex to IFX?

vrmandadi — Wed, 07 Mar 2018 15:33:17 GMT

I am using rex to split an existing field,can I use the same rex in IFX ?

| rex field="External ID" "(?.*)_"

I want to save the field1 in IFX .I went to settings-->fields---> Field extractions---->new--->selected sourcetype and used inline

But it was not showing up in the search

Re: how can I save rex to IFX?

somesoni2 — Wed, 07 Mar 2018 15:45:01 GMT

If you can make your regex work with _raw field (by changing it), then you can save it with settings-->fields---> Field extractions---->new . If not, you'd need to setup field transform, so that you can use other available field.

http://docs.splunk.com/Documentation/Splunk/7.0.2/Knowledge/Configureadvancedextractionswithfieldtransforms

Re: how can I save rex to IFX?

mayurr98 — Wed, 07 Mar 2018 15:51:58 GMT

I did not see field="External ID" 😕 @somesoni2 answer will do .

Re: how can I save rex to IFX?

gcusello — Wed, 07 Mar 2018 16:32:49 GMT

Hi vrmandadi,
I didn't understand why, but there a delay between field creation and availability in searches!
In addition, beware to spaces in the regex when you copy it.

Bye.
Giuseppe

Re: how can I save rex to IFX?

vrmandadi — Wed, 07 Mar 2018 18:55:10 GMT

Hello @cusello

yup I am aware of that it takes time but is there a problem with the quotes when placing in IFX

I just placed "External ID" (?.*)_ in the ifx bu the rex has something like this

| rex field="External ID" "(?.*)_"

Re: how can I save rex to IFX?

vrmandadi — Wed, 07 Mar 2018 18:58:44 GMT

sorry for the confusion @mayur98

I just placed "External ID" (?.*)_ in the ifx but the rex has something like this

| rex field="External ID" "(?.*)_"

Re: how can I save rex to IFX?

vrmandadi — Tue, 29 Sep 2020 18:20:07 GMT

@somesoni2

This is the sample event

RSN,interstitial/live_rsn_desktop_live ,Autozone/RSN_RSN_372462,Autozone/RSN_900014269,DIGITAL- 4Q17-2Q18 NBA Lakers Streaming_101917-042218_Live Stream,Autozone/RSN_ZONA1801_RSN,RSN APP,73369465,RSNAPP_LIVE,XXXXXXXXXXXX Network,Autozone/RSN_RSN_Live Stream,2/15/2018,620

I am trying to extract the one in bold

Re: how can I save rex to IFX?

somesoni2 — Wed, 07 Mar 2018 19:23:22 GMT

Is it always found in the 3rd last value in your raw data?? If yes, out of Autozone/RSN_RSN_Live Stream which part is (currently) extracted as "External ID" and which part should be your new field?

Re: how can I save rex to IFX?

vrmandadi — Tue, 29 Sep 2020 18:20:15 GMT

Nope,It is different for some events,I "External ID" has values like

ID_LIVE

MS_LIVE
RTS_LIVE

TT_LIVE
HG_LIVE

Cp_LIVE

I am trying to extract a new field called field removing the part after _ like ID,MS,TT,HG

Re: how can I save rex to IFX?

somesoni2 — Wed, 07 Mar 2018 20:04:21 GMT

Meanwhile give this regex a try

^([^,]+,){10}(?<YourNewField>([^_]+_)+)

https://regex101.com/r/lOwD2p/1

Re: how can I save rex to IFX?

somesoni2 — Wed, 07 Mar 2018 20:39:33 GMT

How is the field "External ID" extracted?? Do it's value always ends with _LIVE??

Re: how can I save rex to IFX?

vrmandadi — Wed, 07 Mar 2018 20:54:34 GMT

So its a csv file and it extracts that automatically as it is in the header and not all values end with _LIVE

Re: how can I save rex to IFX?

vrmandadi — Wed, 07 Mar 2018 20:55:24 GMT

This did not work,cant we extract from existing field and save it as new field?

Re: how can I save rex to IFX?

somesoni2 — Tue, 29 Sep 2020 18:15:42 GMT

Ok.. One final question, how is CSV field extraction setup, at search-time (using KV_MODE=csv) OR at indexed-time (INDEXED_EXTRACTIONS=csv )? You can see the order in which a search time field extraction setting is applied here. http://docs.splunk.com/Documentation/Splunk/6.5.0/Knowledge/Searchtimeoperationssequence#Search-time_operation_sequence
The field transforms (using which you can extract a field out of existing field) is executed before the KV_MODE field extraction so your "External ID" will not be available to field transform if "External ID" is extracted via KV_MODE.
In that case, I think you can do your extraction using it by using calculated fields which are done after KV_MODE or automatic field extractions. Follow instructions from below link
http://docs.splunk.com/Documentation/Splunk/6.5.0/Knowledge/CreatecalculatedfieldswithSplunkWeb
and use following a eval expression: replace('External ID',"(.+)_(.+)","\1")

Re: how can I save rex to IFX?

vrmandadi — Tue, 29 Sep 2020 18:20:21 GMT

I used INDEXED_EXTRACTIONS=csv ,so should I try uploading the csv again and change it to KV_MODE=CSV and then use it

Re: how can I save rex to IFX?

somesoni2 — Wed, 07 Mar 2018 21:48:18 GMT

You can test with INDEXED_EXTRACTIONS itself. Try both calculated fields and field transforms method.

Re: how can I save rex to IFX?

gcusello — Thu, 08 Mar 2018 08:01:33 GMT

Hi vrmandadi,
Please use Code Sample (button with numbers) to show your regexes, I cannot see them.

Anyway, in IFX you can insert field="External ID" in IFX putting it at the end of the regex, in other words (I cannot use your regex because I cannot see it):

(?<External_ID>.*)_ in "External ID"

I'd prefer (if possible) to rename field dropping spaces

(?<External_ID>.*)_ in External_ID

Bye.
Giuseppe

Re: how can I save rex to IFX?

vrmandadi — Thu, 08 Mar 2018 19:11:20 GMT

This was the rex I was using

| rex field="External Video ID" "(?.*)_"

Re: how can I save rex to IFX?

gcusello — Fri, 09 Mar 2018 07:30:28 GMT

Hi vrmandadi,
sorry if I repeat: I cannot see your regex, please use Code Sample!

Anyway the condition field="External Video ID" can be reproduced in IFX adding after the regex in <fieldname> , see the following example:

(?<External_ID>.*)_ in External_ID

In addition I suggest to not use spaces in field names, you can use field names with spaces at the end of your search using rename.

Bye.
Giuseppe