regex field extraction not from _raw

[tamir]

hey guys
did someone ever happed to come through this problem. I'm using Splunk Cloud 
I'm trying to extract a new field using regex but the data are under the source filed 

| rex field=source "Snowflake\/(?<folder>[^\/]+)"

this is the regex I'm using when i use it in the search it works perfect. but the main goal is to save this search as a permanent field.
i know that the the field extraction draw from the "_raw" there is an option to direct the Cloud to pull from the source and save it a permanent field.

[richgalloway]

If the rex command works perfectly then you should have a field called "folder" with the extracted data in it.  Is that what is happening?  If not, please describe how the rex command is not acting as expected.  Note that the "folder" field will be present only within the query that extracted it.  If you need the field to be available to all queries then it will have to be extracted at index-time using a transform.

[tamir]

@richgalloway  thanks for replay the | rex is working as it should the problem start when I'm trying to save the Regex. and this is cause by the fact i need to save the regex from the "source" field and no from the "_raw" field.
The main goal is to add another field in all searches without using the | rex command every time. 

[isoutamo]

Hi

you must use transforms to get this done. 

1. Create Field transformations on your app e.g.
   1. get_directory_from_linux_audit_source
   2. Type regex-based
   3. Regular expression like: /log/([^/]+)/
   4. Format: directory::$1
   5. Source key: source
   6. Save and give needed permissions like app and roles which can use it
2. Create Field extractions 
   1. Name: <what ever you want to call it>
   2. Apply to: sourcetype named: e.g. linux_audit or what ever this is in your node
   3. Type: Uses transforms
   4. Extraction/Transforms: <from above like get_directory_from_linux_audit_source>
   5. Save and give needed permission like above
3. Wait that this will be applied to all needed places on SCP
4. Use it like: index=<your index> sourcetype=<your sourcetype> <Your field name>=*

r. Ismo

[tamir]

@isoutamo  hey thanks for the replay. 
I've been trying to create the following two you shared, but somehow i still don't see the the field in the field section I'm sharing the process I've taking. 
let me know if I'm missing something. 

  

[isoutamo]

Are you sure that your transformation's name is same on both places (e.g. sonw vs snow etc.)?

[gcusello]

Hi @tamir,

you have to create a new field using the following syntax:

Snowflake\/(?<folder>[^\/]+) in source

in few words you have to add "in" and the firld to use for the extraction.

ciao.

Giuseppe

[tamir]

hey @gcusello  thanks for your replay.
It seems like the capture do not capture any of the fields i needed, I've tried to save it an even to play a bit with the syntax.  but still no success. 

 

[gcusello]

Hi @tamir ,

my solution is to save the extraction in an field extraction,

if you want to use the regex in a search, you have to add it to a search:

index=your_index
| rex field=source "Snowflake\/(?<folder>[^\/]+)"

Ciao.

Giuseppe