Splunk Search
Highlighted

Why are field transformations not capturing from source field?

Loves-to-Learn

I am trying to create a new field called collection which is extracted from the existing source field. I am able to extract the field during an adhoc search, but want to create it using the field transformations without having to generate a regex during each search.

The source field value is just a path (ex: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log) and I am extracting part of that path to aggregate by collection. I want to specifically want to target the source field to regex not _raw.

  • BEFORE Extraction: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log
  • AFTER Extraction: collection=SomeCollectionName

alt text

I have created a field transformation called testcollection and nothing is being extracted at search time.

Here are my settings for the testcollection field transformations (permissions are set for everyone to be able to read in the search app)
alt text

Maybe I have a misunderstanding of Field Transformations and should be using field extractions any guidance would be helpful. I am just using the default formatting, but maybe that is incorrect.

Note:

I have gone through the documentation for field transformations and field extracts. I understand how to extract new fields during a search, but I want this new field to be available to all of the users in our account.

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

SplunkTrust
SplunkTrust

Use and extraction for this instead of a transformation:

alt text
alt text

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

Loves-to-Learn

Thank you. I tried that at first and I was running into this issue (I had to disable my json browser extension)

Your entry was not saved. The following error was reported: SyntaxError: Unexpected token < in JSON at position 0.

https://answers.splunk.com/answers/106487/your-entry-was-not-saved-the-following-error-was-r.html

Once I disabled my extension I was able to create the field extraction. I will report back if it works as expected.

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

Loves-to-Learn

I used both field extractions and field transformations. The problem is raw does not include the source field. I am able to parse anything that matches the regex in _raw but it is not parsing the actual source field. I want to be able to target the source field like I can during adhoc searches example:
source=D:\Logs\MyCollection
SomeDomain\SomeDomain.log

Some search
|rex field=source "(?<CollectionMap>\w+)\_"

Expected: CollectionMap=MyCollection
Actual: Anything with an underscore in _raw matches

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

SplunkTrust
SplunkTrust

Do you have access directly to the props.conf file instead of using the UI?

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

Loves-to-Learn

We are using splunk cloud so no access to the props.conf

0 Karma
Highlighted

Re: Why are field transformations not capturing from source field?

Loves-to-Learn

I think I found my answer here:
https://answers.splunk.com/answers/149597/im-struggling-with-how-i-should-be-doing-inputs-and-also-p...

I will have to open a support ticket with splunk. Thanks for your assistance.

0 Karma