Splunk Search

Why am I unable to extract 2 fields from source at index-time with my current configuration and regex?

lassel
Communicator

All my log files are in foldes named:

 c:\blah\something\myapp_test\logs\somelogfile.log

 => app=myapp 
 => env=test

I want to extract two fields from source, to make it easy to just search for "app=myapp env=test"

Since the fields are always there and should be a part of most queries, it seems like a good idea to add them at index time(?)

In etc/system/local I have added:

transforms.conf

[add_app_env]
SOURCE_KEY=source
REGEX=^.*\\\\([a-zA-Z0-9-]+)_([A-Z]+)\\\\.*
FORMAT=app::$1 env::$2
WRITE_META=true

props.conf

[add_app_field]
TRANSFORMS-app = add_app_env

[add_env_field]
TRANSFORMS-env = add_app_env

fields.conf

[add_app_env]
INDEXED=true

But I do not get my app and env fields and I have no idea how to debug this other than trial and error.

I tested my regular expression with a rex extraction - so I think that part works.
I also tried simplifying and just extracting a single field.

0 Karma
1 Solution

lguinn2
Legend

I don't think that creating these fields at index time will improve performance. Instead, I think it makes your configuration more brittle, complex and hard to manage.

You could easily do the same field extraction at search time:

props.conf

[source::*somelogfile.log]
EXTRACT-xyz=^[cC]\:\\\w+\\\w+\\(?<app>[a-zA-Z0-9\-]+)_(?<env>[a-zA-Z0-9\-]+)\\\w+\\somelogfile\.log in source

View solution in original post

Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...