All my log files are in foldes named:
c:\blah\something\myapp_test\logs\somelogfile.log
=> app=myapp
=> env=test
I want to extract two fields from source, to make it easy to just search for "app=myapp env=test"
Since the fields are always there and should be a part of most queries, it seems like a good idea to add them at index time(?)
In etc/system/local I have added:
transforms.conf
[add_app_env]
SOURCE_KEY=source
REGEX=^.*\\\\([a-zA-Z0-9-]+)_([A-Z]+)\\\\.*
FORMAT=app::$1 env::$2
WRITE_META=true
props.conf
[add_app_field]
TRANSFORMS-app = add_app_env
[add_env_field]
TRANSFORMS-env = add_app_env
fields.conf
[add_app_env]
INDEXED=true
But I do not get my app and env fields and I have no idea how to debug this other than trial and error.
I tested my regular expression with a rex extraction - so I think that part works.
I also tried simplifying and just extracting a single field.
I don't think that creating these fields at index time will improve performance. Instead, I think it makes your configuration more brittle, complex and hard to manage.
You could easily do the same field extraction at search time:
props.conf
[source::*somelogfile.log]
EXTRACT-xyz=^[cC]\:\\\w+\\\w+\\(?<app>[a-zA-Z0-9\-]+)_(?<env>[a-zA-Z0-9\-]+)\\\w+\\somelogfile\.log in source