topic Re: field extraction from source log path in Splunk Search

field extraction from source log path

ravir_jbp — Mon, 25 Sep 2023 14:44:36 GMT

how to extract the node name from the different GC source location:

I have below sample three source location and I am looking for rex that can extract node name as "node02, Node03 and "web39". My rex command is not working.

source= E:\total\int\ts1\Ddoss\node\node02\data\gc.log
source=E:\total\int\ts1\Ddoss\swxx\node03\data\gc.log
source=E:\total\int\ts1\Ddoss\web\web39\data\gc.log

Re: field extraction from source log path

ITWhisperer — Mon, 25 Sep 2023 14:50:22 GMT

| rex field=source "^([^\\\\]+\\\\){6}(?<node>[^\\\\]+)"

Re: field extraction from source log path

ravir_jbp — Thu, 19 Oct 2023 14:37:46 GMT

@ITWhisperer

I tried using above rex for these log source but not working:

For below 5 different log source I like to extract node number like node06, node03, node01

E:\view\int\t4\apch\node\node06\log\server.log
E:\view\int\t4\apch\node\node06\log\run.log
E:\view\int\t4\apch\node\node03\log\server.log
E:\view\int\t4\apch\node\node01\log\server.log
E:\view\int\t4\apch\node\node01\log\run.log

For below 3 log source I like to extract as core02, web37, core01

E:\view\int\t4\logs\core02-core.log
E:\view\int\t4\logs\web37-wfmws.log
E:\view\int\t4\logs\core01-core.log

Since both log format is different above solution you shared is not working. Please help

Re: field extraction from source log path

ITWhisperer — Thu, 19 Oct 2023 15:05:19 GMT

Here is a runanywhere example showing it working

| makeresults format=csv data="source E:\view\int\t4\apch\node\node06\log\server.log E:\view\int\t4\apch\node\node06\log\run.log E:\view\int\t4\apch\node\node03\log\server.log E:\view\int\t4\apch\node\node01\log\server.log E:\view\int\t4\apch\node\node01\log\run.log E:\view\int\t4\logs\core02-core.log E:\view\int\t4\logs\web37-wfmws.log E:\view\int\t4\logs\core01-core.log" | rex field=source "^([^\\\\]+\\\\){5}(?<node>[^-]+)" | rex field=source "^([^\\\\]+\\\\){6}(?<node>[^\\\\]+)"

Note if these different formats for source are used in the same search then the order is significant, otherwise just use the relevant rex pertaining to the source name format

Re: field extraction from source log path

yuanliu — Thu, 19 Oct 2023 16:11:39 GMT

I would be cautious to anchor regex as closely as the data is regular. Something like

| rex field=source "\\\t4\\\(apch\\\node|logs)\\\(?<node>[^-\\\\]+)"

This should give

node	source
node06	E:\view\int\t4\apch\node\node06\log\server.log
node06	E:\view\int\t4\apch\node\node06\log\run.log
node03	E:\view\int\t4\apch\node\node03\log\server.log
node01	E:\view\int\t4\apch\node\node01\log\server.log
node01	E:\view\int\t4\apch\node\node01\log\run.log
core02	E:\view\int\t4\logs\core02-core.log
web37	E:\view\int\t4\logs\web37-wfmws.log
core01	E:\view\int\t4\logs\core01-core.log

You can play with the emulation @ITWhisperer offered and compare with real data.