topic Re: Extract variable between pipe symbole from log and use it for query in Splunk Search

Extract variable between pipe symbol from log and use it for query

krishman23 — Mon, 12 Oct 2020 21:08:18 GMT

I have a log generated in splunk which will have unique id in with pipe symbols:

ex:

19:46:47.146 - [http-nio-8000-exec-9] INFO edu.test.controller |{My Var1}|{My Var2}|{myVar3}| - {log message}.

I need to perform a query based on {My Var1}.

Also need to list (dedup) all logs best on {My Var1}.

Re: Extract variable between pipe symbole from log and use it for query

ITWhisperer — Mon, 12 Oct 2020 21:08:37 GMT

| rex "\|(?<Var1>[^\|]+)\|(?<Var2>[^\|]+)\|(?<Var3>[^\|]+)\|\s\-\s(?<logmessage>.*)"

Re: Extract variable between pipe symbole from log and use it for query

krishman23 — Mon, 12 Oct 2020 21:28:55 GMT

but var1,var2,var3 are unknown to me, those are variables in log

Re: Extract variable between pipe symbole from log and use it for query

ITWhisperer — Mon, 12 Oct 2020 21:41:30 GMT

The rex extracts the value between the first two pipes into a field called Var1, the value between the second and third pipes into a field called Var2, etc. You can then use these fields in your query or just display them in as columns in a table. Is this not what you want to do?

Re: Extract variable between pipe symbole from log and use it for query

krishman23 — Mon, 12 Oct 2020 22:12:51 GMT

my bad , got it what you are saying.,

As second part of my question how can we user Var1 as variable for next search?

Re: Extract variable between pipe symbole from log and use it for query

ITWhisperer — Tue, 13 Oct 2020 07:02:49 GMT

Depends on what you want to get out of your data

| dedup Var1 | where Var2="xyz" | stats count by Var3

Re: Extract variable between pipe symbol from log and use it for query

krishman23 — Fri, 23 Oct 2020 15:14:05 GMT

@ITWhisperer: i additional case in this rex, some time my logmessage will have URL as

Req URL : hello/test/content

or Req URL : hello/test/content/ , i need to truncate / of second request. Can you help with this ?

Re: Extract variable between pipe symbol from log and use it for query

ITWhisperer — Fri, 23 Oct 2020 15:35:15 GMT

| rex field=events mode=sed "s/(?<url>[^\/]+\/[^\/]+\/[^\/]+)(?<slash>\/*)/\1/g"