topic Re: data extraction from log without any links between them in Splunk Search

data extraction from log without any links between them

Devi13 — Thu, 31 Aug 2023 13:13:01 GMT

Hello Team,

I have logs with the below pattern

08/31/2023 8:00:00:476 am ........ count=0

08/31/2023 8:00:00:376 am ........ process started

08/31/2023 8:00:00:376 am...... XXX Process

I need the process name and the count to be displayed together but I dont have any common values/names/strings to match them.

I have 4 similar process and the count together in the logs..is there a way on how we can match them together.

Any help is much appreciated.

Re: data extraction from log without any links between them

ITWhisperer — Thu, 31 Aug 2023 14:29:58 GMT

In the olden days, I would have said computers are dumb, they can only do what you tell them to do, but with advances in AI this is becoming less true. Having said that, Splunk still requires you to tell it what to do and it can automate what you are doing. So, how would you as a human determine how these events are related?

Re: data extraction from log without any links between them

gcusello — Thu, 31 Aug 2023 21:43:19 GMT

Hi @Devi13,

I suppose that at least you have the host where logs coming from and the sourcetype,

in addition, can you say that the first event is "count=0" and the last event is "XXX Process"?

if this is true, this is one of the few situation to use the transaction command:

index=your_index sourcetype=your_sourcetype ("count=0" OR "process started" OR "Process") | transaction host startswith="count=0" endswith="Process" | table Process count

Ciao.

Giuseppe

Re: data extraction from log without any links between them

isoutamo — Fri, 01 Sep 2023 06:32:06 GMT

It's just like @ITWhisperer said. There must be some way. how you can combine those events which belongs to one transaction. With your current example there haven't been any information about that. When you can found some common information which are on all of those then you can you try e.g. @gcusello's way to combine those together.

I assume that there could be outputs from several process on one or more nodes which generates those log events? If there is only one node and only one process at time, then you can use @gcusello's example as is.

Best way to continue this is ask that developer add some unique transaction id (e.g uuidgen -> B49A0412-3EBB-4377-A026-D8E43EC9F7F1 different output on every run) on logs which we could use to combine transactions together.

r. Ismo

Re: data extraction from log without any links between them

Devi13 — Fri, 01 Sep 2023 07:21:45 GMT

Thank you so much all for your inputs, we were able to get the data from another set of logs.

Thank you so muchh!!