topic Re: Matching a field in a string using if/eval command. in Splunk Search

Matching a field in a string using if/eval command.

aotuga001 — Mon, 15 Apr 2024 23:26:40 GMT

I have two logs below, log a is throughout the environment and would be shown for all users. log b is limited to specific users. I only need times for users in log b.

log a: There is a file has been received with the name test2.txt

lob b: The file has been found at the second destination C://user/test2.txt

I am trying to write a query that captures the time between log a and log b without doing a subsearch, so far I have

index=a, env=a, account=a ("There is a file" OR "The file has been found")|field filename from log b | field filename2| eval Endtime = _time | ****Here is where I am lost, I was hoping to use if/match/like/eval to see to capture the start time where log b filename can be found in log a. I have this so far******

| eval Starttime = if(match(filename,"There is%".filename2."%"),_time,0)

I am not getting any 1s, just 0s. I am pretty sure this is the problem "There is%".filename2."%", how do I correct it.

Re: Matching a field in a string using if/eval command.

bowesmana — Mon, 15 Apr 2024 22:44:56 GMT

Does logb come from "index=a env=a account="?

If not, then you need to search both data sets to find loga and logb.

I am not sure what your SPL

|field filename from log b | field filename2|

is doing, as that's not SPL.

your match statement is not valid either, you are using SQL wildcards (%) - match takes regular expressions.

Can you give an example of your data that you'd like to match

Re: Matching a field in a string using if/eval command.

aotuga001 — Mon, 15 Apr 2024 22:50:34 GMT

Yes log b and log a have the same index=a env=a account=.
SPL -----> rex field=_raw "The file has been found at the second destination[a-zA-Z0-9]+\/(?<filename2>[^\"]*)" This works I get the file names.
This is exactly the logs that I am trying to match, I was using if(like....) at first.

log a: There is a file has been received with the name test2.txt

lob b: The file has been found at the second destination C://user/test2.txt

Re: Matching a field in a string using if/eval command.

bowesmana — Mon, 15 Apr 2024 22:59:58 GMT

If you have two events, you can't just match things between events - the text from loga does not exist when running the match statement for the logb data.

Without seeing your SPL it's hard to know what you are doing - can you post the entire SPL - please do this in a code block (</> button)

If you have two events, you need to correlate them together using stats on a common field, in this case, your file name, so extract the file name from both events and then define a "message type" - log a or b and then you can do something like this logic

| eval logtype=if(condition..., "loga", "logb") | rex "....(?<filename>....)" | stats count values(logtype) as logtypes min(_time) as StartTime max(_time) as EndTime by filename | where count>1 AND logtypes="loga" AND logtypes="logb"

Re: Matching a field in a string using if/eval command.

aotuga001 — Mon, 15 Apr 2024 23:10:28 GMT

This is exactly what I am doing, nothing more. Let me try your logic.

index= cloudaccount= cloudenv=impl source= (string in log a OR string in log b) | rex field=_raw "/[a-zA-Z0-9]+\/(?<filename>[^\"]*)"| rex field=_raw "[a-zA-Z0-9]+\/(?<filename2>[^\"]*)" | eval Endtime = strftime(_time, "%H:%M:%S:%Q") | eval Starttime = if(match(filename,"found %".filename2."%"),1,0) | stats values(Starttime) by filename

Re: Matching a field in a string using if/eval command.

bowesmana — Mon, 15 Apr 2024 23:24:39 GMT

Great, thanks - that makes it easier!

OK, so it looks like you are trying to compare fields in two separate events - you can't do that unless you collapse the two.

You should use rex to extract a single filename and then do something similar to my previous post. Here's an example that hopefully will point you in the right direction.

It creates two events 60 seconds apart each containing a filename - the rex statements extract filename and logtype and the stats will join the events together and by using min and max on _time you can get the start and end times for the pair of events. The final where clause will ensure that you have seen both loga and logb events.

| makeresults | eval v=split("log a: There is a file has been received with the name test2.txt###log b: The file has been found at the second destination C://user/test2.txt", "###") | mvexpand v | streamstats c | eval _time=now()-(60*c) | rename v as _raw ``` Above is simply a data set up example ``` | rex field=_raw "(/[a-zA-Z0-9]+\/|name )(?<filename>[^\"]*)" | rex field=_raw "log (?<logtype>\w)" | stats count min(_time) as Starttime max(_time) as Endtime values(logtype) as logtype by filename | where count=2 AND logtype="a" AND logtype="b" | eval diff = Endtime - Starttime

Hope this helps.

Re: Matching a field in a string using if/eval command.

aotuga001 — Wed, 17 Apr 2024 04:28:13 GMT

It was perfect 😁. I ended up doing it like this because of how the logs are stored in our environment.

index=c account=1 env=lower source="logfiles" ("destination" OR "received") | eval logtype = if(like(_raw, "destination%"),"logb","loga") | rex field=_raw filename in loga| rex field=_raw filename in logb| stats count min(_time) as Starttime max(_time) as Endtime values(logtype) as logtype by filename | where count=2 AND logtype="loga" AND logtype="logb" | eval diff = Endtime - Starttime | stats avg(diff)