topic Re: How to extract a particular string from the event logs in Splunk Search

How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 17:21:57 GMT

Hi Guys,

I have a scenario where i need to extract the file name from the event logs. The Event log first line looks like below.

Event Log:

[INFO] 2021-09-30T00:04:17.052Z 8d5eb00a-d033-49a9-9d0f-c61011e4ae51 {"Records": [{"eventVersion": }]

Now i need to write a rex query to extract the file name "8d5eb00a-d033-49a9-9d0f-c61011e4ae51" from above event log. This file name changes for the every search query along with the timestamp.

Can someone suggest me how to resolve this?

Thanks.

Re: How to extract a particular string from the event logs

ITWhisperer — Wed, 06 Oct 2021 18:18:31 GMT

| rex "\[INFO\]\s\S+\s(?<filename>\S+)\s"

Re: How to extract a particular string from the event logs

richgalloway — Wed, 06 Oct 2021 18:22:01 GMT

See if this helps. It looks for the milliseconds in the timestamp, a time zone indicator, and some white space then takes everything up to the next white space to be the file name.

... | rex "\.\d{3}\w+\s(?<filename>\S+)"

Re: How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 18:43:02 GMT

I have tried it as you suggested. But its returning the same result as before. I have pasted the logs below. I have to remove the timestamp and return the just 12345 as the output.

Logs are:

[INFO] 2021-10-02T00:09:50.398Z 12345 {"Records": [{"eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "us-east-1", "eventTime": "2021-10-02T00:09:42.743Z", "eventName": "ObjectCreated:Put", "userIdentity":

Re: How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 18:43:35 GMT

@richgalloway

I have tried it as you suggested. But its returning the same result as before. I have pasted the logs below. I have to remove the timestamp and return the just 12345 as the output.

Logs are:

Re: How to extract a particular string from the event logs

richgalloway — Wed, 06 Oct 2021 18:53:55 GMT

My query works in regex101.com with both of your sample events. If it's not working for you then we need more information. Please share the full query you are using to extract the string. What is the result (the OP didn't say)?

Re: How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 19:15:39 GMT

the full query is looks like below:

I am looking for source lambda logs with a json file to return a reqid.

source=/aws/lambda/sample test.json | rex "\.\d{3}\w+\s(?<file>\S+)"

In log events it returns a reqid '12345' as below. There is some space after timestamp and after reqid.

Logs are:

I hope this is clear now. let me know if i need to add extra details

Re: How to extract a particular string from the event logs

richgalloway — Wed, 06 Oct 2021 19:29:45 GMT

Based on the latest response and the OP, it appears the two regular expressions provided do what was asked.

Please explain how "12345" is not what is needed then tell us what is needed from each event.

Re: How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 19:35:02 GMT

@richgalloway

source=/aws/lambda/sample test.json

source=/aws/lambda/sample test.json | rex "\.\d{3}\w+\s(?<file>\S+)"

Both the above commands returning the same result as below.

Logs are:

Adding the rex to extract and display the '12345' not working in my scenario. do wee need to update rex command?

Thanks.

Re: How to extract a particular string from the event logs

kumarnis45 — Wed, 06 Oct 2021 22:57:07 GMT

@ITWhisperer ,

I still couldn't figure it out to get this work. Tried using rex with different possibilities but none of them are working. can you please suggest a better solution?

Thanks.

Re: How to extract a particular string from the event logs

ITWhisperer — Thu, 07 Oct 2021 06:43:07 GMT

What exactly do you get when you use the command suggested?

Don't forget, the only information we have is what you post! The more information you can give us, the easier it will be for us to help.

Re: How to extract a particular string from the event logs

kumarnis45 — Thu, 07 Oct 2021 12:08:20 GMT

Please check the attached screenshot

Re: How to extract a particular string from the event logs

kumarnis45 — Thu, 07 Oct 2021 12:15:10 GMT

please check the attached picture. It has the command you suggested me to run with the OP

Re: How to extract a particular string from the event logs

ITWhisperer — Thu, 07 Oct 2021 12:15:27 GMT

It looks like you have multiple white spaces - try this

| rex "\[INFO\]\s+\S+\s+(?<filename>\S+)\s"

Re: How to extract a particular string from the event logs

kumarnis45 — Thu, 07 Oct 2021 12:22:28 GMT

Not working though. Returning the same result

Re: How to extract a particular string from the event logs

richgalloway — Thu, 07 Oct 2021 12:22:44 GMT

As @ITWhisperer said, there appears to be more than one space between the timestamp and the regid. Try this command.

... | rex "\.\d{3}\w+\s+(?<file>\S+)"

Re: How to extract a particular string from the event logs

kumarnis45 — Thu, 07 Oct 2021 14:13:38 GMT

This is working. adding | table filename after rex fixed it.

| rex "\[INFO\]\s+\S+\s+(?<filename>\S+)\s" | table filename