Splunk Search

How to extract a particular string from the event logs

kumarnis45
Path Finder

Hi Guys,

   

    I have a scenario where i need to extract the file name from the event logs. The Event log first line looks like below.

Event Log:

[INFO] 2021-09-30T00:04:17.052Z 8d5eb00a-d033-49a9-9d0f-c61011e4ae51 {"Records": [{"eventVersion": }]

 

Now i need to write a rex query to extract the file name "8d5eb00a-d033-49a9-9d0f-c61011e4ae51" from above event log. This file name changes for the every search query along with the timestamp.

 

Can someone suggest me how to resolve this?

 

Thanks.

Labels (1)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

It looks like you have multiple white spaces - try this

| rex "\[INFO\]\s+\S+\s+(?<filename>\S+)\s"

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

See if this helps.  It looks for the milliseconds in the timestamp, a time zone indicator, and some white space then takes everything up to the next white space to be the file name.

... | rex "\.\d{3}\w+\s(?<filename>\S+)"
---
If this reply helps you, an upvote would be appreciated.

kumarnis45
Path Finder

@richgalloway 

 

I have tried it as you suggested. But its returning the same result as before.  I have pasted the logs below. I have to remove the timestamp and return the just 12345 as the output.

 

Logs are:

 

[INFO] 2021-10-02T00:09:50.398Z 12345 {"Records": [{"eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "us-east-1", "eventTime": "2021-10-02T00:09:42.743Z", "eventName": "ObjectCreated:Put", "userIdentity":

0 Karma

richgalloway
SplunkTrust
SplunkTrust

My query works in regex101.com with both of your sample events.  If it's not working for you then we need more information.  Please share the full query you are using to extract the string.  What is the result (the OP didn't say)?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

kumarnis45
Path Finder

the full query is looks like below:

I am looking for source lambda logs with a json file to return a reqid. 

source=/aws/lambda/sample test.json | rex "\.\d{3}\w+\s(?<file>\S+)"

 

In log events it returns a reqid '12345' as below. There is some space after timestamp and after reqid.

Logs are:

 

[INFO2021-10-02T00:09:50.398Z 12345 {"Records": [{"eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "us-east-1", "eventTime": "2021-10-02T00:09:42.743Z", "eventName": "ObjectCreated:Put", "userIdentity":

 

I hope this is clear now. let me know if i need to add extra details

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Based on the latest response and the OP, it appears the two regular expressions provided do what was asked.

Please explain how "12345" is not what is needed then tell us what is needed from each event.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

kumarnis45
Path Finder

please check the attached picture. It has the command you suggested me to run with the OP

0 Karma

kumarnis45
Path Finder

@richgalloway 

source=/aws/lambda/sample test.json 

source=/aws/lambda/sample test.json | rex "\.\d{3}\w+\s(?<file>\S+)"

Both the above commands returning the same result as below.

Logs are:

[INFO2021-10-02T00:09:50.398Z 12345 {"Records": [{"eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "us-east-1", "eventTime": "2021-10-02T00:09:42.743Z", "eventName": "ObjectCreated:Put", "userIdentity":

 

Adding the rex to extract and display the '12345' not working in my scenario. do wee need to update rex command?

Thanks.

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

As @ITWhisperer said, there appears to be more than one space between the timestamp and the regid.  Try this command.

... | rex "\.\d{3}\w+\s+(?<file>\S+)"
---
If this reply helps you, an upvote would be appreciated.
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust
| rex "\[INFO\]\s\S+\s(?<filename>\S+)\s"

kumarnis45
Path Finder

I have tried it as you suggested. But its returning the same result as before.  I have pasted the logs below. I have to remove the timestamp and return the just 12345 as the output.

 

Logs are:

 

[INFO] 2021-10-02T00:09:50.398Z 12345 {"Records": [{"eventVersion": "2.1", "eventSource": "aws:s3", "awsRegion": "us-east-1", "eventTime": "2021-10-02T00:09:42.743Z", "eventName": "ObjectCreated:Put", "userIdentity":

0 Karma

kumarnis45
Path Finder

@ITWhisperer ,

 

   I still couldn't figure it out to get this work. Tried using rex with different possibilities but none of them are working. can you please suggest a better solution?

 

Thanks. 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

What exactly do you get when you use the command suggested?

Don't forget, the only information we have is what you post! The more information you can give us, the easier it will be for us to help.

0 Karma

kumarnis45
Path Finder

Please check the attached screenshot

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It looks like you have multiple white spaces - try this

| rex "\[INFO\]\s+\S+\s+(?<filename>\S+)\s"

View solution in original post

kumarnis45
Path Finder

This is working. adding | table filename after rex fixed it.

 

| rex "\[INFO\]\s+\S+\s+(?<filename>\S+)\s" | table filename

0 Karma

kumarnis45
Path Finder

Not working though. Returning the same result

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!