Dashboards & Visualizations

How to fetch the keywords from raw logs?

aditsss
Motivator

Hi Team,

How we can fetch the below keywords from raw logs:

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1] ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

2023-07-13 02:42:02.915 [INFO ] [pool-2-thread-1] FileSensor - Start Reading Account balance Data File, QACDU.D062623.T065000

2023-07-13 18:53:10.226 [INFO ] [pool-5-thread-1] FileSensor - Completed Account balance file processing, QACDU.D062623.T065000 records processed: 105932244, Kafka counter: 0

Labels (3)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @aditsss,

sorry but it isn't clear the rule of your extraction: do you want the string after the minus sign?

If this is your requirement, please try:

| rex "-\s+(?<message>.*)"

that you can test at https://regex101.com/r/JUWcZh/1

Ciao.

Giuseppe

0 Karma

aditsss
Motivator

@gcusello 

For this particular logger I just need file name:

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

How Can I fetch it.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @aditsss,

please try this:

| rex "-\s+(?<message>.*)"
| rex "\/(?<message1>.+)$
| eval message=if(match(message,"finished reading file%",message1,message)

Ciao.

Giuseppe

0 Karma

aditsss
Motivator

@gcusello 

I tried this:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" | rex "\/(?<message1>.+)$"|eval message=if(match(message,"finished reading file%",message1,message))|stats count by message1

 

getting this result:

Error in 'EvalCommand': The arguments to the 'match' function are invalid.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @aditsss,

change parenthesis and use message not message1:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" 
| rex "\/(?<message1>.+)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message

 Ciao.

Giuseppe

0 Karma

aditsss
Motivator

@gcusello 

I just want file name

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

with this query :

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)"
| rex "\/(?<message1>.+)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message1

I am getting result like this:

absin/TRIM.ARCH.D062223.T081112

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @aditsss,

as I said, use message, not message1 in the stats command, anyeay, please try this:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" 
| rex "\/\w+\/(?<message1>.*)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message

 Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...