How to fetch the keywords from raw logs?

[aditsss]

Hi Team,

How we can fetch the below keywords from raw logs:

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1] ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

2023-07-13 02:42:02.915 [INFO ] [pool-2-thread-1] FileSensor - Start Reading Account balance Data File, QACDU.D062623.T065000

2023-07-13 18:53:10.226 [INFO ] [pool-5-thread-1] FileSensor - Completed Account balance file processing, QACDU.D062623.T065000 records processed: 105932244, Kafka counter: 0

[gcusello]

Hi @aditsss,

sorry but it isn't clear the rule of your extraction: do you want the string after the minus sign?

If this is your requirement, please try:

| rex "-\s+(?<message>.*)"

that you can test at https://regex101.com/r/JUWcZh/1

Ciao.

Giuseppe

[aditsss]

@gcusello 

For this particular logger I just need file name:

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1] ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

How Can I fetch it.

[gcusello]

Hi @aditsss,

please try this:

| rex "-\s+(?<message>.*)"
| rex "\/(?<message1>.+)$
| eval message=if(match(message,"finished reading file%",message1,message)

Ciao.

Giuseppe

[aditsss]

@gcusello 

I tried this:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" | rex "\/(?<message1>.+)$"|eval message=if(match(message,"finished reading file%",message1,message))|stats count by message1

getting this result:

Error in 'EvalCommand': The arguments to the 'match' function are invalid.

[gcusello]

Hi @aditsss,

change parenthesis and use message not message1:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" 
| rex "\/(?<message1>.+)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message

 Ciao.

Giuseppe

[aditsss]

@gcusello 

I just want file name

2023-06-29 09:41:53.884 [INFO ] [pool-2-thread-1] ArchivalProcessor - finished reading file /absin/TRIM.ARCH.D062923.T052525

with this query :

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)"
| rex "\/(?<message1>.+)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message1

I am getting result like this:

absin/TRIM.ARCH.D062223.T081112

[gcusello]

Hi @aditsss,

as I said, use message, not message1 in the stats command, anyeay, please try this:

index="600000304_d_gridgain_idx*" sourcetype=600000304_gg_abs_ipc2 source!="/var/log/messages" "finished reading file"
| rex "-\s+(?<message>.*)" 
| rex "\/\w+\/(?<message1>.*)$"
| eval message=if(match(message,"finished reading file%"),message1,message)
| stats count by message

 Ciao.

Giuseppe