topic Re: Rex with optional end of message in Splunk Search

Rex with optional end of message

splunksogetiht — Thu, 18 Sep 2014 14:15:01 GMT

I have a log like that :

2014-29-08 08:28:45,817.366  - INFO - message1
2014-29-08 08:28:45,817.366  - WARN - message2 ID = 458315
2014-29-08 08:28:45,817.366  - DEBUG- message3 ID = 8
2014-29-08 08:28:45,817.366  - INFO - message4

I want to extract message1, message2, message3 and message4 in the same field "App_message".

I try this but it's not working :

rex "\s-\s+(?<App_level>[A-Z]+)\s*-\s+(?<App_message>.+)(?:ID\s=\s[0-9]+)?"

Help pls 🙂

Re: Rex with optional end of message

aweitzman — Thu, 18 Sep 2014 14:20:56 GMT

Replacing

(?<App_message>.+)

with

(?<App_message>\w+)

should help.

Re: Rex with optional end of message

splunksogetiht — Thu, 18 Sep 2014 14:27:28 GMT

But "App_message" can contain many caracters and not only alphanumeric :
( : ; - \ [ " ' $ ....

Re: Rex with optional end of message

martin_mueller — Thu, 18 Sep 2014 14:27:41 GMT

Assuming the message actually is longer than just a word, you can do this based off your original regex:

 rex "\s-\s+(?<App_level>[A-Z]+)\s*-\s+(?<App_message>.+?)(?:\s*ID\s=\s[0-9]+)?"

All I did was make the .+ non-greedy and added a \s* to the start of the optional end.

Re: Rex with optional end of message

splunksogetiht — Thu, 18 Sep 2014 14:39:26 GMT

The message can be something like :

cCMSyncManager::LCRRerunperARNThread::Send ABDCng Requestfor ARN = F-WWTF, with TransactionID = 1857182350

cCMSyncManager::UpdateDatabaseThreshold::Data is pushed for deleting ARN::F-WWTF, TLM = WIREDand StreamID = 12

I just want extract the message without the "ID= ..." sequence.

I did a mistake in my original post : there is no space between "App_message" and the "ID = ..."

Re: Rex with optional end of message

aweitzman — Thu, 18 Sep 2014 15:02:17 GMT

You should take @martin_mueller's rex, remove the \s he added (since you've since stated it's not really there), and then instead of making the ID part optional with ?, make that clause something that either matches the ID part or the end of the line:

rex "\s-\s+(?<App_level>[A-Z]+)\s*-\s+(?<App_message>.+?)((?:ID\s=\s\d+)|$)"

Re: Rex with optional end of message

splunksogetiht — Thu, 18 Sep 2014 15:07:49 GMT

Ohhhh nice, it's working now ! 🙂

Really thank you !

Re: Rex with optional end of message

martin_mueller — Thu, 18 Sep 2014 17:03:09 GMT

Ah yeah, adding the end-of-line anchor is necessary.