How to extract all log events excluding JSON messages?

[nareshinsvu]

Hi,

I want to extract all the log events (normal lines) except JSON messages. There should be an easy way for this. Any hints, please?

My log file is a mix something like below

----------

normal line

normal line

json events {

{json messages}

}

normal line

etc

etc

Thanks,

Naresh

[richgalloway]

If you can produce a regular expression that defines a JSON event then you can use a transform to filter them out.

Put this in a tranforms.conf file

[indexdata]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

[filterjson]
REGEX = <your regex that detects JSON messages>
DEST_KEY = queue
FORMAT = nullQueue

Then add this to the corresponding props.conf file:

[mysourcetype]
TRANSFORMS-nojson = indexdata, filterjson

[nareshinsvu]

Hi @richgalloway 

I am struggling with regex actually.  My regex is only capturing partial json message (until the first "}")

I am trying to search all lines between "line starting with {" and "line starting with }". But ^ is not picking my search

So, I am stuck with this regex currently   --    \{[\s\S]*?\}

{

  {

    {},

  },

}.

[to4kawa]

@nareshinsvu 

Regular expressions require a fairly strict definition. You haven't presented anything here.