How to capture the whole lines where keyword matches using props.conf and transforms.conf

[sureshkumaar]

Feb 3 11:10:15 server-server-server-server systemd[1]: Removed slice User Slice of UID 0.

Feb 3 04:14:23 server-server-server-server rsyslogd[679024]: imjournal: 16021 messages lost due to rate-limiting (20000 allowed within 600 seconds)

Feb 3 11:01:01 server-server-server-server CROND[3905399]: (root) CMDEND (run-parts /etc/cron.hourly)

Feb 3 11:10:55 server-server-server-server esfdaemon[3938104]: 0

Feb 3 10:24:36 server-server-server-server auditd[2689]: Audit daemon rotating log files

Is there a way to capture the whole line where systemd, rsyslogd and auditd keyword matches using props.conf and transforms.conf?

Below captures till the specific keyword, how about remaining lines after the keyword?

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = ^\w{3}\s\s\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2}\s+(?:[+\-A-Z0-9]*\s+)?(systemd|rsyslogd|auditd)
DEST_KEY = queue
FORMAT = indexQueue

[livehybrid]

Hi @sureshkumaar 

Are your events across multiple lines?

You might have more success with the following transform

[setParsing]
INGEST_EVAL = queue=IF(match(_raw, "systemd|rsyslogd|auditd"),queue,"nullQueue")

Then in your props.conf refer to this for your sourcetype

[yourSourcetype]
TRANSFORMS-filter1 = setParsing

This will set the queue depending on a match within the IF statement

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will

[livehybrid]

Hi @sureshkumaar 

The value within the match command is actually a regular expressions (I used a pipedelimited list) so you could update this with a regex to match the filter you are looking for (e.g. hostname space keyword)?

You will only need the single INGEST_EVAL because it uses an IF statement and sets the queue to nullQueue if the match is not met.

Please let me know how you get on and consider accepting this answer or adding karma this answer if it has helped.
Regards

Will

[sureshkumaar]

Hi @livehybrid ,

Thanks for the reply.

I have 2 questions

1. The If condition which is given it will pick the events where ever the keyword matches right being the keyword whether at the start, middle, end of the events "systemd", "rsyslogd" and "auditd"

In my case i am looking for the events to be picked to a sourcetype when those keywords are there after the server name

server-server-server-server systemd

server-server-server-server rsyslogd

2. we need to have below one also right in props.conf to ignore other events getting forwarded to the sourcetype?

[sourcetype]

TRANSFORMS-set = setnull