Ignore unlike rows

[Mkaz]

I have a log that a software package provides which creates a standard record for each event.

The standard format is:

Wed Oct 26 10:41:14 2016 0 10.40.112.27 437434 /dirlevel1/dirlevel2/dirlevel3/dirlevel4/chr26104109.txt b s o r aaa_aaaaaaa ssh 0 *

We also have customer scripts that write to this log in a similar, but different format. These entries are few and would like to ignore these records if possible. The records that I would like to keep and report on always ends in an *. All other records can be ignored. Is it possible to ignore the record in the query statement or at at best, do a different query based on the last character o the record?

Query is:

index="ti_is_st" sourcetype="xfer_log" | rex field=_raw "^(\S+\s+){8}\/(([^\s\/]+\/)+)(?<fileName>.+)(\s+\S+){8}$" |rex field=File_Status "(?<File_Status>(i|j|k|o|p|q))\s"|search "$field2$" "$field3$" | streamstats count as Row | table Row _time ip_address Service_Account fileName File_Size File_Status |replace o with "Download Successful" i with "Upload Successful" j with "Upload Errored" k with "Upload Aborted" p with "Download Errored" q with "Download Aborted" in File_Status

Thanks,

[niketn]

Can you see if following filter in the base search of your query helps?

"* \*"

[somesoni2 · Accepted solution]

Give this a try (to include on the rows which ends with an asterisk)

index="ti_is_st" sourcetype="xfer_log" | regex _raw="^.+\*$" | rex field=_raw "^(\S+\s+){8}\/(([^\s\/]+\/)+)(?<fileName>.+)(\s+\S+){8}$" |rex field=File_Status "(?<File_Status>(i|j|k|o|p|q))\s"|search "$field2$" "$field3$" | streamstats count as Row | table Row _time ip_address Service_Account fileName File_Size File_Status |replace o with "Download Successful" i with "Upload Successful" j with "Upload Errored" k with "Upload Aborted" p with "Download Errored" q with "Download Aborted" in File_Status

[Mkaz]

Works! Thank you