Splunk Search

regex to avoid the 2016-MM-DD events in a splunk search?

Builder

The following were some of the events

html tags 2016-04-21 09:42:38,574 DEBUG lksjfd laskdfj lskfj alsdkfj

htmltags2016-05-31T13:50:41.883450Z jhgsd kkjahdf klasjh

htmltags2016-06-11T13:50:41.883450Z kdf ouier lsijcf lkhefr

[ 2016-04-21 09:42:38,574]

abc.def.net 2016-05-31T13

Now, I am trying to write a query which can display the events which shouldn't contain 2016-MM-DD format which is YYYY-MM-DD format

For example it shouldn't display any one of the above mentioned events since they all contains the YYYY-MM-DD format

Note :-
YY- YEAR
MM-MONTH
DD-DATE

Any ideas

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Give this a try
Updated to show host/source

your base search | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source 

OR

your base search "*2016-*" | stats count by host, source 

View solution in original post

0 Karma

Legend

Try this

your base search | regex _raw!="\d{4}-\d\d-\d\d"
0 Karma

SplunkTrust
SplunkTrust

Give this a try
Updated to show host/source

your base search | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source 

OR

your base search "*2016-*" | stats count by host, source 

View solution in original post

0 Karma

Builder

and how to list out the hosts and sources that satisfies my search?

0 Karma

Champion

to list out the hosts and sources that satisfies my search -

your search | regex _raw!="\d{4}-\d{2}-\d{2}" | table host source

0 Karma

Builder

what if we want to display only the windows events. is there any unique search stanza to display only the windows hosts and filtr out the other os's?

0 Karma

SplunkTrust
SplunkTrust

Since both the timestamp and OS info for the hosts are written in totally different logs, you can't achieve this using same base search. My suggestion would be to create a lookup table with all host and their corresponding OS and use that lookup to filter out hosts from above query.

Query to generate lookup

index=_internal source=*metrics.log os=* earliest=-1h@h  | stats latest(os) as os by host | outputlookup host_os.csv 

Use lookup to exclude hosts

your base search [| inputlookup host_os.csv | where os="Windows" | table host ] | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source
0 Karma

Builder

So without using the lookup's can I below the below search result was accurate?

index=* [ search index=_internal os=Windows sourcetype=splunkd | stats count by hostname | rename hostname as host | fields host ] | regex _raw!=".2016-\d{2}-\d{2}." | regex _raw!=".2016/\d{2}/\d{2}." | stats values(source) as sources values(sourcetype) as sourcetypes by host

0 Karma

SplunkTrust
SplunkTrust

Yes it will be. Lookups will be good for performance as you don't have to go through internal logs every time.

0 Karma

Champion
your search | regex _raw!="\d{4}-\d{2}-\d{2}"

will "retain" all YYYY-MM-DD format events and results will be all events other than YYYY-MM-DD format events.

Explorer

Hi Pavanae. I agree with the other answers here, although it's not been made clear whether or not the specific format of your timestamp needs to be part of the exclusion - i.e. should the exclusion encapsulate both YYYY-MM-DD and YYYY-DD-MM?

Also as a general rule - and if possible - it's better to know what you're looking for - rather than what you're not. 🙂

0 Karma