Splunk Search

regex to avoid the 2016-MM-DD events in a splunk search?

pavanae
Builder

The following were some of the events

html tags 2016-04-21 09:42:38,574 DEBUG lksjfd laskdfj lskfj alsdkfj

htmltags2016-05-31T13:50:41.883450Z jhgsd kkjahdf klasjh

htmltags2016-06-11T13:50:41.883450Z kdf ouier lsijcf lkhefr

[ 2016-04-21 09:42:38,574]

abc.def.net 2016-05-31T13

Now, I am trying to write a query which can display the events which shouldn't contain 2016-MM-DD format which is YYYY-MM-DD format

For example it shouldn't display any one of the above mentioned events since they all contains the YYYY-MM-DD format

Note :-
YY- YEAR
MM-MONTH
DD-DATE

Any ideas

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try
Updated to show host/source

your base search | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source 

OR

your base search "*2016-*" | stats count by host, source 

View solution in original post

0 Karma

sundareshr
Legend

Try this

your base search | regex _raw!="\d{4}-\d\d-\d\d"
0 Karma

somesoni2
Revered Legend

Give this a try
Updated to show host/source

your base search | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source 

OR

your base search "*2016-*" | stats count by host, source 
0 Karma

pavanae
Builder

and how to list out the hosts and sources that satisfies my search?

0 Karma

inventsekar
Super Champion

to list out the hosts and sources that satisfies my search -

your search | regex _raw!="\d{4}-\d{2}-\d{2}" | table host source

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

pavanae
Builder

what if we want to display only the windows events. is there any unique search stanza to display only the windows hosts and filtr out the other os's?

0 Karma

somesoni2
Revered Legend

Since both the timestamp and OS info for the hosts are written in totally different logs, you can't achieve this using same base search. My suggestion would be to create a lookup table with all host and their corresponding OS and use that lookup to filter out hosts from above query.

Query to generate lookup

index=_internal source=*metrics.log os=* earliest=-1h@h  | stats latest(os) as os by host | outputlookup host_os.csv 

Use lookup to exclude hosts

your base search [| inputlookup host_os.csv | where os="Windows" | table host ] | regex _raw!=".*2016-\d{2}-\d{2}.*" | stats count by host, source
0 Karma

pavanae
Builder

So without using the lookup's can I below the below search result was accurate?

index=* [ search index=_internal os=Windows sourcetype=splunkd | stats count by hostname | rename hostname as host | fields host ] | regex _raw!=".2016-\d{2}-\d{2}." | regex _raw!=".2016/\d{2}/\d{2}." | stats values(source) as sources values(sourcetype) as sourcetypes by host

0 Karma

somesoni2
Revered Legend

Yes it will be. Lookups will be good for performance as you don't have to go through internal logs every time.

0 Karma

inventsekar
Super Champion
your search | regex _raw!="\d{4}-\d{2}-\d{2}"

will "retain" all YYYY-MM-DD format events and results will be all events other than YYYY-MM-DD format events.

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.

tin_fish
Explorer

Hi Pavanae. I agree with the other answers here, although it's not been made clear whether or not the specific format of your timestamp needs to be part of the exclusion - i.e. should the exclusion encapsulate both YYYY-MM-DD and YYYY-DD-MM?

Also as a general rule - and if possible - it's better to know what you're looking for - rather than what you're not. 🙂

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...