topic Re: Weird Behavior with Splunk Time Range in Splunk Search

Why is there weird behavior with Splunk Time Range?

zacksoft_wf — Mon, 14 Mar 2022 19:17:58 GMT

I see a strange behaviour in Splunk.

There is this SPL, when ran between 3/13/2022 6:00 AM to 3/14/2011 6:00 AM time range shows some events at 3/13/2022 - 7:00 AM (Between 7-8 AM).

But when I re-run the same SPL between 3/13/2022 6:00 AM to 3/13/2011 8:00 AM , hoping to see the same set of events, But I see ZERO events !!

This is very strange !! Am I missing something simple here..? Why this weird behaviour ?

Additional Observation :
When I change the time range between 2/12 to 3/13 - the events shows,
But when I keep the same date 3/13 7 AM to 3/13 10 AM - It doesn't show.

It works when the time range is more that 24 hours

Re: Weird Behavior with Splunk Time Range

ITWhisperer — Mon, 14 Mar 2022 11:51:39 GMT

It might depend on the actual SPL you are using - please can you provide more detail?

Re: Weird Behavior with Splunk Time Range

zacksoft_wf — Mon, 14 Mar 2022 12:01:40 GMT

index=web_short NOT uco_id=UCOAF NOT uco_id=HRX [ search index=phutan uco_id=PALTO source_zone=isp transport=tcp sourcetype="pan:threat" (source_location="Pacific" OR src_location="Stars Fed") (dest_ip!="179.45.143.47" threat_name!="TVS Vulneribility") severity!="informational" severity!="low" | eval source_ip_type=case( cidrmatch("184.31.77.0/24",source_ip),"UCO_src", true(),"unknown") | where source_ip_type="unknown" | stats count by source_ip | table source_ip | rename source_ip as search | format]

The timestamp of the resulting events are between 7 Am to 8 Am 3/13/2022.
But I don't see the events when I search with in the time rage 3/13/2022 6 Am to 10 Am
I only see when I change the date time range between 3/13 to 3/14 OR 3/12 to 3/13

Re: Weird Behavior with Splunk Time Range

ITWhisperer — Mon, 14 Mar 2022 12:12:30 GMT

So, the subsearch restricts the outer search to ip addresses found in the subsearch during the timeframe.

For example,

Index	web_short	phutan
time	ip	ip
06:30	1.1.1.1
07:30	2.2.2.2
08:30	3.3.3.3	1.1.1.1
09:30		2.2.2.2

if timeframe is restricted to 6am to 8am, ip addresses 1.1.1.1 and 2.2.2.2 are not found in phutan, and are therefore not searched for in web_short, but when the timeframe is wider to at least 9:30, the ip addresses are found and therefore the 6:30 and 7:30 events are found

Re: Weird Behavior with Splunk Time Range

zacksoft_wf — Mon, 14 Mar 2022 12:50:40 GMT

That was brilliant . Thank you @ITWhisperer for such lucid explanation.