topic Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows? in Splunk Search

Why does search only display 24 hours of event data on Linux, but all-time on Windows?

jmasat — Mon, 08 Jun 2020 17:24:17 GMT

There are approximately 1.5 Billion ingested entries from 40 forwarders.
Performing a search with any criteria on Windows hosts lists all events as all-time.
Performing the same search on Linux hosts only returns 24 hours of data, regardless of time/date ranges supplied. Each day the data only covers the last 24.

What settings could be causing this?

Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows?

richgalloway — Wed, 03 Jun 2020 14:49:10 GMT

Please share your searches.

Have you checked the time window settings? The Windows and Linux servers may have different default values.

Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows?

jmasat — Wed, 03 Jun 2020 22:28:50 GMT

The searches are generic:

host=* (all variations of day, date range) returns one day of Linux and all of the expected windows
Host=linuxhostname (all variations of day,date range) returns one day of Linux regardless of day/date/range
host= windowshostname (all variations of day,date range) returns all data as expected

Where are the "time windows settings"?

Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows?

jmasat — Wed, 03 Jun 2020 22:51:21 GMT

The searches are generic - time/day/range filters applied (60 min, 1 day, 30 days, all-time)
host=* displays all Windows data as expected and returns 1 day of Linux data

host=Linuxhostname returns 1 day of Linux data
host=windowshostname returns all data as expected

"Time window settings" different from MS to Linux?
Where is that setting?

Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows?

richgalloway — Thu, 04 Jun 2020 00:06:56 GMT

Time window perhaps is better known as the time picker. It's a drop-down menu to the right of the search box where you tell Splunk what time range to search. The default setting can be different on each Splunk server. From your comment I know understand you are not running Splunk on mixed platforms.

Are your Windows and Linux data stored in different indexes with different retention periods?

Re: Why does search only display 24 hours of event data on Linux, but all-time on Windows?

jmasat — Thu, 04 Jun 2020 14:09:12 GMT

Could the issue be related to bucket settings (Hot>Warm>Cold>Frozen)

I last ran search @ 1700 last night host=Linuxhostname - returned the 6.2 and 6.3 (24 hours) results (Modifiers were for last 30 days)

I Ran same search @ 0600 It returned 6.3 and 6.4 (Time modifiers set for 30 days)

Could the issue be bucket freeze?

I ran index=_internal sourcetype=splunkd component=BucketMover
and saw 27 moves to cold or freeze--- mostly freeze

Based on the fact that until yesterday- Linux hosts were overrunning indexer with 10,000,000 inputs per 8 hours This was due to Issues with the Linux UNIX addon , which has now been disabled.
/
Question is thawing buckets en-masse advisable?

I have been googling-- but want to not use a "poke and hope" method to thaw.

I have seen different methods- including this.
https://splunkonbigdata.com/2019/02/27/retrieving-data-from-archive-state/

Thoughts please