Getting Data In

Log events are missing

anil1432
Explorer

Hello everyone,

When I was trying to search source type=... Xxx and checked from date from 3 /09/2021 to 6 /07/2021 it's showing me millions of records. And again I searched for 1 may 2021 to 25 July 2021 it's showing only 370 events , so  results are not  as we expected, it should be more. Please could you help me. Regarding this issue.

 

Thanks

Labels (1)
0 Karma

PickleRick
Ultra Champion

What are your index settings? Maybe it's simply a case of data retention period expiry - you simply had too much data or the data was too old and oldest events got "pushed out" of the indexes.

0 Karma

anil1432
Explorer

Hello @PickleRick 

 

my index setting is . Index=idx_common_6mon. 

0 Karma

PickleRick
Ultra Champion

No. It's your search. Your index settings are in the spllunk's setting.

BTW, index name suggests 6 months of retention.

0 Karma

anil1432
Explorer

hello @PickleRick 

I was confused,could you  please give brief explanation it will be good to me 

0 Karma

PickleRick
Ultra Champion

In your search parameters you might specify a timerange from which you want your results. That's ok. But an index itself has some limits as to for how long it holds events and how much data it can hold. If the events get too old or if you have too many of them, oldest events will get removed from the index.

You can check from what period and how many events you have by using this search:

| dbinspect where index=idx_common_6mon
| stats min(startEpoch) as earliest max(endEpoch) as latest sum(eventCount) as count sum(rawSize) as size by index splunk_server
| fieldformat earliest=strftime(earliest,"%c")
| fieldformat latest=strftime(latest,"%c")
| eval size=round(size/(1024*1024))

If you want to check your index parameters, check the settings menu->indexes or run

| rest /services/data/indexes
| search title=idx_common_6mon
| fields title frozenTimePeriodInSecs maxDataSize
| stats values(frozenTimePeriodInSecs) as retention values(maxDataSize) as sizelimit by title
| eval retention=round(retention/86400)

 

0 Karma

anil1432
Explorer

Hello @PickleRick ,

Thanks For Sending Those Information , But I Have Tried all Queries  Which You Have Sent me . But The Result is Same . I Didn’t  Find any New . Only 383 Results i Found . Please Let me Know Any other Chances to Check . Please Help Me .

 

Kind Regards,

Anil km

0 Karma

PickleRick
Ultra Champion

Ok, let me put this straight - if your simple search with no additional filters produces no results, the events are simply not there. And there is no way to  produce them out of thin air.

I was merely trying to help you find out why they aren't there. But if you don't care, have it your way.

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

hi @anil1432,

You can check the event count per day is matching in both cases.

sourcetype= sourcetypename | bin span=1d _time | stats count by _time
0 Karma

anil1432
Explorer

Hello @manjunathmeti 

I tried the query which you have sendeD ME BEFORe it showed 370 events and I tried with your query and showing me 380. Only , I think it's not our result

0 Karma
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...

Reminder! Splunk Love Promo: $25 Visa Gift Card for Your Honest SOAR Review With ...

We recently launched our first Splunk Love Special, and it's gone phenomenally well, so we're doing it again, ...