Getting Data In

Log events are missing

anil1432
Explorer

Hello everyone,

When I was trying to search source type=... Xxx and checked from date from 3 /09/2021 to 6 /07/2021 it's showing me millions of records. And again I searched for 1 may 2021 to 25 July 2021 it's showing only 370 events , so  results are not  as we expected, it should be more. Please could you help me. Regarding this issue.

 

Thanks

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

What are your index settings? Maybe it's simply a case of data retention period expiry - you simply had too much data or the data was too old and oldest events got "pushed out" of the indexes.

0 Karma

anil1432
Explorer

Hello @PickleRick 

 

my index setting is . Index=idx_common_6mon. 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

No. It's your search. Your index settings are in the spllunk's setting.

BTW, index name suggests 6 months of retention.

0 Karma

anil1432
Explorer

hello @PickleRick 

I was confused,could you  please give brief explanation it will be good to me 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

In your search parameters you might specify a timerange from which you want your results. That's ok. But an index itself has some limits as to for how long it holds events and how much data it can hold. If the events get too old or if you have too many of them, oldest events will get removed from the index.

You can check from what period and how many events you have by using this search:

| dbinspect where index=idx_common_6mon
| stats min(startEpoch) as earliest max(endEpoch) as latest sum(eventCount) as count sum(rawSize) as size by index splunk_server
| fieldformat earliest=strftime(earliest,"%c")
| fieldformat latest=strftime(latest,"%c")
| eval size=round(size/(1024*1024))

If you want to check your index parameters, check the settings menu->indexes or run

| rest /services/data/indexes
| search title=idx_common_6mon
| fields title frozenTimePeriodInSecs maxDataSize
| stats values(frozenTimePeriodInSecs) as retention values(maxDataSize) as sizelimit by title
| eval retention=round(retention/86400)

 

0 Karma

anil1432
Explorer

Hello @PickleRick ,

Thanks For Sending Those Information , But I Have Tried all Queries  Which You Have Sent me . But The Result is Same . I Didn’t  Find any New . Only 383 Results i Found . Please Let me Know Any other Chances to Check . Please Help Me .

 

Kind Regards,

Anil km

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Ok, let me put this straight - if your simple search with no additional filters produces no results, the events are simply not there. And there is no way to  produce them out of thin air.

I was merely trying to help you find out why they aren't there. But if you don't care, have it your way.

0 Karma

manjunathmeti
Champion

hi @anil1432,

You can check the event count per day is matching in both cases.

sourcetype= sourcetypename | bin span=1d _time | stats count by _time
0 Karma

anil1432
Explorer

Hello @manjunathmeti 

I tried the query which you have sendeD ME BEFORe it showed 370 events and I tried with your query and showing me 380. Only , I think it's not our result

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...