Getting Data In

IndexScopedSearch deleting bad events?

jalfrey
Communicator

I'm getting the following error:
Error in 'IndexScopedSearch': The search failed. More than 1000000 events found at time 1390985456.

I found this existing question:
http://answers.splunk.com/answers/76392/how-to-delete-a-huge-number-of-old-events-from-the-test-data...
as well as this one
http://answers.splunk.com/answers/119467/indexscopedsearch-error-while-searching
as well as this one
http://answers.splunk.com/answers/3397/indexscopedsearch-error-details

I tried removing the db directories and restarting splunk. Still getting same error. I tried this command:
_time="1389689456" | delete

Does anyone know how to remove the bad data?

0 Karma

wrangler2x
Motivator

This may have nothing to do with indexed data corruption. If you had something that could generate 1,000 log events in a second, and if your timestamp is accurate only to the second, then all of those log entries would have the same epoch time (which is given in the error message for the frame of time where > 1,000,000 events have been seen). The epoch time is accurate to the millisecond, so in a single second you have 1,000 epoch times but only one is being used for all the events begin logged in any given second. Windows iis logs are by default accurate to the second. But you can google for your version and see about turning on millisecond time stamp. Whatever logs you are using, that's what I'd try.

If this is being caused by corrupted data, there is quite a bit of information about dealing with that here: http://wiki.splunk.com/Community:PostCrashFsckRepair

0 Karma

wrangler2x
Motivator
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automated Threat Analysis: Available in ES Premier

Automated Threat Analysis: Centralize and Accelerate Phishing Investigations in Splunk Enterprise ...

What’s New in Splunk AI: Volume 02

Welcome to the second edition of “What’s New in Splunk AI” where we look at the latest and greatest updates, ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...