Getting Data In

IndexScopedSearch deleting bad events?

jalfrey
Communicator

I'm getting the following error:
Error in 'IndexScopedSearch': The search failed. More than 1000000 events found at time 1390985456.

I found this existing question:
http://answers.splunk.com/answers/76392/how-to-delete-a-huge-number-of-old-events-from-the-test-data...
as well as this one
http://answers.splunk.com/answers/119467/indexscopedsearch-error-while-searching
as well as this one
http://answers.splunk.com/answers/3397/indexscopedsearch-error-details

I tried removing the db directories and restarting splunk. Still getting same error. I tried this command:
_time="1389689456" | delete

Does anyone know how to remove the bad data?

0 Karma

wrangler2x
Motivator

This may have nothing to do with indexed data corruption. If you had something that could generate 1,000 log events in a second, and if your timestamp is accurate only to the second, then all of those log entries would have the same epoch time (which is given in the error message for the frame of time where > 1,000,000 events have been seen). The epoch time is accurate to the millisecond, so in a single second you have 1,000 epoch times but only one is being used for all the events begin logged in any given second. Windows iis logs are by default accurate to the second. But you can google for your version and see about turning on millisecond time stamp. Whatever logs you are using, that's what I'd try.

If this is being caused by corrupted data, there is quite a bit of information about dealing with that here: http://wiki.splunk.com/Community:PostCrashFsckRepair

0 Karma

wrangler2x
Motivator
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...