Splunk Search
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: is it possible to run remove duplicates on spe...
Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
gooza
Communicator
07-19-2011
12:07 AM
When using the remove duplicate event python is it possible to run it on specific date range?
1 Solution
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
zpavic
Path Finder
07-19-2011
10:42 AM
I create new script for you. http://pastebin.com/JusgWRMy
Usage: ./splunk cmd python rem_dupl_event_spec_date.py <index> <earliest_date> <latest_date>
earliest_date & latest_date syntax: %m/%d/%Y:%H:%M:%S
For example:
./splunk cmd python rem_dupl_event_spec_date.py test 05/20/2011:0:0:0 05/24/2011:23:59:59
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
zpavic
Path Finder
07-19-2011
10:42 AM
I create new script for you. http://pastebin.com/JusgWRMy
Usage: ./splunk cmd python rem_dupl_event_spec_date.py <index> <earliest_date> <latest_date>
earliest_date & latest_date syntax: %m/%d/%Y:%H:%M:%S
For example:
./splunk cmd python rem_dupl_event_spec_date.py test 05/20/2011:0:0:0 05/24/2011:23:59:59
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
karthy
Explorer
11-16-2012
05:10 AM
Hmm... I believe this script it a bit dangerous. What happens, if you have 3 events (one unique and two, which is duplicates) within the same second? Then the script will delete one of the duplicates AND the single unique event, right?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
kphillipson
Path Finder
11-07-2012
12:54 PM
Really like this. Is there a way for it to only look at a source or sourcetype within a given index?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
gooza
Communicator
07-19-2011
11:59 PM
works great , Thanks!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
gooza
Communicator
07-19-2011
12:24 AM
Found it..
Edited the script and added to line 56 (starting with search1) earliest= latest dates :
"search1 = 'search index=' + index + 'earliest=129461760 latest=1294704000 | ..."
I don't know Perl , anyone knows how to add these dates as parameters ?
Get Updates on the Splunk Community!
Aligning Observability Costs with Business Value: Practical Strategies
Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...
Mastering Data Pipelines: Unlocking Value with Splunk
In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...
Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0
Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...
