I have accepted that it will always have a risk of occurring and that there is no fix.
Depending on how you're using the data this might not be effective but this is sloppy pseudo code on how we handled it for now;
when passing the data into javascript function
set variable counter = 0
if variable counter == 0 then
use the data however you want
counter++
Now at least it won't loop.
... View more
There are many ways to do a lot of things in Splunk. This should work but that rex will be attempting to extract from all events, including both indexes. If you can get ips extracted through props/transforms then that would be much better. Give it a shot and provide feedback.
... View more
Try out the following as for regex101.com
(?<drive>\w)\:\\(?<first>[\w]+)\\(?<second>[\w]+)\\(?<third>[\w]+)\\(?<filename>[\d\w\.]+)
Try this out in splunk;
| rex field=string "(?<drive>\w)\:\\\(?<first>[\w]+)\\\(?<second>[\w]+)\\\(?<third>[\w]+)\\\(?<filename>[\d\w\.]+)"
This will parse the entire path that you listed.
... View more
Is index foo is your data good ips?
Is index doo is your known bad ips?
Are you regexing your good ips?
If you combine both sets of data with the OR statement then you'll have all data in one searchable bucket. Then dedup by index and ip so that you have an ip from each index to compare. Then compare those two by counting by ip (if each index contains an ip then the count will be 2).
(index=foo source=foo) OR (index=doo sourcetype="aaf" NOT ip="NULL)
| rex field=_raw "(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
| dedup index ip
| stats count by ip
| where count > 1
Each result in this search means that it matches your good data (your indexed data) and is also on the known bad list.
... View more
To troubleshoot, do the following (this works but just to explain the process);
| makeresults count=1
| eval source="D:\example\report-2015-05-05.csv"
| rex field=source mode=sed "s/\\\/\\\\\\\/g"
You'll see the result you desired, hopefully, I do.. But once your regex is confirmed then put it in your search and carry on.
I tried using the regex provided above and didn't see the result you wanted so it most likely wouldn't work even with your search provided.
... View more
For how much data is written on index, you could view it through the gui by going to Settings > Indexes and viewing the summary of the index. If you need more granular let me know but I don't have access to verify searches right now.
This link provides a search which uses the _audit index to view what users are doing. Again, I can't verify right now but if you follow the advice you should be able to get retrieve all the events which include search queries. You would then need to search for "index=test" within those results and do a stats count. Please take a look and report back.
https://answers.splunk.com/answers/149332/how-to-view-the-list-of-search-queries-run-for-a-given-time.html
... View more
Copy.
If you want to provide more information we can attempt to work further into making sure it works correctly otherwise I'm glad you were able to make it work and if satisfied, please mark answered 🙂
Have a good day!
... View more
Adding context for anyone coming here for answers.
Assuming msg is the field name, the following would result in the first IP found
| makeresults
| eval msg = "192.128.22.2 202.134.55.89"
| rex field=msg "(?<firstip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
If additional IPs needed to be extracted then a max_match = # could be added to the rex line.
... View more
Assuming you have a table reflecting a column of http codes along with a column of count (as "count"), try the following adding the following to your search query;
| eventstats sum(count) as total
| eval code_per = round((count / total)*100,2)."%"
| fields - total
... View more
You can navigate to the Monitoring Console and view indexes with amount of data over time. It uses "index=_internal source=license_usage.log type=Usage" by default.
If you're searching "index=test source=license_usage.log type=Usage" then you will not be able to find license_usage.log because they are in index=_internal.
... View more
Is this still an expected behavior? I notice this issue quite frequently when dealing with a savedsearch and a postprocess vice a search.
If this is still expected behavior, are there any workarounds to guaranteeing only one set of a data? I am creating divs based on the data returned and my workaround is to run a function to delete all divs when it starts otherwise I end up with 2-4 times as many sets of data.
... View more