After looking in the limits.conf specs I must have misread that number, it's actually 100,000 values per field before it truncates. This is much better but still could cause an issue down the road. Can you verify that I'm understanding this correctly? [anomalousvalue] maxresultrows = 50000 # maximum number of distinct values for a field maxvalues = 100000 My manager is sold on the idea that our GUID and JSESSION fields which have tons of values are slowing the searching down if it's put in verbose mode. I guess a good way of testing this would be to search a similar result set with no fields having many values than do another search with a field having many values then compare the search times. ... View more

What kind of data is in your log file? I know PDF files and such are treated as binary ... View more

Cool idea! ... View more

Here's my query which compares tax errors in the previous hour vs yesterdays previous hour and alerts if the errors in the previous hour were 25% higher than the previous hour of yesterday So to answer your question, yes it's possible. You will need to use a subsearch which will look at yesterdays house and your main search will look back the previous hour index=vertex7-access RTG_Error="500" earliest=-1h@h latest=@h | append [| noop | stats count AS RTG_Tax | eval RTG_Tax = "LookupTaxAreas70"] | append [| noop | stats count AS RTG_Tax | eval RTG_Tax = "CalculateTax70"] | stats count AS TodayLastHour by RTG_Tax | eval TodayLastHour = TodayLastHour - if((RTG_Tax = "LookupTaxAreas70"), 1, 0) | eval TodayLastHour = TodayLastHour - if((RTG_Tax = "CalculateTax70"), 1, 0) | addtotals col=t row=f labelfield=RTG_Tax label=Total | appendcols [search index=vertex7-access RTG_Error="500" earliest=-25h@h latest=-24h@h | append [| noop | stats count AS RTG_Tax | eval RTG_Tax = "LookupTaxAreas70"] | append [| noop | stats count AS RTG_Tax | eval RTG_Tax = "CalculateTax70"] | stats count AS YesterdayLastHour by RTG_Tax | eval YesterdayLastHour = YesterdayLastHour - if((RTG_Tax = "LookupTaxAreas70"), 1, 0) | eval YesterdayLastHour = YesterdayLastHour - if((RTG_Tax = "CalculateTax70"), 1, 0) | addtotals col=t row=f labelfield=RTG_Tax label=Total | table RTG_Tax, YesterdayLastHour | rename RTG_Tax AS Total] | rename RTG_Tax AS Total | where TodayLastHour > 1.25 * YesterdayLastHour ... View more

I'm looking for the data sample (Also known as events) which are returned when you run a query. It's impossible to create a regular expression without seeing the patterns in the data sample.. An example would be this 2/19/2016 12:01:00 - User gollam logged in 2/19/2016 12:34:01 - User gollam logged out ... View more

If you provide a data sample.. ... View more

I would suggest creating a field for users who logged in then create another field for users who logged out then do a ...| stats count by To make the fields you will need to find a pattern then write a regular expression to capture this.. Post some a sample and I'll help write your regular expression ... View more

I'm not 100% sure of what your asking here. So you have logs on a server which are in the form of XML and you need those log files available in your Splunk web portal? If so then you will need to set up a forwarder on that server which produces the log files, then configure your output.conf to point to your indexer which parses the data. The indexer will also make it available in your Splunk web interface. Once this XML file is on your indexer you can then parse the XML, line break it, etc.. If your asking how to make each portion of your XML into a sperate event then you will need to write a regular expression and put that in your props.conf .. If this is true then post your current stanza you have in your props.conf and I'll help you correct it ... View more

I'm confused. There appears to be a space before some of the timestamps but in others there is no space before the timestamp, regex wouldn't pick this up as it doesn't fit a pattern. This should work and if not, then I would suggest modifying your log files so it includes a space BREAK_ONLY_BEFORE = \d{4}\-\d{2}\-\d{2}T\d{2}\:\d{2}\:\d{2}\+\d{2}\:\d{2} ... View more

Is there a space between the text and the timestamp in your logs? ... View more

It looks like there is a space before <666> which would break the regular expression I provided. If you wanted to use the ^ and anchor this in, you then need to do ^\s\<666\> OR ^\s\<\d{3}\> .. Regex is super easy to learn, I'd recommend using a site like www.regex101.com to test your regex to verify it works ... View more

Try this.. Make sure you put this stanza in your props.conf on the indexer and restart your Splunk service after adding this [SOURCE] SHOULD_LINEMERGE=true BREAK_ONLY_BEFORE = \d{4}\-\d{2}\-\d{2}T\d{2}\:\d{2}\:\d{2}\+\d{2} ... View more

Create a new app, place your dashboard in that app Go to settings>User Control, select the user you want, then go to the second drop down where it says 'Default App' and select that app you just made ... View more

This looks like the CSV file, can you provide us with the regular expression that is parsing out the fields? ... View more

Can you provide us with your regular expression that is breaking? Are you trying to create a field for each piece of text between the comas? ... View more

I'm pretty sure you CANNOT modify your props.conf from the UI. You will need to edit the .conf file on the indexer ... View more

Do the events have the same timestamp? Try hardcoding the digits in your regular expression to see if that will work. If the below stanza doesn't work then I would suggest hardcoding the host instead of the source and try that. Don't forget to restart the Splunk service, these changes won't work until the service is restarted [SOURCE] TIME_FORMAT = %Y-%m-%d %H:%M:%S,$3N SHOULD_LINEMERGE=true BREAK_ONLY_BEFORE = \<666\> ... View more

Thanks, this is exactly what I'm trying to do, but running into a problem I'm currently on my Splunk indexer and opened up powershell and navigated to Splunk\bin where the python script is and I'm entering the below command. After executing this, I'm getting back "Microsoft Windows Version 6.3 2013 Microsoft Corp. All rights Reversed" cmd python fill_summary_index.py -index "endeca-summary" -et 1449730800 -lt 1452898800 -dedup true -auth admin:xxxxxxx *Summary index = endeca-summary *Regular index = endeca *I used epoch time for -et and -lt.. I want to test this out by running it for 1 hour in the middle of my missing data to confirm this works before attempting to backfill the month ... View more

The speed of processing depends on how many cores you have on the indexer along with how many indexers you have indexing the data. It also depends on if you were consuming that data within a 1 hour time frame or if it was distributed over 24 hours. If the latter case then it works very fast. We currently have 2 clustered indexers with 16 cores each which index around 100-120 GB/day. We have 1 application which has around 5 billion events per month and takes 8 days to process the report on an accelerated data model, so we had to do a workaround and set up a summery index. So my advise is to not to buy tons of hardware for 1-2 applications, there's always a workaround to boost the performance. You also have cloud infrastructure you could leverage if you needed to ... View more