Your estimation of 5.7T is pretty close. I'd include a little extra room just in case and let your frozen time period take care of trimming the index. maxTotalDataSizeMB is the right one to use here as you don't seem to be using volume management on this index. ... View more

I like option 4. Your num_cpus is way too high for a 16 core system. That setting is per-machine, not per-cluster. You're simply running too many searches than your cluster can handle at once. ... View more

Hey jedatt01, I'm good friends with @skoelpin and couldn't help but snoop. @skoelpin, Splunk's immutable way of indexing kind of breaks this use case by knocking the D off CRUD. There are ways of hiding data in an index but I think jedatt01 is on the right track with his idea to use the KV store. I've seen it done for case management systems within Splunk using the KV store. @jedatt01, if you're looking for a short answer to is it possible, then yes. How is it different? in my experience when I think of Splunk's KV store, the word "fragile" comes to mind. Be sure to back it up often. ... View more

Something like index=foo sourcetype=bar ssh | stats count by date_hour ... View more

Why not just start with rising column? If you run it as a batch, it will run again on your configured interval and re-index all the records again if you do not disable the input before the 2nd run. Max rows to retrieve limits the total amount of records pulled for a batch input, the default is 0. Fetch size is how many records dbconnect will pull per pass to keep from requesting 26 million rows at once. ... View more

Can you share your search string? How much data is this searching and for what size time window? ... View more

"Additionally, the only way to get the sourcetype stanza to kick in would then be to have an input that does not match any source:: stanza but which is assigned that sourcetype manually. Doesn't sound right." - Yes Re - multiple matching categories, this is where precedence kicks in. They are not all applied, they are overridden as you only get one pass through props.conf, meaning you cannot use props.conf to set a sourcetype from a source:: spec and expect that same data to be evaluated again at index time via the sourcetype stanza because there is no second pass through the parsing phase. **[<spec>] stanza precedence:** For settings that are specified in multiple categories of matching [<spec>] stanzas, [host::<host>] settings override [<sourcetype>] settings. Additionally, [source::<source>] settings override both [host::<host>] and [<sourcetype>] settings. ... View more

You will want to configure proper line breaking on this input by specifying the following parameters for this in props.conf on your indexer. For this file it should look something like... [source::C:\Windows\Logs\Powershell] SHOULD_LINEMERGE = false LINE_BREAKER = (End time:\s+\d+)([\r\n]+) TIME_FORMAT = %Y%m%d%H%M%S TIME_PREFIX = ^Start Time: MAX_TIMESTAMP_LOOKAHEAD = 14 ... View more

No they cannot be chained, You can modify the sourcetype later on in transforms and you can apply more than one transform to a props stanza by listing them out comma separated. When it comes to props, you can only match once. The example where [source::.../syslog(.\d+)?] implicitly matches any source where the file ends in a number. Those sources will take only this path through the parsing phase. Anything else already having the sourcetype of syslog via inputs will match the other stanza. ... View more

No, you can only go through the parsing phase once. The first will apply and the second will never match anything. ... View more

source:: has precedence over sourcetype so the sourcetype stanza would be ignored. https://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Attributeprecedencewithinafile ... View more

A batch input can be used to delete a file after it is forwarded but this should not be used for streaming logs. It sounds like you'll need to set up log rotate to tackle this issue. ... View more

These are written using various HTTP calls to online threat lists. You could populate any index with collect or lookup with outputlookup ... View more

This doesn't mean the alerts are slow it means this is the amount of time the alerts have been activated due to high memory usage, so in this case, 30 - 50 seconds of high memory usage per search head. ... View more

index=wholesale_app CustomAnalytic Properties.index=30 OR Properties.index=21 buildTarget= product= |rename Properties.args as properties|stats min(_time) AS earliest max(_time) AS latest by clientSessionId | eval duration=latest-earliest, _time=latest | timechart avg(duration) as adur |eval adur=round(adur/60,2) ... View more

I think you missed a part. Be sure to pass _time through with this | eval duration=latest-earliest, _time=latest | timechart avg(duration) as adur ... View more

The maximum length of an event is governed by your TRUNCATE setting in props.conf (default is 10,000 bytes) and can also be manipulated by other parameters. The maximum length of a tag is not documented as far as I can tell but a quick test tells me its 1024 characters. ... View more

90th percentile is what I usually use. Max is pretty misleading ... View more

A search bundle is basically a copy of your search head configurations which are replicated to your indexer. If you have a lot of apps or some that contain large files, it can sometimes take awhile to replicate. ... View more

Whichever vendor you purchase Splunk Cloud from may sign a BAA but Splunk will not directly if I remember correctly. ... View more

try wrapping your token in single quotes in your case statement, case('$env:user$' == "admin" ... View more

Have you thought about simply renaming the index and turning on data integrity control? After that you should be able to regenerate the hashes. If this isn't an option for you, you'll have to replay the historical data from your forwarders by clearing the fishbucket once pointed to the new index, if that data it still exists. https://answers.splunk.com/answers/28134/rename-an-index-in-4-1-8.html https://docs.splunk.com/Documentation/Splunk/7.0.1/Security/Dataintegritycontrol#Regenerate_hashes ... View more