hmm, host!=* sound like certain way to receive no events. Essentially this would require that the field host is present in the event (which it is), and that the value does not match any string... would that be some way of saying "is_null"? ... View more

I was more thinking about maxDataSize . maxHotSpanSecs might work too, but I think I would prefer the combination of frozenTimePeriodInSecs and maxDataSize . I have not played around with this extensively, so do not take my advice as Divine Truth. Oh, and the freeze checks are controlled through the rotatePeriodInSecs parameter. /K ... View more

oops, one should not watch TV while editing answers. 🙂 ... View more

Maybe not exactly what you're asking for, but a general optimization tip, especially if there are a large amount of transactions, and only a few of them has errors. Instead of creating transactions and then looking for possible ERRORs, it might be better to do it the other way round: index=blah sourcetype=bleh [index=blah sourcetype=bleh ERROR | dedup RID | fields + RID] | transaction RID The subsearch will find the RIDs where there are ERRORs, and the outer search will build transactions based on them. ... View more

Hi, You know that splunk will only freeze (i.e. delete) a bucket when the newest event in that bucket is older than your retention limit. Unless you have very high volume of traffic, you need to set your bucket size rather small. For example, if your index receives 60 MB of logs per hour, you could set your bucket size to 10 MB. With average compression rates (~50%) you should have about 20 minutes worth of log data per bucket (as long as you only have one hot bucket at a time). Given that you only freeze when the newest event is too old, the oldest events in your index should be 3h20m at any given time. However, working with so small indexes and buckets is not what Splunk was engineered for, and I don't know how often freeze checks are actually made. Also, I think that from a performance perspective, larger buckets (>750 MB) are more efficient, but then again, your data set seems rather small, so that perhaps has less impact in your case. /K ... View more

oops. 2nd part of the dirname, but the 1st epoch. my bad. corrected above already. ... View more

You need to define a frozen path, so that splunk knows where to put them (and not delete them). However, once they are frozen, they are no longer searchable, as only the raw data, and not the tsidx files are retained. You can figure out which of the buckets that are likely to be frozen. This will require that you find the size reduction / bucket size number of buckets and move them manually. In your case that could be (400 GB / 10 GB) ~40 buckets, but you should probably take a few more. Take the 50 buckets with the lowest values of X, where X is the 1st epoch timestamp part of the dirname ... View more

Would something like this work? index=msexchange eventtype=storedriver-mail OR sourcetype=msexchange:2010:mailbox-usage | timechart span=1d dc(UserPrincipalName) as users dc(message_id) as msgs | eval ratio = msgs/users /K ... View more

Please provide some more information; Will a process where a stage fails move on to have ARCHIVED status? Or will it stop with the FAILED message? What is "Report"? Some unique ID? Does the string "ARCHIVED NOT FAILED" exist as such in a single event? Please provide a few sample events. /K ... View more

Am I correct in guessing that you want to extract - 23255909 from the first event - 23244639 from the second event ... View more

Interesting, you learn something new every day! So that means that you @MegSplunk are trying to break the intent of the license agreement that you have, and you're being stopped from doing that. Just quit doing that and download a free version of Splunk, and no one will complain. /k ... View more

Yes, probably something like; primary [secondary | rex FieldA | fields fieldA] | join FieldA [secondary | rex FieldB] | more stuff There might be more elegant solutions. You can do a lot with stats 🙂 ... View more

Without knowing what data you have, it's a bit hard to give good guidance. Have you invesigated any alternative route? transaction, append, multisearch , or even join ? ... View more

Could you post a few samples? And also, indicate at what points the file is currently being broken. ... View more

"Our Splunk license is for a fixed sourcetype". I have never heard of that, could you describe further? The 500 MB limit (or indeed any license limit) is for the combined uncompressed size of the log files that are indexed by the Splunk instance during a 24-hour period (midnight to midnight). Splunk internal logs, which are stored in the _internal index does not count towards the license. So perhaps there are other log files that - combined with your logs - will exceed the total allowed limit. /K ... View more

Slightly rewritten for readability - does this produce the desired results? index=fxr (SUCCESS File successfully uploaded) OR (MAJOR NOT "SFTP Inbound - Zero Byte File Check") /K ... View more

Not sure I've understood completely - but what sourcetype does your events have? Also, can you post a few of them? ... View more

see update above. Should work better... ... View more

yes, well, reverse is not really a nice thing.. ... View more

One way of doing that is to use the following search; sourcetype=your_sourcetype index=your_index | reverse | dedup field01 field02 | eval uniqueString = field01.field02 | timechart span=15m dc(uniqueString) /k UPDATE: Keep reverse before dedup . ... View more

Did you see the documentation for DB Connect? output.timestamp.column = <string> * The column of the result set from which the timestamp is fetched. If this is omitted, the monitor execution time is used as the timestamp value. http://docs.splunk.com/Documentation/DBX/latest/DeployDBX/inputsspec ... View more