I've looked into this, and unfortunately the only answer (for this version, which works for Firepower 5) is to run the app on multiple forwarders, one forwarder per eStreamer node. The good news is that this wouldn't prevent the events from going into the same index. ... View more

As mentioned in a comment above, it's easy to selectively forward, and it's easy to selectively index. What's more difficult is when you want to selectively not index. If you set queue to nullQueue to try to prevent local indexing, they also won't be forwarded. You can't unset a DEST_KEY (though you can set it to be empty, but the mere presence of _INDEX_AND_FORWARD_ROUTING will result in indexing locally). This means you can't set _INDEX_AND_FORWARD_ROUTING for everything (either in inputs or a default props stanza) and then clear it later. But, you can override (in this case clear) the value of the transform class that would set that value. This is how I was able to get this working as I believe you want. outputs.conf: [indexAndForward] index = true selectiveIndexing = true [tcpout] defaultGroup = nothing [tcpout:externalIDX] server = <forwarder>:9997 props.conf: [default] # unless a more specific stanza clears the value of this class, the transform will be run TRANSFORMS-selectiveIndexing = selectiveIndexing [<sourcetype to forward>] TRANSFORMS-routing = ForwardToEXT # note the empty list of transforms to run in this class, overridden from the [default] TRANSFORMS-selectiveIndexing = transforms.conf: [ForwardToEXT] REGEX = . DEST_KEY = _TCP_ROUTING FORMAT = externalIDX [selectiveIndexing] REGEX = . DEST_KEY = _INDEX_AND_FORWARD_ROUTING # FORMAT doesn't really matter here, the key _INDEX_AND_FORWARD_ROUTING just has to be set FORMAT = local ... View more

Put *batman*error.log in the blacklist for your *error.log stanza. From inputs.conf spec: blacklist = <regular expression> * If set, files from this input are NOT monitored if their path matches the specified regex. ... View more

Try: sourcetype=webaccess clientip!="-" | stats values(url) AS url, dc(url) AS url_dc BY clientip | search url="/test.html" url_dc=1 And I can't imagine the search you included in your question actually does anything. Once you call your first stats command, the only fields you'd have are count and clientip . ... View more

I bet you have a regex that is misbehaving. Did you recently add a search time extraction? If so, what does the regex look like? I've had this happen a few times when a regex wasn't specific enough and would essentially have infinite matches or possible matches. ... View more

You don't need SHOULD_LINEMERGE, just LINE_BREAKER. This should split lines into new events only when they start with a timestamp: props.conf: [<sourcetype>] SHOULD_LINEMERGE = false LINE_BREAKER = ([\n\r]+)\d{2}:\d{2}:\d{2} Oh, and my regex assumes that the leading space on your log lines isn't actually present in the actual data, but an artifact of your copy/paste into answers. ... View more

You have no capture groups defined in your regex, and thus $1 and $2 can't be used. Try this: REGEX = ^(.*)-p [0-9A-Za-z]\w{9,255}(.*?)$ FORMAT = $1-p ########$2 Though it may make sense to use SEDCMD here instead, because you can more easily replace part of a string (instead of rewriting the entire string from components, as you're doing above). In props.conf: [syslog_log_control] SEDCMD-anonymize = s/-p [0-9A-Za-z]\w{9,255}/-p ########/ ... View more