I need some help 🙂
scheme: 3 Universal Forwarders -> collecting/forwarding -> Indexer
Changed every UF host (windows:applications and services logs) from
I added a tcp listener in: Manager -> Forwarding and receiving -> Configure receiving
[default] host = splunk.domain.local [script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path] disabled = 0 [WinEventLog:Application] disabled = 1 [WinEventLog:ForwardedEvents] disabled = 1 [WinEventLog:HardwareEvents] disabled = 1 [WinEventLog:Internet Explorer] disabled = 1 [WinEventLog:Security] disabled = 1 [WinEventLog:Setup] disabled = 1 [WinEventLog:System] disabled = 1
[host::*.domain.local] TZ = GMT+4 TRANSFORMS-set= setnull,setdbls,kix_exclude_dbls
[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setdbls] REGEX = (?msi)^EventType=(1|2) DEST_KEY = _MetaData:Index FORMAT = db_ls [kix_exclude_dbls] REGEX = (?msi)^EventCode=(1722|1332|53).+ComputerName=E[1-5]TS1 DEST_KEY = queue FORMAT = nullQueue
If I comment [setnull] block, all works fine. But logs, which are not EventType=(1|2), will be collected in the default index. If I enable the [setnull] block, ALL logs will be removed. However, I want to put [setdbls] in the "db_ls" index and remove the others.
check your props with btool:
$SPLUNK_HOME/bin/splunk cmd btool props list
also keep in mind each change in props and/or transforms needs a reload. this can be done with this search command on the fly:
| extract reload=T
here are some sources which are useful in this case:
hope this helps, cheers - MuS
There error here seems to be a mixup of configurations and concepts (nullQueueing and index-time transformation in general). Considering your props.conf settings;
[your host, source or sourcetype] TRANSFORMS-blah= setnull, setdbls, kix_exclude_dbls
will take each event of the host/source/sourcetype through the three transforms.
First the destination queue will be set to the nullQueue for all events and the index will be
main, unless you have specified a different index in inputs.conf.
Second, if the regex matches in
[setdbls] the destination index will be set to
db_ls, but the destination queue will still be nullQueue. Thus all events will be deleted.
The third transform will not make a difference.
If you comment out the first transform
[setnull], no events will have the nullQueue set, and events will flow into the
db_ls index (when the REGEX matches).
To achieve the desired results I would suggest that you set the following;
inputs.conf (where the files are read / scripts are executed
[monitor / script / WinEventLog:blah blah blah] disabled = 0 index=db_ls
props.conf (on the indexer)
[host / source / sourcetype] TRANSFORMS-blah_null = setnull, setdbls, kix_exclude
transforms.conf (on the indexer)
[setdbls] change to
That way the correct index will be set from the start, and the transformations will only deal with the queues.
Hope this helps,
Thank you, Kristian Kolb! Very informative answer 🙂
Maybe it can be useful for someone:
I changed my aim from wmi::applications/system to wmi::security. Let it be as an example.
After Kristian's kick to the right way 🙂 I removed from "Manager -> Forwarding and receiving -> Configure receiving" all tcp receivers. Then I set only one in inputs.conf (indexer side).
[splunktcp://10997] disabled = 0
As Damien Dallimore (thx too) said in that post even if you install a simple uf, you will be able to change the index on the forwarder side. This is only the one right way.
inputs.conf (universal forwarder side)
[default] index = db_ls
Now all logs stream to the right index::db_ls. Then I removed any old rules from props.conf and transforms.conf (both on indexer side). Set the new rules for incoming wmi::security traffic:
props.conf (indexer side)
[WinEventLog:Security] priority = 5 TRANSFORMS-wmisecr=setnull,setsecrdbls
Then I described actions for these rules in transforms.conf :: set the nullQueue for all default queue for the index db_ls -> set the indexQueue only for REGEX pattern:
transforms.conf (indexer side)
[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setsecrdbls] REGEX = (?msi)^EventCode=(528|538|529) DEST_KEY = queue FORMAT = indexQueue
Now all logs're forwarded from universal forwarder (some node) to the Indexer, to the right index and right queue. In the queue I can filter unnecessary events and leave only important.
Nevertheless, can you point me out to the article where I can read about queues?
Short comment first: you should set your index=blahblah for each
[script:xxxx]) in inputs.conf. Having it under
[default] will work, but if you have more than one input, and want them in separate indexes, you'll want to do this.
I recommend that you always set
sourcetype for each input separately.
Here are a few links to information regarding queues;
In practice you'll probably only use nullQueue and indexQueue in your configurations. Other queues like typingQueue and aggQueue will only reveal themselves when there is a problem, like with blocked queues.