Hello
I want to index the events in the firewalls log based in the alert level and the virtual domain in witch they have been generated. I have followed the guide in https://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Routeandfilterdatad#Perform_selective_...
but the indexer is indexing all the events not only the events that match the regular expression. I think that the setnull stanza may not be working. But I am not sure how to fix it
-This is the contain in props.conf
[source::/opt/LOGs/firewalls]
TRANSFORMS-set= setnull,setindexone,setindextwo,setindexthree,setindexfour
-And this is the contains in transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setindexone]
REGEX = level="(error|critical|alert|emergency)".*(vd="one")
DEST_KEY = queue
FORMAT = indexQueue
[setindextwo]
REGEX = level="(critical|alert|emergency)".*(vd="two")
DEST_KEY = queue
FORMAT = indexQueue
[setindexthree]
REGEX = level="(critical|alert|emergency)".*(vd="three")
DEST_KEY = queue
FORMAT = indexQueue
[setindexfour]
REGEX = level="(alert|emergency)".*(vd="four")
DEST_KEY = queue
FORMAT = indexQueue
-Also, I have checked with the command "btool --app=Splunk_TA_fortinet_fortigate transforms list and props list" that the .conf files configuration is being loaded.
Thanks
Ok, now it is working. I have added the props.conf and tranforms.conf to the forwarder (/opt/splunk/etc/system/local/) and it is working.
For future similar issues: How can I check if the input data is being already parsed by the forwarder before reaching the indexer? As I have read if the indexer receives the data already parsed by the forwarders it will not transform and filter it and will be directly indexed
Thanks for your help.
If that forwarder is a universal forwarder as you claimed, it doesn't make too much sense that this fixes it...
@yosoypako If your problem is resolved, please accept the answer to help future readers.
Hello.
If i use the btool command (splunk.exe cmd btool props list fgt_utm --debug) I can see that the setnull stanza that is being chosen is this one, with REGEX = .
Thanks.
How can you see that from a btool on props?
Can you run a btool on transforms and grep it for setnull, to see if there are conflicting setnull stanzas?
Are you trying to send events to diff indexes based on the values of vd? If so, you would need DEST_KEY = _MetaData:Index and FORMAT = yourindexname
Hello
I want to send all the logs from the different vd to the same index but I want to filter witch alert level messages are going to be indexed from each vd. So in one index I could index only the alert and emergency level but in other vd the critical, alert nad emergency levels.
I have not written this in the initial post but there are two different splunk machine in this set up an indexer and a universal forwarder. I am editing the props.conf and transforms.conf files only on the indexer, not on the forwarder. How can I confirm that the forwarder is not parsing any event?
Thanks.
If there is only a UF and an indexer involved, no heavy forwarder, then the indexer should indeed be the right place for this config.
"setnull" is a very generic name though. Any chance you have another transforms stanza named that way, that happens to take precedence and has a more specific REGEX setting, that doesn't match your firewall data?
Hello
I have tried to use the sourcetype in the props.conf file. But it is still not working.
-If I use the btool *cmd btool props list * it shows:
TRANSFORMS =
TRANSFORMS-set = setnull,setindexone,setindextwo,setindexthree,setindexfour
-And If I use the btool command with both the app and the sourcetype it shows:
TRANSFORMS-set = setnull,setindexone,setindextwo,setindexthree,setindexfour
Any ideas why it is not filtering the events
Regards.