I have a dedicated server which is running syslog-ng and a universal forwarder.
i want to set 3 things one of them dynamically:
# /opt/splunkforwarder/etc/system/local/inputs.conf [monitor:///data/syslog-ng/logs/u514/cisco/ios/*/*.log] sourcetype = syslog source = syslog-ng:udp514 host_segment = 7
The problem is that i cannot set source and hostsegment (or hostregex) at the same time.
Because the hostsegment uses (why ever on earth i don't know) the source string.
Host segment defines the "7" position as the host variable.
So if i define the source by myselft hostsegment will fail.
Is it possibile to have a manually created source field and a dynamically generated host field?
I could do this by creating a new props.conf and transforms.conf to manipulate the source segment. But i do not want this to be generally done... There are a few logs for which i do not want that.
In the case of a monitor stanza, the source, by definition, is the name of the file it came from.
If changing the order of those lines in the monitor stanza doesn't work, then move the alteration of the source to a transforms stanza and use a regex to identify which ones do or do not qualify to become the new source.
That would look something like this. (Try this in a sandbox first, because it's all air code.):
[monitor:///data/syslog-ng/logs/u514/cisco/ios/*/*.log] sourcetype = syslog host_segment = 7
[syslog] transforms-newsource1 = somestanza
[somestanza] SOURCE_KEY = however you are deciding which ones to change REGEX = however you are deciding which ones to change DEST_KEY = MetaData:Source FORMAT = source::syslog-ng:udp514
You can do really tricky stuff, but you have to do it in the exact right order. Here's an old example.
I thought about your idea before and this could work if there wouldn't be a problem with all other stuff which has the sourcetype [syslog].
As far as i know there is no possibility to divide INPUT Data in props.conf by two keys - sourcetype match AND host match. The only possibility would be to use the source in props.conf and manipulate via transform.conf.
If i want to use the source i have to specify all relevant information in the file path. For example:
/data/syslog/syslog-ng/u514/cisco/ios/ciscoswitch/20200417-cisco-ios.log and the take the information out of it that i want.
After thinking about those possibilities i decided to let go of this requirement.
I changed the folder path to include all important information and live with the fact that there is no easy solution for that. To manipulate 3 confs to achive this goal could backfire in a year or two.
Thanks for your suggestion - i check it as answer cause it is for all what i know the only possibilty to achive this. BR Michele
That's probably a good decision. In essence, if you wanted to do that, you could build a synthetic field in multiple steps, but you are leaving land mines for a later admin... who might very well be yourself.
More importantly, overriding the "source" is not perhaps the best strategy.
Consider adding another index-time field that contains what you want: a tag, or another named field, or whatever. Think about the overhead, and make your best guess.
If you only added it to the ones that you wanted to change, the overhead cost might be limited.