I want all syslog data to come in as a general sourcetype. If it matches a transforms, it should be changed. Splunk is on 8.0.2.1.
Config files are modified by external script. Confirmed via GUI and splunk cmd btool props list SyslogServer --debug and splunk cmd btool transforms list set_sourcetype_UPS:TrippLite --debug that Splunk is seeing my config. Even after a Splunk restart, the sourcetype is still SyslogServer.
transforms.conf
[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = src_ip
DEST_KEY = MetaData:Sourcetype
props.conf
[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG =
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK =
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite
inputs.conf (on the syslog server)
[monitor://C:\ProgramData\SyslogServer]
disabled = false
# whitelist = *.csv
recursive = true
index = syslog
sourcetype = SyslogServer
After working it out on a support call, here's what finally worked.
transforms.conf (source/dest key were incorrect)
[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = MetaData:Host
DEST_KEY = MetaData:Sourcetype
props.conf (no change)
[UPS:TrippLite]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1
[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG =
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK =
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite
Config should be in your own app, not system. May apply with an app restart, otherwise restart Splunk.
After working it out on a support call, here's what finally worked.
transforms.conf (source/dest key were incorrect)
[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = MetaData:Host
DEST_KEY = MetaData:Sourcetype
props.conf (no change)
[UPS:TrippLite]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1
[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG =
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK =
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite
Config should be in your own app, not system. May apply with an app restart, otherwise restart Splunk.
Because you are tagging your data with the SysLogServer sourcetype in inputs.conf that is what will always be used. You can change the rules applied but not the name, given your current configuration.
You would need to use a host or source override in props.conf to accomplish what you're after.
See the props.conf documentation for more details:
<spec> can be: 1. <sourcetype>, the source type of an event. 2. host::<host>, where <host> is the host, or host-matching pattern, for an event. 3. source::<source>, where <source> is the source, or source-matching pattern, for an event.
So this would have to be done on the heavy forwarder, correct?
It would be configured on the forwarder monitoring your Syslog server.
OK, this is what I've applied through an app and it's not working:
Transforms.conf
[set_sourcetype_UPS:AmericanPowerConversionCorp.]
REGEX = 192\.168\.0\.101|192\.168\.1\.101|192\.168\.2\.101
FORMAT = sourcetype::UPS:AmericanPowerConversionCorp.
DEST_KEY = MetaData:Sourcetype
[set_sourcetype_UPS:TrippLite]
FORMAT = sourcetype::UPS:TrippLite
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
DEST_KEY = MetaData:Sourcetype
Props.conf
[sourcetype::SyslogWatcher]
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite
Inputs.conf
[monitor://C:\ProgramData\SyslogServer\Export]
disabled = false
recursive = true
index = syslog
sourcetype = SyslogServer
Hi @tmontney
Probably your SOURCE_KEY might require prefix of field:, something like this?
set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = field:src_ip
DEST_KEY = MetaData:Sourcetype
Unfortunately that didn't change anything.