topic Re: Rename sourcetype not working in Getting Data In

Rename sourcetype not working

tmontney — Wed, 07 Jul 2021 17:43:48 GMT

I want all syslog data to come in as a general sourcetype. If it matches a transforms, it should be changed. Splunk is on 8.0.2.1.

Config files are modified by external script. Confirmed via GUI and splunk cmd btool props list SyslogServer --debug and splunk cmd btool transforms list set_sourcetype_UPS:TrippLite --debug that Splunk is seeing my config. Even after a Splunk restart, the sourcetype is still SyslogServer.

transforms.conf

[set_sourcetype_UPS:TrippLite] REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100 FORMAT = sourcetype::UPS:TrippLite SOURCE_KEY = src_ip DEST_KEY = MetaData:Sourcetype

props.conf

[SyslogServer] CHARSET = UTF-8 DATETIME_CONFIG = FIELD_DELIMITER = | HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg INDEXED_EXTRACTIONS = csv LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = category = Custom pulldown_type = 1 disabled = false REPORT-SyslogServer1 = REPORT-SyslogServer1 TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

inputs.conf (on the syslog server)

[monitor://C:\ProgramData\SyslogServer] disabled = false # whitelist = *.csv recursive = true index = syslog sourcetype = SyslogServer

Re: Rename sourcetype not working

venkatasri — Thu, 08 Jul 2021 02:58:09 GMT

Hi @tmontney

Probably your SOURCE_KEY might require prefix of field:, something like this?

set_sourcetype_UPS:TrippLite] REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100 FORMAT = sourcetype::UPS:TrippLite SOURCE_KEY = field:src_ip DEST_KEY = MetaData:Sourcetype

Re: Rename sourcetype not working

codebuilder — Thu, 08 Jul 2021 15:50:54 GMT

Because you are tagging your data with the SysLogServer sourcetype in inputs.conf that is what will always be used. You can change the rules applied but not the name, given your current configuration.

You would need to use a host or source override in props.conf to accomplish what you're after.

See the props.conf documentation for more details:

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.

Re: Rename sourcetype not working

tmontney — Fri, 09 Jul 2021 14:01:29 GMT

Unfortunately that didn't change anything.

Re: Rename sourcetype not working

tmontney — Fri, 09 Jul 2021 14:07:54 GMT

So this would have to be done on the heavy forwarder, correct?

Re: Rename sourcetype not working

codebuilder — Fri, 09 Jul 2021 15:34:59 GMT

It would be configured on the forwarder monitoring your Syslog server.

Re: Rename sourcetype not working

tmontney — Fri, 09 Jul 2021 17:01:48 GMT

OK, this is what I've applied through an app and it's not working:

Transforms.conf

[set_sourcetype_UPS:AmericanPowerConversionCorp.] REGEX = 192\.168\.0\.101|192\.168\.1\.101|192\.168\.2\.101 FORMAT = sourcetype::UPS:AmericanPowerConversionCorp. DEST_KEY = MetaData:Sourcetype [set_sourcetype_UPS:TrippLite] FORMAT = sourcetype::UPS:TrippLite REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100 DEST_KEY = MetaData:Sourcetype

Props.conf

[sourcetype::SyslogWatcher] TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

Inputs.conf

[monitor://C:\ProgramData\SyslogServer\Export] disabled = false recursive = true index = syslog sourcetype = SyslogServer

Re: Rename sourcetype not working

tmontney — Thu, 05 Aug 2021 19:03:49 GMT

After working it out on a support call, here's what finally worked.

transforms.conf (source/dest key were incorrect)

[set_sourcetype_UPS:TrippLite] REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100 FORMAT = sourcetype::UPS:TrippLite SOURCE_KEY = MetaData:Host DEST_KEY = MetaData:Sourcetype

props.conf (no change)

[UPS:TrippLite] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true category = Custom pulldown_type = 1 [SyslogServer] CHARSET = UTF-8 DATETIME_CONFIG = FIELD_DELIMITER = | HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg INDEXED_EXTRACTIONS = csv LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = category = Custom pulldown_type = 1 disabled = false REPORT-SyslogServer1 = REPORT-SyslogServer1 TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

Config should be in your own app, not system. May apply with an app restart, otherwise restart Splunk.