Getting Data In

Rename sourcetype not working

tmontney
Builder

I want all syslog data to come in as a general sourcetype. If it matches a transforms, it should be changed. Splunk is on 8.0.2.1.

Config files are modified by external script. Confirmed via GUI and splunk cmd btool props list SyslogServer --debug and splunk cmd btool transforms list set_sourcetype_UPS:TrippLite --debug that Splunk is seeing my config. Even after a Splunk restart, the sourcetype is still SyslogServer. 

transforms.conf

 

 

[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = src_ip
DEST_KEY = MetaData:Sourcetype

 

 

props.conf

 

 

[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG = 
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = 
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

 

 

inputs.conf (on the syslog server)

 

 

[monitor://C:\ProgramData\SyslogServer]
disabled = false
# whitelist = *.csv
recursive = true
index = syslog
sourcetype = SyslogServer

 

 

 

Labels (2)
0 Karma
1 Solution

tmontney
Builder

After working it out on a support call, here's what finally worked.

transforms.conf (source/dest key were incorrect)

 

[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = MetaData:Host
DEST_KEY = MetaData:Sourcetype

 

props.conf (no change)

 

[UPS:TrippLite]
DATETIME_CONFIG = 
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1

[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG = 
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = 
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

 

 

Config should be in your own app, not system. May apply with an app restart, otherwise restart Splunk.

View solution in original post

0 Karma

tmontney
Builder

After working it out on a support call, here's what finally worked.

transforms.conf (source/dest key were incorrect)

 

[set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = MetaData:Host
DEST_KEY = MetaData:Sourcetype

 

props.conf (no change)

 

[UPS:TrippLite]
DATETIME_CONFIG = 
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1

[SyslogServer]
CHARSET = UTF-8
DATETIME_CONFIG = 
FIELD_DELIMITER = |
HEADER_FIELD_LINE_NUMBER = timeWritten,src_ip,facility,severity,timeGenerated,msg_tag,msg_origin,msg
INDEXED_EXTRACTIONS = csv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = 
category = Custom
pulldown_type = 1
disabled = false
REPORT-SyslogServer1 = REPORT-SyslogServer1
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

 

 

Config should be in your own app, not system. May apply with an app restart, otherwise restart Splunk.

0 Karma

codebuilder
SplunkTrust
SplunkTrust

Because you are tagging your data with the SysLogServer sourcetype in inputs.conf that is what will always be used. You can change the rules applied but not the name, given your current configuration.

You would need to use a host or source override in props.conf to accomplish what you're after.

See the props.conf documentation for more details:

<spec> can be:
1. <sourcetype>, the source type of an event.
2. host::<host>, where <host> is the host, or host-matching pattern, for an
                 event.
3. source::<source>, where <source> is the source, or source-matching
                     pattern, for an event.

 

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

tmontney
Builder

So this would have to be done on the heavy forwarder, correct?

0 Karma

codebuilder
SplunkTrust
SplunkTrust

It would be configured on the forwarder monitoring your Syslog server.

----
An upvote would be appreciated and Accept Solution if it helps!

tmontney
Builder

OK, this is what I've applied through an app and it's not working:

Transforms.conf

[set_sourcetype_UPS:AmericanPowerConversionCorp.]
REGEX = 192\.168\.0\.101|192\.168\.1\.101|192\.168\.2\.101
FORMAT = sourcetype::UPS:AmericanPowerConversionCorp.
DEST_KEY = MetaData:Sourcetype

[set_sourcetype_UPS:TrippLite]
FORMAT = sourcetype::UPS:TrippLite
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
DEST_KEY = MetaData:Sourcetype

Props.conf

[sourcetype::SyslogWatcher]
TRANSFORMS-changesourcetype = set_sourcetype_UPS:AmericanPowerConversionCorp.,set_sourcetype_UPS:TrippLite

 Inputs.conf

[monitor://C:\ProgramData\SyslogServer\Export]
disabled = false
recursive = true
index = syslog
sourcetype = SyslogServer
0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @tmontney 

Probably your SOURCE_KEY might require prefix of field:, something like this?

set_sourcetype_UPS:TrippLite]
REGEX = 192\.168\.0\.100|192\.168\.1\.100|192\.168\.2\.100
FORMAT = sourcetype::UPS:TrippLite
SOURCE_KEY = field:src_ip
DEST_KEY = MetaData:Sourcetype

 

0 Karma

tmontney
Builder

Unfortunately that didn't change anything.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...