In our distributed enterprise Splunk environment we have a log file being generated on each Splunk host (indexers, search head, deployment server, etc) located at: /opt/splunk/var/log/splunk/foo.log
By default this gets logged to _internal using the foo-too_small source type.
We now want to change the source type to one we created (my:custom:sourcetype). I have created the following props.conf file on the deployment server as a custom app and deployed successfully via apply cluster-bundle. However, new log data is still being associated with the existing source type of foo-too_small. We also set the local.meta file (under metadata) for permissions.
I have verified this file is making it to the indexers in peer-apps.
[my:custom:sourcetype]
TIME_FORMAT = %Y-%m-%d %H:%M:%S,%3N
MAX_TIMESTAMP_LOOKAHEAD = 25
[source::.../var/log/splunk/foo.log]
sourcetype = my:custom:sourcetype
Questions:
Thank you in advance!
Answering my own question here:
Answering my own question here:
Hi @tretrigh ,
good for you, see next time!
Ciao and happy splunking
Giuseppe
P.S.: Karma Points are appreciated by all the contributors 😉
Hi @tretrigh,
check if in your forwarders, there's some input without sourcetype definition.
You can do it analyzing host and source values.
Assign the correct sourcetype to all your inputs.
ciao.
Giuseppe
Thank you for the reply. Do you have any specific guidance on how to apply the correct source type to our data in our situation?
Hi @tretrigh,
sometimes (I don't know why) there's some situation when in the add-on isn't defined the sourcetype, so Splunk automatuically assign the sourcetype based on its knowledge and sometimes it cannot find the correct one.
So analyze your logs where there's a too small sourcetype, find the Add-On with that input and manually assign the correct one in the add-on.
Ciao.
Giuseppe
Thanks for the reply @gcusello . In this situation there is no add on. The log file on each Splunk host is generated by a script we wrote. We have attempted to manually define the source type for this specific log unsuccessfully. Do you have any suggestions for how to correctly manually define the source type other than what we've already done? Thank you for the assistance!
Every Splunk input should have a sourcetype assigned to it.
Every sourcetype needs to be defined in a props.conf file.
Every props.conf stanza should have the "Great Eight" attributes, at a minimum.
[mysourcetype]
TIME_PREFIX =
TIME_FORMAT =
MAX_TIMESTAMP_LOOKAHEAD =
TRUNCATE =
SHOULD_LINEMERGE = false
LINE_BREAKER =
EVENT_BREAKER =
EVENT_BREAKER_ENABLE = true
Set values for each attribute that correspond to the data being ingested.
Thank you for the reply. I might be missing something obvious, but unsure how any of these settings might help us reassign the source type to something else. Could you please provide further elaboration? Thank you!