I have added a monitor stanza for the log folder which contains log files that I want to ingest into Splunk.
I have set sourcetype for each log file in props.conf but in some Splunk version(like 7.3.3, 8.0.0, 8.0.1) it is not working and Splunk set sourcetype for those log files to one of the following:
1) log_file_name-too_small
2) log_file_name-{digit}(like log_file_name-2, log_file_name-4)
I have read some answers like this is happening because of the small size of the log file, but this is not an issue for some Splunk version(like 8.0.4), this is happening for windows and Linux both(mostly with windows).
I have tried the below approaches in props.conf but none of them seem to be working.
1) [source::.../etc/apps/<app_name>/local/logs/log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
2) [source::...*etc*apps*<app_name>*local*logs*log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
3) [source::....*etc.*apps.*<app_name>.*local.*logs.*log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
4) [source::...(.)*etc(.)*apps(.)*<app_name>(.)*local(.)*logs(.)*log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
5) [source::...(.*)etc(.*)apps(.*)<app_name>(.*)local(.*)logs(.*)log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
6) [source::...\\etc\\apps\\<app_name>\\local\\logs\\log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
7) [source::...\etc\apps\<app_name>\local\logs\log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
😎 [source::C:\\Program Files\\Splunk\\etc\\apps\\<app_name>\\local\\logs\\log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
9) [source::C:\Program Files\Splunk\etc\apps\<app_name>\local\logs\log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
I have tried all of this because I thought in windows it might be an issue with path separator but none of them are working, but I got one solution that is working and giving right sourcetype which is like this.
In props.conf:
[source::...log_file.log(.\d+)?]
sourcetype = <sourcetype-name>
but I don't wont to rely upon this approach because it is possible that the same log file name is present in some other apps so it may get into that way and also this approach is time-consuming as it will going to find the file in all folders.
I have tried to solve this in another way like providing sourcetype stanza of "log_file_name-too_small" sourcetype and changing sourcetype with help of transform.conf file, it is working for "log_file_name-too_small" as below.
In props.conf:
[log_file_name-too_small]
TRANSFORMS-remove_too_small_sourcetype = remove_too_small
In transform.conf:
[remove_too_small]
DEST_KEY = MetaData:Sourcetype
REGEX = .*
FORMAT = sourcetype::<sourcetype-name>
but as I mentioned above that sourcetype value might be "log_file_name-{digit}" so I need to do solve this the same way as above (like specifying [log_file_name-2]) but I think this is not the right way as the value of digit may be anything, so I tried regex (log_file_name-*) in sourcetype stanza of props.conf but it is not working maybe because sourcetype stanza does not allow regex.
It would be great if anyone able to solve this problem.
Regards.
... View more
