topic Re: Events not appearing in All Apps and Add-ons

Events not appearing

chriswong187 — Fri, 04 Oct 2019 14:00:26 GMT

I configured the sourcetype to fml:log according to the details but no events appear.

When the sourcetype is changed to syslog events start to appear.

Switching back to fml:log stops events again. Any ideas?

Re: Events not appearing

prakash007 — Fri, 04 Oct 2019 15:51:24 GMT

@chriswong187 looks like you have to make some tweaks to bypass syslog(in built splunk sourcetype) parsing according to your requirements..

Check this out....
https://www.splunk.com/blog/2008/04/16/overriding-default-syslog-host-extraction.html

Check your props/transforms with btool..

$SPLUNK_HOME/bin/splunk cmd btool props list --debug syslog
$SPLUNK_HOME/bin/splunk cmd btool props list --debug fml:log

Re: Events not appearing

chriswong187 — Mon, 07 Oct 2019 14:13:23 GMT

Thanks for pointing me to the article. I took some steps to correct everything based on the article but it didn't seem to work, unless i overlooked something. Here's some info on what I found:

PS C:\Program Files\Splunk\bin> splunk cmd btool props list --debug syslog
C:\Program Files\Splunk\etc\system\default\props.conf            TRANSFORMS = syslog-host

PS C:\Program Files\Splunk\bin> splunk cmd btool props list --debug fml:log
C:\Program Files\Splunk\etc\apps\Splunk_TA_fortimail\local\props.conf   TRANSFORMS =

I modified the fortimail props.conf to include this:

TRANSFORMS = fortimail

And included this in the transforms.conf:

[fortimail]
DEST_KEY = MetaData:Host
REGEX = .
FORMAT = host::host.fqdn host::host2.fqdn

Re: Events not appearing

prakash007 — Mon, 07 Oct 2019 14:17:46 GMT

which splunk instance you have this configs deployed..?? heavy forwarder or indexer..??

Re: Events not appearing

chriswong187 — Mon, 07 Oct 2019 14:31:22 GMT

Indexer only, I have a single server deployment.

Re: Events not appearing

pruthvikrishnap — Mon, 07 Oct 2019 20:59:34 GMT

Hi Chris,
one thing we can do is to go to Splunk UI

Settings > Sourcetypes > select syslog > advanced settings
Copy the settings and create a new sourcetype with the same settings and then use it.
It should have it resolved.

Re: Events not appearing

chriswong187 — Wed, 30 Sep 2020 02:31:27 GMT

Hi all, after going through the splunkd.log and I saw a few entries that indicated the LINE_BREAKER entry in props.conf wasn't working right.

LineBreakingProcessor - Truncating line because limit of 99999 bytes has been exceeded with a line length >= 100063

I tried a few variations of LINE_BREAKER in props.conf and ultimately got it to work. I ended up changing

LINE_BREAKER = ([\n\r]+)date=\d{4}\-\d{2}\-\d{2}

LINE_BREAKER = ([\n\r]+)

I also reverted the previous changes I had tried with TRANSFORMS back to their default values and things still seemed to work with this new LINE_BREAKER. ¯\(ツ)/¯

Thanks to all that replied with suggestions.

*edit: code blocks

Re: Events not appearing

woodcock — Sun, 13 Oct 2019 06:30:17 GMT

Come back and click Accept on your answer to close your question.