topic Re: How come empty event information is being displayed? in Splunk Search

How come empty event information is being displayed?

cschavarro — Fri, 05 Oct 2018 16:24:49 GMT

I've been seeing some occurrences in Splunk that I haven't been able to find a reason why this is being shown
We use splunk to read our Apache Tomcat console logs from QA and Production env.

Normal log event is shown like this:
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | [m[0;32mINFO [Class] Log info message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | [m[0;32mDEBUG [Class] Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | [m[0;32mWARNING [Class] Log warning message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | [m[0;32mERROR [Class] Log error message

But I have encountered, in great quantities, this kind on log events
INFO | jvm 1 | main | 2018/09/04 00:53:59.734 | [m
INFO | jvm 1 | main | 2018/09/04 00:54:59.734 | [m

They look like empty log entries, but we don't log 20 empty log entries subsequently. I tried testing this in a local environment with a local splunk server and I notice this kid of message as well, but not int he same amount as in our QA or Production servers.

Does anyone have an idea of what this could mean?

We are using Apache Tomcat 7. (not sure about the build)

Thank you very much.

Re: How come empty event information is being displayed?

maciep — Sun, 07 Oct 2018 13:14:25 GMT

you are going to have to add a LOT more context this question if you want help...no idea what you're asking, what this data is, what you mean by empty event, etc.

Re: How come empty event information is being displayed?

cschavarro — Mon, 08 Oct 2018 14:57:10 GMT

I understand.
This data is a Apache Tomcat console log, we used Splunk to read those files
As I see it, a nomal log appears like this:
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mINFO Log Info message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mDEBUG Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mERROR Log Error message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mWARNING Log Warning message

But I see several occurrences of, as I named it, empty log entries
INFO | jvm 1 | main | 2018/10/05 20:37:39.157 | [m
INFO | jvm 1 | main | 2018/10/05 20:37:38.290 | [m
INFO | jvm 1 | main | 2018/10/05 20:37:37.455 | [m
INFO | jvm 1 | main | 2018/10/05 20:37:36.755 | [m
INFO | jvm 1 | main | 2018/10/05 20:37:36.389 | [m

I've got no idea what they could mean. I tried a local Splunk installation to read my Apache Tomcat local server and i encountered this kind on "empty log events". I would dare say it is a server issue, because we don't log subsequent empty logs in our application.

I hope this helps a little. Thank you for the remark about not enough information.

Re: How come empty event information is being displayed?

althomas — Mon, 08 Oct 2018 15:27:01 GMT

Do the messages exist anywhere else in the index? In other words, if you're looking for a specific message, search for that in "All time" so you can see if it's linebreaking correctly. It could be that it is linebreaking due to the numbers later in the message. If this is the case, set up a LINE_BREAKER in props.conf on the indexers.

Best idea, however, is to grab the log file and drop it into your test environment. This way you can see what's happening to the file and how it is being processed at indextime.
(Launcher -> Add Data -> Upload files from my computer)
https://MY_TEST_SERVER:8000/en-GB/manager/search/adddata

Re: How come empty event information is being displayed?

cschavarro — Mon, 08 Oct 2018 16:13:41 GMT

I will take a look if I can get the log files. Thank you!