Splunk Search

How come empty event information is being displayed?

cschavarro
New Member

I've been seeing some occurrences in Splunk that I haven't been able to find a reason why this is being shown
We use splunk to read our Apache Tomcat console logs from QA and Production env.

Normal log event is shown like this:
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | INFO [Class] Log info message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | DEBUG [Class] Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | WARNING [Class] Log warning message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | ERROR [Class] Log error message

But I have encountered, in great quantities, this kind on log events
INFO | jvm 1 | main | 2018/09/04 00:53:59.734 | 
INFO | jvm 1 | main | 2018/09/04 00:54:59.734 | 

They look like empty log entries, but we don't log 20 empty log entries subsequently. I tried testing this in a local environment with a local splunk server and I notice this kid of message as well, but not int he same amount as in our QA or Production servers.

Does anyone have an idea of what this could mean?

We are using Apache Tomcat 7. (not sure about the build)

Thank you very much.

0 Karma

maciep
Champion

you are going to have to add a LOT more context this question if you want help...no idea what you're asking, what this data is, what you mean by empty event, etc.

0 Karma

cschavarro
New Member

I understand.
This data is a Apache Tomcat console log, we used Splunk to read those files
As I see it, a nomal log appears like this:
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mINFO Log Info message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mDEBUG Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mERROR Log Error message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mWARNING Log Warning message

But I see several occurrences of, as I named it, empty log entries
INFO | jvm 1 | main | 2018/10/05 20:37:39.157 | 
INFO | jvm 1 | main | 2018/10/05 20:37:38.290 | 
INFO | jvm 1 | main | 2018/10/05 20:37:37.455 | 
INFO | jvm 1 | main | 2018/10/05 20:37:36.755 | 
INFO | jvm 1 | main | 2018/10/05 20:37:36.389 | 

I've got no idea what they could mean. I tried a local Splunk installation to read my Apache Tomcat local server and i encountered this kind on "empty log events". I would dare say it is a server issue, because we don't log subsequent empty logs in our application.

I hope this helps a little. Thank you for the remark about not enough information.

0 Karma

althomas
Communicator

Do the messages exist anywhere else in the index? In other words, if you're looking for a specific message, search for that in "All time" so you can see if it's linebreaking correctly. It could be that it is linebreaking due to the numbers later in the message. If this is the case, set up a LINE_BREAKER in props.conf on the indexers.

Best idea, however, is to grab the log file and drop it into your test environment. This way you can see what's happening to the file and how it is being processed at indextime.
(Launcher -> Add Data -> Upload files from my computer)
https://MY_TEST_SERVER:8000/en-GB/manager/search/adddata

0 Karma

cschavarro
New Member

I will take a look if I can get the log files. Thank you!

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...