Splunk Search

How come empty event information is being displayed?

cschavarro
New Member

I've been seeing some occurrences in Splunk that I haven't been able to find a reason why this is being shown
We use splunk to read our Apache Tomcat console logs from QA and Production env.

Normal log event is shown like this:
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | INFO [Class] Log info message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | DEBUG [Class] Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | WARNING [Class] Log warning message
INFO | jvm 1 | main | 2018/10/05 20:37:30.549 | ERROR [Class] Log error message

But I have encountered, in great quantities, this kind on log events
INFO | jvm 1 | main | 2018/09/04 00:53:59.734 | 
INFO | jvm 1 | main | 2018/09/04 00:54:59.734 | 

They look like empty log entries, but we don't log 20 empty log entries subsequently. I tried testing this in a local environment with a local splunk server and I notice this kid of message as well, but not int he same amount as in our QA or Production servers.

Does anyone have an idea of what this could mean?

We are using Apache Tomcat 7. (not sure about the build)

Thank you very much.

0 Karma

maciep
Champion

you are going to have to add a LOT more context this question if you want help...no idea what you're asking, what this data is, what you mean by empty event, etc.

0 Karma

cschavarro
New Member

I understand.
This data is a Apache Tomcat console log, we used Splunk to read those files
As I see it, a nomal log appears like this:
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mINFO Log Info message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mDEBUG Log debug message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mERROR Log Error message
INFO | jvm 1 | main | 2018/10/05 20:37:46.463 | [m[0;32mWARNING Log Warning message

But I see several occurrences of, as I named it, empty log entries
INFO | jvm 1 | main | 2018/10/05 20:37:39.157 | 
INFO | jvm 1 | main | 2018/10/05 20:37:38.290 | 
INFO | jvm 1 | main | 2018/10/05 20:37:37.455 | 
INFO | jvm 1 | main | 2018/10/05 20:37:36.755 | 
INFO | jvm 1 | main | 2018/10/05 20:37:36.389 | 

I've got no idea what they could mean. I tried a local Splunk installation to read my Apache Tomcat local server and i encountered this kind on "empty log events". I would dare say it is a server issue, because we don't log subsequent empty logs in our application.

I hope this helps a little. Thank you for the remark about not enough information.

0 Karma

althomas
Communicator

Do the messages exist anywhere else in the index? In other words, if you're looking for a specific message, search for that in "All time" so you can see if it's linebreaking correctly. It could be that it is linebreaking due to the numbers later in the message. If this is the case, set up a LINE_BREAKER in props.conf on the indexers.

Best idea, however, is to grab the log file and drop it into your test environment. This way you can see what's happening to the file and how it is being processed at indextime.
(Launcher -> Add Data -> Upload files from my computer)
https://MY_TEST_SERVER:8000/en-GB/manager/search/adddata

0 Karma

cschavarro
New Member

I will take a look if I can get the log files. Thank you!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...