Splunk IT Service Intelligence

huge waitomo.log file

s2801871r
Explorer

We see ~5GB log file in var\log\splunk folder of our Splunk Insights for Infrastructure installation. It is growing. What is this file? We googled about this file but no information is found.

mattymo
Splunk Employee
Splunk Employee

Looks like a similar bug was marked fixed in 1.2.

waitomo was the project name, I assume it's the log being referenced in this bug, but by it's GA name.

SII-2878 splunk_app_infra.log is not being rotated.

http://docs.splunk.com/Documentation/Infrastructure/1.2.0/ReleaseNotes/Fixedissues

I'd recommend upgrade or just add to logrotate rules on the box.

- MattyMo

s2801871r
Explorer

Great. Thanks for the info. Can you please provide guidance on how to add the logrotation for this file? Which configuration file should be updated? Appreciate it. We will plan for upgrade.

0 Karma

mattymo
Splunk Employee
Splunk Employee

I would probably just stop splunk, archive the log, delete it, then restart Splunk. Then monitor it till upgrade.

Do you have a Splunk support contract?

It would be good to sync with support on this one.

- MattyMo
0 Karma

mattymo
Splunk Employee
Splunk Employee

Hi s2801871r!

Can you provide some info around where/how you deployed (ami? tarball?) and what version you are on?

I believe it could be debug/app logs not being rolled. likely safe to reap to regain the space and add to logrotate.

- MattyMo
0 Karma

s2801871r
Explorer

We are running on Windows server 2012 R2. Following are the SII version details. There are lot of INFO messages in that file. I did not find any config set up related to this file waitomo.log. It looks like WAITOMO is a Splunk project. There is some info I got while googling. Any ideas?

Splunk Insights For Infrastructure

Version ..................................................................................1.1.0
Build ..................................................................................2
Server ..................................................................................4 CPU 8 GB RAM
Splunk TA AWS Version ..................................................................................4.5.0
Splunk Version ..................................................................................7.1.0
Splunk Build ..................................................................................cc33fbdac2cf

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...