skipped indexing of internal audit event will keep dropping events until indexer congestion is remedied. Check disk space and other issues that may cause indexer to block

[nocsnetwork]

Using 4.3.4 on Windoze XP
No forwarding, no scheduled searches, no apps, minimal input to indexes all pushed to splunk.
Only use the GUI, no idea how to do diag or what a bucket is, so previous answers mean nothing to me.
Please help, but with basic instructions.
Thanks

[Drainy]

I'm sorry but if you're running a technical product like Splunk then you can't just discount all previous answers as being too technical. Read up a bit and learn how to manage and administer Splunk so you do understand in better detail.

In regards to both your problems, do you start and stop it when needed? Indexing a large number of Windows events can take a fair bit of time and cause congestion on a slow disc, I expect that is the case here. Also if you have any level of auditing enabled on Windows this can also cause a fair bit of noise.
156meg (if i've read that right) isn't particularly big for an internal index, this stores all internal splunk related logs. Have you tried searching _internal for the word error in case there are any system problems?

Jobs expiring or getting cancelled could be a sign of high IO or CPU usage, install the Splunk on Splunk (SoS) app to get a better insight into the performance of your system

[Drainy]

well, from; but why is _internal so big and what is it? I assumed you didn't know what it was. What are you currently indexing on the hosts? And how much space is free on C: atm?

[nocsnetwork]

I've worked out for myself how to search _internal files - I do RTFM. I still have no idea what they are indexing, but this error seems to appear every 5 minutes or so in it.
ERROR databasePartitionPolicy - Still throttling, indexing paused waiting for optimize for _internal. Check to see if the disk is nearly full, as this situation may prevent splunk-optimize from running, causing perpetual throttling.
I am not knowingly indexing windows events - does splunk do this by default? If it does, ho do I stop it? The PC only runs Splunk, so if CPU is high, Splunk is the cause.

[mhdali]

Hi,
It also happened to me,
Did you fix this issue

Mhd-Ali

[nocsnetwork]

I'm now getting this message...
The running job "rt_1353419849.11" was canceled remotely or expired (but for multiple jobs)
I see from another solution, this could be a clock error, but my times are correct, or to do with .lock files. Could this relate to my large _internal size?

[nocsnetwork]

I don't know if it helps, but i've analysed the indexes from the GUI and this is what was displayed.

series sum(MB)

_internal 156.101745613
main 1.4854517016
_audit 0.176021572

I'm guessing main is our syslog data, but why is _internal so big and what is it? Can we reduce it's size or what it is accumulating?

[nocsnetwork]

OOps, The software is on the C: partition.
Restarted splunk again and this is the output...
splunk start

Splunk> All batbelt. No tights.

Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking configuration... Done.
Checking index directory...
Validated databases: _audit _blocksignature _internal _thefishbucket his
tory main summary
Done
Success
Checking conf files for typos...
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Splunkd: Starting (pid 4576)

[nocsnetwork]

Hi
The hard drive is partitioned.
The splunk partition (F:) contains all things splunk.
According to Windoze it is 54.9GB
I have used 654MB - this includes the software, indexes, everything.
As I said, we have very low usage!

Restarting doesn't seem to have made any difference.

[lguinn2]

How much space remains on the hard drive where you installed Splunk? Is this the same hard drive where you are storing the indexes?

[kphillipson]

How large is the Hard drive you installed Splunk on? Did restarting the pc or service have any affect?