Getting Data In

How to view status of indexing of evtx files in a folder

rockb
Explorer

I am using Splunk to review logs from disconnected systems.  We have the users export the evtx files and send them to us.  I then put them in a folder and Splunk indexes the new files.  

Is there an easy way to see the indexing process?  Right now I just keep hitting refresh occasionally until nothing changes.

Labels (1)
0 Karma

Azeemering
Builder

The easiest way to check if a file has finished being ingested is to check the _internal index like this:

 

index=_internal sourcetype=splunkd ArchiveProcessor yourfilename.evtx

 

The order of internal logging is

1. Handling FIle
2. Reading Path
3. Processing xxxx
4. Finished processing file

It will contains entries like this. You will see the message Finished processing file when the file has been ingested completely. (I used a file called sectest.evtx)

10-05-2021 20:53:52.975 +0200 INFO ArchiveProcessor [17620 archivereader] - Finished processing file 'H:\evtx\sectest.evtx', removing from stats
10-05-2021 20:53:52.975 +0200 INFO ArchiveProcessor [17620 archivereader] - Processing windows event log file H:\evtx\sectest.evtx
10-05-2021 20:53:52.975 +0200 INFO ArchiveProcessor [17620 archivereader] - reading path=H:\evtx\sectest.evtx (seek=0 len=21041152)
10-05-2021 20:53:52.973 +0200 INFO ArchiveProcessor [17620 archivereader] - Handling file=H:\evtx\sectest.evtx

Obviously you can set an alert, or some type of message when you see that message come by.

 

0 Karma

rockb
Explorer

Thank you for the reply. 

 

There are 18 evtx files in a single folder.  Is there a way to get status of indexing of the entire folder?

0 Karma

Azeemering
Builder

It will follow the same principal as the previous example.

For each individual file it will go through that process. So you could create a saved search with a stats count command that count the number of "Finished processing file" messages.
Try and be creative and use Splunk _internal event logging to your own advantage.

Another brainfart might be to send an alert when you are not seeing "Finished processing file" anymore in the _internal logging. This might also be an indicator that all files have been processed.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Introduction to Splunk AI

How are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. Lucky for ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Maximizing the Value of Splunk ES 8.x

Splunk Enterprise Security (ES) continues to be a leader in the Gartner Magic Quadrant, reflecting its pivotal ...