How to re-ingest data under a different sourcetype?

[dionrivera]

Recently, I ingested data from a windows event log going back 3 years using the XmlWinEventLog sourcetype. Later, I switched the sourcetype to wineventlog which gave me a easier way to extract fields in the events. I deleted the data and the index and re-created it in hopes of re-ingesting all the events using wineventlog going back 3 years. However. Now, I'm only able to ingest the new events flowing from that event log.  And yes, I am using "ALL TIME" as the timeline.

Is there a way to force Splunk to  scrape everything in an event log?

[gcusello]

Hi @dionrivera,

in the forwardr used for ingestion, are you ingesting only wineventlogs or also other logs?

if only wineventlogs, you could clean the _fishbucket on the forwarder, but beware because if you do it, you reindex all!

I usually avoid to use this approach!

In my opinion it's easier to maintain the old sourcetype (for the old data not dor the new ones) and use the TA_Windows extractions that works for you, so you haven't to extract any field.

How do you extract fields? are you using TA_Windows or manually?

Ciao.

Giuseppe

[dionrivera]

@gcusello I am using a separate app to pull the printer logs but I am also collecting windows logs using the same forwarder. I am pulling the logs manually through an app and inputs.conf file. I'll try to delete the fishbucket sub-directory and report back.

[Tom_Lundie · Accepted solution]

Yes, this is possible!

1. Stop Splunk:
   %SPLUNK_HOME%\bin\splunk.exe stop
2. Nagivate to the WinEventLog modinputs directory.
   %SPLUNK_HOME%\var\lib\splunk\modinputs\WinEventLog
3. Delete the files that correspond with the source you want to reset.
4. Restart Splunk
   %SPLUNK_HOME%\bin\splunk.exe start

Don't forget, you'll only be able to re-index whatever your Windows Host is retaining.

[dionrivera]

@Tom_Lundie Thank you. This resolved my issue. I'm seeing most events using the wineventlog sourcetype. Although, I see some gaps in the data from 2018-2022 and with the XmlWinEventLog sourcetype, I was able to go back to 2017. Any ideas? Thank you again.

[Tom_Lundie]

Glad to hear it, you're welcome!

Is there a gap within a specific source on the same host e.g. XmlWinEventLog:Security has a gap on a specific host?

On the face of it, I can't imagine why a particular eventlog would have a large gap unless the machine has been switched off or that particular eventlog has been disabled.

I would compare what you're seeing in Splunk with what you can see in Windows Event Viewer (particularly for the gap). Could it also be that one the eventlogs has got a large retention set and Splunk is catching up?