topic Re: How can I ingest windows logs that were backed up in Getting Data In

How can I ingest windows logs that were backed up

Funderburg78 — Thu, 08 Jul 2021 19:48:37 GMT

Prior to a customer getting splunk, they 7-zipped there logs and copied them to a server. I just got them a brand new Splunk 8.2.1 enterprise system stood up with awesome dashboards. Now the customer is asking if I can import there archived .evtx files. Do we do this by just putitng them in a folder and using monitor to point at them?

Re: How can I ingest windows logs that were backed up

venkatasri — Fri, 09 Jul 2021 03:25:06 GMT

Hi @Funderburg78

Splunk forwarder can unzip the archive files and ingest them, if unpacked file is non-binary. As you said you have to set-up inputs.conf monitor stanza on UF and give a try.

---

An upvote would be appreciated and Accept solution if this reply helps!

Re: How can I ingest windows logs that were backed up

Funderburg78 — Fri, 09 Jul 2021 11:17:26 GMT

I have unzipped the logs already to a directory C:\EventLogs\Restore. It is literally 50+ evtx files. The monitor://C:\EventLogs\Restore\* does not appear to be working. Normally with Windows logs you need to use the wineventlog:// tool, however, it makes a conenction to a channel to monitor the channel. Since these are not in a channel it will not work. Since they are evtx files I am not sure the monitor:// can actually read it. If they were .csv or .log I am sure monitor:// would work fine. I was wondering if there is something Splunk has to tackle this challenge. I really do not want to mount them all manually and convert them to csv files manually. that would take forever...

EDIT: I cannot be the only person to install splunk and have there customer want to ingest old backed up evtx files....

Re: How can I ingest windows logs that were backed up

venkatasri — Fri, 09 Jul 2021 11:21:41 GMT

@Funderburg78 As i mentioned Splunk can not read binary formats even though enforce it to read they won't be readable. wiki says - "EVTX File Extension - What is an .evtx file and how do I open it? What is an EVTX file? Log file created by the Windows 7 Event Viewer; contains a list of events recorded by Windows; saved in a proprietary binary format that can only be viewed within the Event Viewer program. "

Hence you shall export them to CSV or other text formats then ingest using UF.

---

An upvote would appreciated and Accept solution if this reply helps!

Re: How can I ingest windows logs that were backed up

Funderburg78 — Fri, 09 Jul 2021 12:00:18 GMT

Looks like I need to use a powershell script or logparser to do the conversion. I found some stuff on the internet to help with this for those who may have this issue in the future!

Convert evtx Files:

Powershell Try:

$a = Get-Item *.evtx

$output_file = [System.IO.StreamWriter] $("all.csv")

foreach($file in $a){

$events = get-winevent -path $file.FullName

foreach ($Event in $events) {

$xml = [xml]($Event.ToXml())

foreach ($s in $xml.Event.System.ChildNodes) {

$output_file.Write($s.Name + ":" + $s.InnerText + ",")

}

foreach ($d in $xml.Event.EventData.Data) {

$text = $d.InnerText

$text = if ($text) { $text.replace("`n","") } else { $text }

$output_file.Write($d.Name + ":" + $text + ",")

}

$output_file.WriteLine()

}

$output_file.Flush()

$output_file.Close()

LogParser: (Download from Microsoft.com)

$logparser = "c:\program files (x86)\Log Parser 2.2\logparser.exe"$query = "SELECT * INTO c:\logs\logs.csv FROM c:\logs\logs.evtx" & $logparser -i:evt -o:csv $query

I have not actually tried this yet, but will be giving it a shot in the near future, I expect some tweaking to the scripting may need to be required.

EDIT: Links for reference:

https://serverfault.com/questions/783708/convert-saved-evtx-files-to-text

LogParser Download location:

https://www.microsoft.com/en-us/download/details.aspx?id=24659

Re: How can I ingest windows logs that were backed up

venkatasri — Fri, 09 Jul 2021 12:00:03 GMT

Yes you need to automate it otherwise its hard. All the best.