Can you index evtx on linux heavy forwarder? 4.3

[r999]

Does not make it clear here:

http://splunk-base.splunk.com/answers/141/can-splunk-index-windows-event-logevtevtx-files

Does this absolutely have to be a windows forwarder?

[Kate_Lawrence-G · Accepted solution]

Unfortunately yes - event files are actually binary and Splunk needs to utilize native Windows APIs to extract information from these files, you need to run Splunk on windows.

@Kate

[rhysbee]

Has there been any changes to this since newer releases of Splunk Enterprise 6.5 or 7.0.2?

We use Splunk Enterprise 6.5.0 on Linux for Index cluster and capturing live AD Domain Controller security logs using universal forwarder. For Audit purposes, we want to re-import some periods from archived .evtx files where live capture had failed.

I can successfully Index .evtx files on a standalone Windows Splunk Enterprise 7.0.2 server by setting up a folder monitor and Automatic source type.
When I tried on Windows Splunk Enterprise 6.5.1 server the folder monitor fails to index .evtx files at all.

Is there any way to forward these on to Linux Indexer?
Can this be achieved with the "indexAndForward" attribute in "outputs.conf"?

One option could be to copy bucket folders from Windows Splunk indexer to Linux Splunk indexer. Copy to linux - chown - then run repair, but we would like to automate/simplify the process as much as possible.

Alternately we have also looked at scripts using either logparser.exe / python-evtx / get-winevent and output to .csv/.xml but the formatting is different than existing events. Also logparser is unable to pull the EventRecordID field we are using to validate there are no missing records.

[Kate_Lawrence-G]

Hmm...OK maybe we could tackle this in a different way.

The next things I'd try doing is going back to the universal forwarder on the windows boxens, and then a windows(there seem to be differing comments on whether it will work on linux) indexer with the http://splunk-base.splunk.com/apps/22315/splunk-app-for-windows app loaded?
This might be able to read those EVT files correctly?

It definitely sounds like an API related issue that the forwarder can't parse the files so the indexer is likely rejecting it as binary input. Maybe with the app on the indexer will do the trick.

[r999]

No this didnt work either. there must be some config to tell it to cook the data? the queue = winparsing didint work.

[Chubbybunny]

@bob999: try configuring your Windows forwarder to use an input like so.

[monitor://c:\import_exported_EVT\] 
host_segment = 3 
recursive = true 
queue = winparsing 
crcSalt = <source>

[r999]

thanks. i have tried with full splunk. still the same. do i need to configure anything special in inputs, props or transforms.conf to make sure it cooks the data first... i only have inputs.conf.

do i need to do something different in inputs.conf?

[monitor://$SPLUNK_HOME\evtmon
recursive = false
sourcetype = sevtx
index = indevtx

i tried [WinEventLog://$SPLUNK_HOME\evtmon but nothing gets forwarded

[Kate_Lawrence-G]

well the universal forwarder will just forwarder data and NOT parse it (i.e. run it through the props/transforms actions) so it will not have the api available to parse and the linux indexer will not have the api either...so.

I'd recommend a heavy forwarder (or basically a full splunk instance with the web turned off) on the windows host in this case so that you can parse the data at read time and then forward it over already cooked to the linux indexer.

[r999]

i just tried on a windows universal forwader, forwading to a linux indexer.

the forwarder has read the file and sent to indexer but it has indexed in its binary format:

4:53:58.000 AM

\x00\x1\x00\x4\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+\x2!\x00\x4\x000\x00t'0@\x00\x00\x00\x00......... etc!

Does the indexer have to be windows too!!!?

[Chubbybunny]

additional details:
07-10-2012 11:36:11.180 -0700 INFO TailingProcessor - Ignoring file '/home/Chubbybunny/tmp/Sec_EVT.evt' due to: binary