topic Re: Need help in getting wineventlogs to go to a new index rather than the main index in Getting Data In

Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Wed, 30 Sep 2020 01:22:02 GMT

I set up a new index for one of my groups. In it they want to store their servers wineventlogs. I am unable to successfully get the logs to go to the new index. I did set up the inputs.conf file with an index=wineventlog and the index exists. There appear to be some logs showing in the new index, however they are not as full as the ones that go into the main index. I need to get all the logging into the wineventslog index and not put anything into the main index. How can I accomplish this?
My inputs.conf file:

Monitor all EAM-BOUD Windows logs

[WinEventLog://Application]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5
index = wineventlog
renderXml=false

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
blacklist1 = EventCode="4662" Message="Object Type:(?!\s*groupPolicyContainer)"
blacklist2 = EventCode="566" Message="Object Type:(?!\s*groupPolicyContainer)"
index = wineventlog
renderXml=false

[WinEventLog://System]
disabled = 0
start_from = oldest
current_only = 0
checkpointInterval = 5
index = wineventlog
renderXml=false

note that the wineventlog is on all 3 stanzas. I verified that the index does exist in the indexes.conf file. This in production environment so any help is greatly appreciated.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Wed, 30 Sep 2020 01:22:05 GMT

Quick comment: I also sent out the Splunk_TA_Windows along with my new application so I'm thinking that's why some logs are going to the main index, though I noted that an index is not specified.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

harsmarvania57 — Fri, 12 Jul 2019 15:17:31 GMT

With new version of Splunk_TA_windows , there are no index configuration present in inputs.conf so by default everything goes to main index.

As you mentioned that you already configured index=wineventlog, have you restarted Splunk service on Forwarder ? Also double check your configuration using btool command.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

pgerke_cc — Fri, 12 Jul 2019 15:46:48 GMT

Just want the new data going to the wineventlog index or do you also want the already indexed data there?

There appear to be some logs showing in the new index, however they are not as full as the ones that go into the main index.

which logs are going to the new index and which still to the old?

Re: Need help in getting wineventlogs to go to a new index rather than the main index

oscar84x — Fri, 12 Jul 2019 16:56:42 GMT

As mentioned above, run btool to determine what configurations are being applied:

splunk btool inputs list WinEventLog --debug

This will show you if some other default configuration is overriding your inputs.
You could also specify a global Window event log stanza specifying the index as well as specifying it for each individual input. This might override defaults set somewhere else.

[WinEventLog]
index = wineventlog

Re: Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Fri, 12 Jul 2019 18:23:48 GMT

should I run the btool on one of the indexers or on the deployment server? The 2nd note above, should that be in the inputs.conf file?

Re: Need help in getting wineventlogs to go to a new index rather than the main index

oscar84x — Fri, 12 Jul 2019 18:32:13 GMT

You want to run that command where your inputs are located, so in one of your forwarders where the logs are being ingested.

And the stanza I mentioned would go in your inputs.conf.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Mon, 15 Jul 2019 14:28:31 GMT

The sourcetype is: Active Directory. The Application logs are going to the new index.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

woodcock — Mon, 15 Jul 2019 15:54:07 GMT

What version of the Splunk_TA_windows are you using? Be aware that if everything in main should be in wineventlog and right now there is nothing there, you can just shutdown your Indexers and rename the index directory to change its name. But you still have a problem getting new events into wineventlog.... or do you? You do realize that changing this setting will only effect newly forwarded/indexed events and that older events will stay in main, right? Also, you must restart all Splunk instances on your Windows UFs and then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly forwarded/indexed events.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Mon, 15 Jul 2019 17:48:00 GMT

The version is older, I'll download a more current one. I have eliminated the ones going to main, but I have found that those going to my new wineventlog index are not as complete as those that were going to the main index. I will look into a more current TA file for windows and download it to the search servers, then try again with what I have. I did make a change to the local copy of inputs.conf in the TA for windows I commented out all but the 3 types of logs I wanted and removed the index=wineventlog that I had placed in it as a trial to see if they would then be the full logs showing in my preferred new index. Taking it out did stop the main index from ingesting the logs. But I still don't understand why only parts of the logs show in my new index and not the full listing that was in the main index from the TA for windows.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

woodcock — Mon, 15 Jul 2019 19:38:01 GMT

I would not upgrade without reading ALL OF THE DOCS. I am asking what version you have for specific reasons, not encouraging you to upgrade.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

woodcock — Mon, 15 Jul 2019 19:42:24 GMT

If the new events are going to your new wineventlog index but are "not as complete" as those that were going to the old main index, the only thing that makes sense is that you are a victim of the change in default from this:

renderXml=0

To this:

renderXml=1

So you might try changing it back to renderXml=0.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

nls7010 — Fri, 19 Jul 2019 14:37:38 GMT

Thank you it was indeed the default, once I made changes and pushed it out again, the logs came in non-xml format. What I did to get it all the way I needed it and not to add more than I needed was to comment out in the default app what I wouldn't need to come in for this client and in their actual app, I put in the values I did want to come in. Everything seems to be working correctly now.

Re: Need help in getting wineventlogs to go to a new index rather than the main index

woodcock — Fri, 19 Jul 2019 21:34:20 GMT

If you recently upgraded (or are planning to upgrade) the Splunk_TA_windows app, then you might consider using my new Upgrade Planner for Splunk Add-on for Windows app to see if you have any Knowledge Objects that are compatible with the new sourcetypes:

https://splunkbase.splunk.com/app/4594/