What index should sysmon data go into and how /where to change the index?

[packet_hunter]

I have successfully installed sysmon and verified the schemaversion matches the schemaversion in the config file (sysmonconfig-export.xml by SwiftonSecurity). I have confirmed that sysmon is running in event viewer (Application and Service Logs > Microsoft > Windows > Sysmon > Operational).

I downloaded and installed the TA-microsoft-sysmon on the search head I use.
I also copied the TA-sysmon folder to the deployment server (\Splunk\etc\deployment-apps\TA-microsoft-sysmon) and then deployed it to my UF running on my test host.

I ran my handy query

|tstats values(sourcetype) WHERE index=* by index

and noticed the data was rolling into the default main index...

How do I change the index to winsysmon ? or does anyone have a better idea which index the sysmon data should go in?

Thank you

[dstaulcu · Accepted solution]

It would be more (computationally) efficient to define the desired on index on the endpoints via index = winsysmon spec in inputs.conf than it would be to transform/reroute the events on the indexers via props/transforms.conf. The indexers are going to busy enough extracting XML fields at search time for that dense sysmon data set.

[packet_hunter]

Please convert your comment to an answer...

[dstaulcu]

done & thank you

[packet_hunter]

Thank you dstaulcu.
Your comments confirm what I was thinking and what other team members have done.

1) Put the new index on the indexers (in indexes.conf)
2) Put the new index in the inputs.conf - we don't edit default so I create a new inputs.conf in local of the deployment app.

Please convert your comment to an answer.

[richgalloway]

Add index=winsysmon to the appropriate stanza in your props.conf file.

[damode]

After I did the above step, I got this message during Splunk restart,

Invalid key in stanza [XmlWinEventLog:Microsoft-Windows-Sysmon/Operational] in c:\Program Files\Splunk\etc\apps\TA-microsoft-sysmon\local\props.conf, line 33: index (value: winsysmon).

[dstaulcu]

what does line 33 of props.conf say?

[damode]

Props.conf
line 33- index =winsysmon

[dstaulcu]

that spec does not belong in props.conf. It belongs in inputs.conf.

[damode]

Thanks for clarifying.

[packet_hunter]

Admittedly, I don't have experience creating new indexes in this scenario.

I was thinking that I had to define the new index on the indexers (not clustered) first and then define the index in a local file to the app I want to deploy...

Would your solution automatically create the index on the indexers too?

[packet_hunter]

here is the beginning of the props.conf (default>props.conf) from the TA-microsoft-sysmon

[XmlWinEventLog:Microsoft-Windows-Sysmon/Operational]
#SEDCMD-pwd_rule1 = s/ -pw ([^\s\<])+/ -pw ***MASK***/g
REPORT-sysmon = sysmon-eventid,sysmon-version,sysmon-level,sysmon-task,sysmon-opcode,sysmon-keywords,sysmon-created,sysmon-record,sysmon-correlation,sysmon-channel,sysmon-computer,sysmon-sid,sysmon-data,sysmon-md5,sysmon-sha1,sysmon-sha256,sysmon-imphash,sysmon-hashes,sysmon-filename,sysmon-registry
EVAL-src_ip = SourceIp
EVAL-src_host = SourceHostname
EVAL-src = if(isnotnull(SourceHostname),SourceHostname,SourceIp)
EVAL-src_port = SourcePort
EVAL-action = "allowed"
EVAL-app = Image
EVAL-dest_ip = DestinationIp

should this be as follows?
[winsysmon://XmlWinEventLog:Microsoft-Windows-Sysmon/Operational]

but I am not sure where else the winsysmon index needs defining....

[richgalloway]

Yes, the index must exist on the indexers first.
The index = attribute merely tells Splunk where to store your data. It does not create the index itself.
Put index = winsysmon in the XmlWinEventLog stanza of props.conf. Restart Splunk and data should go to the right place.

[packet_hunter]

Ok thank you for the reply.
So then (following your answer) please verify that I am understanding correctly,
step one put the index on the indexers in indexes.conf and restart the indexers
step two put [index=winsysmon] in the props.conf in the Sysmon-TA prior to deploying to the UF

anything else that needs to be done???

I will admit that I am still somewhat confused because I am used to seeing an index defined in an indexes.conf in the app. Sometimes its in the default folder or created new in a local folder by one of my team mates. But I don't usually see it in the props.conf.

Is there an advantage to defining the index in props.conf vs in a separate/new indexes.conf under local in the app?

Thank you

[packet_hunter]

I am not sure where I define the new index name and whether I a just add a new indexes.conf to the app,

[winsysmon]
homePath   = $SPLUNK_DB\winsysmon\db
coldPath   = $SPLUNK_DB\winsysmon\colddb
thawedPath = $SPLUNK_DB\winsysmon\thaweddb
disabled = false