Use OSSEC archives.log to collect logs of different systems

[banaie]

Hi all,

I am trying to use OSSEC archives.log to collect logs of different systems. It can collect whatever you need from windows and Linux systems and gather them inside the archives.log file as a raw log for all.

Then, I need to parse the file and assign correct sourcetypes and source and host variables to them. I tried using props.conf and transforms.conf to do this using available transformations. I have succeeded getting for example windows events a WinEventLog sourcetype using that method and it works correctly on assigning the sourcetype and trimming the event body  from the original log file. However, the fields are not correctly extracted from that Windows Log.

Sample archives.log of two windows and linux events are as follows:

2020 Jun 16 00:01:04 (E-Fl) 192.168.3.2->WinEvtLog 2020 Jun 16 00:01:00 WinEvtLog: Security: AUDIT_SUCCESS(4672): Microsoft-Windows-Security-Auditing: (no user): no domain: eFl: Special privileges assigned to new logon. Subject:  Security ID:  S-1-5-21-3960285484-3209917605-2958509563-1006  Account Name:  t_apx  Account Domain:  EFL  Logon ID:  0x133a050c7  Privileges:  SeSecurityPrivilege     SeTakeOwnershipPrivilege     SeLoadDriverPrivilege     SeBackupPrivilege     SeRestorePrivilege     SeDebugPrivilege     SeSystemEnvironmentPrivilege     SeImpersonatePrivilege     SeDelegateSessionUserImpersonatePrivilege
2020 Jun 16 00:01:06 (SE-Cloud) 192.168.9.194->/var/log/messages Jun 16 00:01:05 ccrtl13c snmpd[1204]: Connection from UDP: [192.168.9.202]:50515->[192.168.9.194]:161
2020 Jun 16 00:01:08 (FTP) 192.168.9.230->WinEvtLog 2020 Jun 16 00:01:05 WinEvtLog: System: WARNING(51): Disk: (no user): no domain: FTPPublic.serv.local: An error was detected on device \Device\Harddisk5\DR5 during a paging operation.  

my props.conf

[ossec_archives]
TRANSFORMS-assignSourcetype = extractEvent, assignWinEvtLog
#,assignSyslog

my transforms.conf

###### OSSEC_Archives ######
[extractEvent]
SOURCE_KEY = _raw
REGEX = WinEvtLog\s(.*)$
FORMAT = $1
DEST_KEY = _raw
#CLONE_SOURCETYPE = WinEventLog


[assignWinEvtLog]
#CLONE_SOURCETYPE = WinEventLog
REGEX = WinEvtLog:
DEST_KEY =MetaData:Sourcetype
FORMAT =sourcetype::WinEventLog

#[assignSyslog]
#REGEX = \s[WinEvtLog:].*$
#DEST_KEY =MetaData:Sourcetype
#FORMAT =sourcetype::syslog

Can you please help me get the data in correctly and make default windows and linux add-ons extract the related fileds?

Thanks

[richgalloway]

May I ask *why* you are aggregating data is OSSEC before sending it to Splunk? I believe you're just making more work for yourself.
As you've discovered, using a transform to change sourcetypes is not a perfect solution. That's because the props for the new sourcetype are not applied at index time.
There are some apps available for OSSEC on splunkbase (https://splunkbase.splunk.com/apps/#/search/ossec/). Perhaps one of them will help you accomplish your goal.

[banaie]

@richgalloway  Thanks for your reply. I wanna use OSSEC because it already is installed on the premise and I don't want to add another UF or HF on those systems! Moreover, It gives almost all the needed features and I just need to index it correctly!

Thanks for the offer! I already has tested all of them and they only focus on alerts.log and do not have any solution for logs that are generated as raw on archives.log of the OSSEC.

Isn't there any solution for applying those props for the new sourcetype configuration?

[richgalloway]

I don't know enough about OSSEC to give a good answer, but I think you've painted yourself into a corner by putting different sourcetypes into the same file.

[banaie]

@richgalloway Unfortunately, as far as I know, there is no option for ossec to differentiate the log files. However, I thought Splunk should have some solution for this problem. Because, these events are events that are embedded into another event and I thought I can extract the event and provide a sourcetype and ask the default Splunk process to do the rest! 

Don't you have any proposition even for extracting a limited set of logs from this type of log?