topic Re: Why is my REGEX in transforms.conf not working to filter data to nullQueue? in Splunk Search

Why is my REGEX in transforms.conf not working to filter data to nullQueue?

Aexyn — Tue, 21 Jun 2016 13:54:33 GMT

Hi,

I want to filter Windows Security event logs in (/etc/system/local/)props.conf/transforms.conf.

Here is my transforms.conf file:

[FilterEventWindows]
REGEX=(?is)^.*EventCode=(5145|4656).*(Nom de l’objet :\t\t(?!C:\\epic\\*))(.*)$
DEST_KEY = queue
FORMAT = nullQueue

ie I only want events 5145|4656 with the ObjectName (of the filesystem) in C:\epic.

A standard log looks like this:

06/21/2016 02:31:14 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4656
EventType=0
Type=Information
ComputerName= 
TaskCategory=Système de fichiers
OpCode=Informations
RecordNumber=1764752
Keywords=Succès de l’audit
Message=Un handle vers un objet a été demandé.

Sujet :
    ID de sécurité :      TEST\Administrateur
    Nom du compte :     Administrateur
    Domaine du compte :     TEST
    ID d’ouverture de session         0x1C307

Objet :
    Serveur de l’objet :      Security
    Type d’objet :        File
    Nom de l’objet :      C:\Users\Administrateur\Documents
    ID du handle :      0x18b0
    Attributs de ressource :    -

Informations sur le processus :
    ID du processus :       0x7d4
    Nom du processus :      C:\Windows\explorer.exe

After testing it on a (PHP) regex tester, it seems it should work...
Obviously the problem is not fixed.
I tried many things around this syntax, like deleting/adding ^ and $, starting without .*, specifying (?s)(?i) or (?si)... I tested this regex with /gsi, /si is right too, but I keep receiving events of other folders.
I tried Nom de l’objet :\t\t(?!C:\\epic\\* since I can filter eventsID with whitelist, but it is wrong too.

My props.conf file is simply

[WinEventLog:Security]
TRANSFORMS-FilterEvent = FilterEventWindows

Am I missing something?

Re: Why is my REGEX in transforms.conf not working to filter data to nullQueue?

richgalloway — Tue, 21 Jun 2016 17:07:16 GMT

It's not entirely clear what you want the regex to match, but this string matches the event code and the object name. It works at regex101.com.

(?s)EventCode=(5145|4656).*(Nom de l’objet :\s*C:\\(?!epic\\))(.*?)\n

Re: Why is my REGEX in transforms.conf not working to filter data to nullQueue?

Aexyn — Wed, 22 Jun 2016 07:02:04 GMT

No that was not exactly what I meant, sorry for my bad english.
Actually I want the string match with the EventID and with " Nom de l’objet : " but NOT with the ObjectName ie the filename (ie the file is not in C:\epic).

This regex doesn't work since it matchs pretty anything on the folder name.

About this, it is possible to define a configuration that match the ObjectName to send ONLY the matching logs instead of dropping logs which don't match through nullQueue?

Re: Why is my REGEX in transforms.conf not working to filter data to nullQueue?

richgalloway — Thu, 23 Jun 2016 18:18:29 GMT

A small change to the regex fixed it. See my edited answer.

Re: Why is my REGEX in transforms.conf not working to filter data to nullQueue?

Aexyn — Mon, 27 Jun 2016 08:10:09 GMT

Mhh this time I'm sure your regex is correct, good job.
However, it still doesn't work and I think there is something tricky I don't see in my files.

Since ObjectName is not directly a field of the log (I mean it is just displayed as a part of the message field), maybe there is a problem :(.