topic Re: Splunk creates field from from wrong string in my LDAP logs in Security

Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Mon, 15 Dec 2014 19:32:09 GMT

Hi everyone,

I am attempting to graph LDAP transactions from our OpenLDAP LDAP servers, however I'm having trouble getting Splunk to use the correct fields.

In the example below, the first line of the transaction has two IP addresses, IP=192.168.1.111 & IP=0.0.0.0. Splunk assigns the field namedIP to IP=0.0.0.0, but I want it to use the first field IP=192.168.1.111 instead. How can I tell Splunk to match IP=192.168.1.111 & not IP=0.0.0.0?

Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=207 ACCEPT from IP=192.168.1.111:34792 (IP=0.0.0.0:389)
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 EXT oid=1.2.6.1.6.1.1666.37723
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 STARTTLS
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 RESULT oid= err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 TLS established tls_ssf=256 ssf=256
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 BIND dn="" method=128
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 RESULT tag=97 err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH base="dc=example,dc=org" scope=0 deref=0 filter="(objectClass=*)"
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH attr=contextCSN
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=3 UNBIND
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 closed

Re: Splunk creates field from from wrong string in my LDAP logs

aholzer — Mon, 15 Dec 2014 19:41:26 GMT

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

during the search

your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"
- or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

Re: Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Mon, 15 Dec 2014 20:52:00 GMT

Thanks. Can you explain what the ?P does in this regex?

Re: Splunk creates field from from wrong string in my LDAP logs

aholzer — Mon, 15 Dec 2014 21:08:08 GMT

It's not necessary, but it's a habit from an older Splunk version. If I remember correctly it used to be needed to specify using "python" regex expressions, indicating that what was coming next (inside the <> brackets) was the name of the field.

You can simply use ? instead of ?P it'll work the same.

Re: Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Mon, 15 Dec 2014 22:35:44 GMT

Thanks for your tip about the field extractions. When I try the current regex, I don't get a total match. See https://www.regex101.com/r/oI7gI2/1

What I settled on was the following: from IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*?\(IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Re: Splunk creates field from from wrong string in my LDAP logs

aholzer — Tue, 16 Dec 2014 14:36:23 GMT

you still need to provide the name of the field after the ? using these brackets "<" and ">"

Re: Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Fri, 19 Dec 2014 17:47:56 GMT

Thanks, yes I assigned a name to the field. I just didn't include this in my comment. Thanks.

Re: Splunk creates field from from wrong string in my LDAP logs

aholzer — Fri, 19 Dec 2014 19:38:39 GMT

If you are using the field name then I don't know why it's not working. Try putting the "P" back in after the "?" in the capture group ?P

Re: Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Fri, 19 Dec 2014 21:39:23 GMT

What is the technical difference between ?P<MyField> and ?<MyField> without the P?

Re: Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski — Fri, 19 Dec 2014 21:40:44 GMT

Arg. This software eats the angle bracket characters, and doesn't allow their HTML equivalents.