Security

Splunk creates field from from wrong string in my LDAP logs

Contributor

Hi everyone,

I am attempting to graph LDAP transactions from our OpenLDAP LDAP servers, however I'm having trouble getting Splunk to use the correct fields.

In the example below, the first line of the transaction has two IP addresses, IP=192.168.1.111 & IP=0.0.0.0. Splunk assigns the field namedIP to IP=0.0.0.0, but I want it to use the first field IP=192.168.1.111 instead. How can I tell Splunk to match IP=192.168.1.111 & not IP=0.0.0.0?

Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=207 ACCEPT from IP=192.168.1.111:34792 (IP=0.0.0.0:389)
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 EXT oid=1.2.6.1.6.1.1666.37723
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 STARTTLS
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 RESULT oid= err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 TLS established tls_ssf=256 ssf=256
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 BIND dn="" method=128
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 RESULT tag=97 err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH base="dc=example,dc=org" scope=0 deref=0 filter="(objectClass=*)"
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH attr=contextCSN
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=3 UNBIND
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 closed
Tags (2)
0 Karma
1 Solution

Motivator

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

  • during the search

    your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"

    • or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

View solution in original post

Motivator

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

  • during the search

    your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"

    • or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

View solution in original post

Contributor

Thanks. Can you explain what the ?P does in this regex?

0 Karma

Motivator

It's not necessary, but it's a habit from an older Splunk version. If I remember correctly it used to be needed to specify using "python" regex expressions, indicating that what was coming next (inside the <> brackets) was the name of the field.

You can simply use ? instead of ?P it'll work the same.

0 Karma

Contributor

Thanks for your tip about the field extractions. When I try the current regex, I don't get a total match. See https://www.regex101.com/r/oI7gI2/1

What I settled on was the following: from IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*?\(IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

0 Karma

Motivator

If you are using the field name then I don't know why it's not working. Try putting the "P" back in after the "?" in the capture group ?P

0 Karma

Contributor

What is the technical difference between ?P&lt;MyField&gt; and ?&lt;MyField&gt; without the P?

0 Karma

Contributor

Arg. This software eats the angle bracket characters, and doesn't allow their HTML equivalents.

0 Karma

Motivator

you still need to provide the name of the field after the ? using these brackets "<" and ">"

0 Karma

Contributor

Thanks, yes I assigned a name to the field. I just didn't include this in my comment. Thanks.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!