Security

Splunk creates field from from wrong string in my LDAP logs

stefanlasiewski
Contributor

Hi everyone,

I am attempting to graph LDAP transactions from our OpenLDAP LDAP servers, however I'm having trouble getting Splunk to use the correct fields.

In the example below, the first line of the transaction has two IP addresses, IP=192.168.1.111 & IP=0.0.0.0. Splunk assigns the field namedIP to IP=0.0.0.0, but I want it to use the first field IP=192.168.1.111 instead. How can I tell Splunk to match IP=192.168.1.111 & not IP=0.0.0.0?

Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=207 ACCEPT from IP=192.168.1.111:34792 (IP=0.0.0.0:389)
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 EXT oid=1.2.6.1.6.1.1666.37723
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 STARTTLS
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=0 RESULT oid= err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 TLS established tls_ssf=256 ssf=256
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 BIND dn="" method=128
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=1 RESULT tag=97 err=0 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH base="dc=example,dc=org" scope=0 deref=0 filter="(objectClass=*)"
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SRCH attr=contextCSN
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=2 SEARCH RESULT tag=101 err=0 nentries=1 text=
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 op=3 UNBIND
Dec 15 10:13:27 ldap1 slapd[50625]: conn=12341234 fd=206 closed
Tags (2)
0 Karma
1 Solution

aholzer
Motivator

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

  • during the search

    your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"

    • or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

View solution in original post

aholzer
Motivator

Root cause:
The reason this is occurring is because you have 2 key=value pairs with the same key. So Splunk will extract the first value into the key, and then overwrite it with the second value.

Solution:
Define a new field extraction that specifies exactly how you want the fields extracted.

  • during the search

    your search ... | rex "from IP=(?P[^:]+?).*?(IP=(?P[^:])"

    • or using the field extraction to create a new field extraction object. You can use the above regex (what is inside the quotes).

If you do it this way, you'll get two fields, one called "IP" with the first IP, and another called "IP_2" with the second IP.

Hope this helps

--- EDIT ---

I edited my rex command to stop at the ":"; this is to avoid catching the port in the first IP.

stefanlasiewski
Contributor

Thanks. Can you explain what the ?P does in this regex?

0 Karma

aholzer
Motivator

It's not necessary, but it's a habit from an older Splunk version. If I remember correctly it used to be needed to specify using "python" regex expressions, indicating that what was coming next (inside the <> brackets) was the name of the field.

You can simply use ? instead of ?P it'll work the same.

0 Karma

stefanlasiewski
Contributor

Thanks for your tip about the field extractions. When I try the current regex, I don't get a total match. See https://www.regex101.com/r/oI7gI2/1

What I settled on was the following: from IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*?\(IP=(?\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

0 Karma

aholzer
Motivator

If you are using the field name then I don't know why it's not working. Try putting the "P" back in after the "?" in the capture group ?P

0 Karma

stefanlasiewski
Contributor

What is the technical difference between ?P&lt;MyField&gt; and ?&lt;MyField&gt; without the P?

0 Karma

stefanlasiewski
Contributor

Arg. This software eats the angle bracket characters, and doesn't allow their HTML equivalents.

0 Karma

aholzer
Motivator

you still need to provide the name of the field after the ? using these brackets "<" and ">"

0 Karma

stefanlasiewski
Contributor

Thanks, yes I assigned a name to the field. I just didn't include this in my comment. Thanks.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...