Splunk Search

Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ?

johnjj7141
Explorer

I am importing a dump from my openLDAP into splunk via on one-time "data-import" . The fields, O, OU, DN, MAIL, etc are one value fields and are no issue. I am having issues with two fields that hold multi-values. One multivalue (comment) starts with a date value.

ie:

comment=20180604123700|admin-name|some admin comment string here
comment=20180604125700|admin-name|some admin comment string here
comment=20180612105700|admin-name|some admin comment string here
comment=20180616021500|admin-name|some admin comment string here
...

The other mulitvalue is groupmembers:

groupmembers=group_1
groupmembers=group_2
groupmembers=group_3
groupmembers=group_4
...

My problem with the comments appears that splunk sees the "date" and stops processing after that. It wont treat the entire line as a value string which is what I need.

My problem with "groupmembers" is that splunk only takes the first value and ignores the rest. I have tried setting MV_ADD to "true" on the "advanced settings" but it does not change how splunk handles it. The same is true for comments....all I get is the first "date" and nothing else.

My props.conf file from the import:

[LDAP Format]
BREAK_ONLY_BEFORE = dn=
MAX_TIMESTAMP_LOOKAHEAD = 4096
NO_BINARY_CHECK = true
TIME_PREFIX = approvaldate=
description = Ldap Import Sourcetype
disabled = false
pulldown_type = true
MV_ADD = true
MAX_DAYS_AGO = 5475
category = Application

Any advice would be greatly appreciated. Thanks.

0 Karma
1 Solution

darrenfuller
Contributor

MV_ADD is a transforms.conf directive, not props. If you want to rerun a regular expression multiple times , you need to combine props and transforms.

I can't see your data, but I think something like the following would work:

#add these lines to your props.conf
[LDAP Format]
disabled = 0
REPORT-01-parseGroupMembers = parsegroupmembers
REPORT-02-parseComments = parsecomments

#add these to transforms.conf
[parsegroupmembers]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = groupmembers
REGEX = groupmembers\=([^\r\n]+)
FORMAT = groupmembers::$1
MV_ADD = true

[parsecomments]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = comments
REGEX = comment\=([^\r\n]+)
FORMAT = comments::$1
MV_ADD = true

View solution in original post

0 Karma

darrenfuller
Contributor

MV_ADD is a transforms.conf directive, not props. If you want to rerun a regular expression multiple times , you need to combine props and transforms.

I can't see your data, but I think something like the following would work:

#add these lines to your props.conf
[LDAP Format]
disabled = 0
REPORT-01-parseGroupMembers = parsegroupmembers
REPORT-02-parseComments = parsecomments

#add these to transforms.conf
[parsegroupmembers]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = groupmembers
REGEX = groupmembers\=([^\r\n]+)
FORMAT = groupmembers::$1
MV_ADD = true

[parsecomments]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = comments
REGEX = comment\=([^\r\n]+)
FORMAT = comments::$1
MV_ADD = true
0 Karma

johnjj7141
Explorer

Awesome....worked like a champ.... AND I now understand MV_ADD better! Thanks!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...