topic Re: Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ? in Splunk Search

Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ?

johnjj7141 — Thu, 21 Jun 2018 15:22:04 GMT

I am importing a dump from my openLDAP into splunk via on one-time "data-import" . The fields, O, OU, DN, MAIL, etc are one value fields and are no issue. I am having issues with two fields that hold multi-values. One multivalue (comment) starts with a date value.

ie:

comment=20180604123700|admin-name|some admin comment string here
comment=20180604125700|admin-name|some admin comment string here
comment=20180612105700|admin-name|some admin comment string here
comment=20180616021500|admin-name|some admin comment string here
...

The other mulitvalue is groupmembers:

groupmembers=group_1
groupmembers=group_2
groupmembers=group_3
groupmembers=group_4
...

My problem with the comments appears that splunk sees the "date" and stops processing after that. It wont treat the entire line as a value string which is what I need.

My problem with "groupmembers" is that splunk only takes the first value and ignores the rest. I have tried setting MV_ADD to "true" on the "advanced settings" but it does not change how splunk handles it. The same is true for comments....all I get is the first "date" and nothing else.

My props.conf file from the import:

[LDAP Format]
BREAK_ONLY_BEFORE = dn=
MAX_TIMESTAMP_LOOKAHEAD = 4096
NO_BINARY_CHECK = true
TIME_PREFIX = approvaldate=
description = Ldap Import Sourcetype
disabled = false
pulldown_type = true
MV_ADD = true
MAX_DAYS_AGO = 5475
category = Application

Any advice would be greatly appreciated. Thanks.

Re: Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ?

darrenfuller — Thu, 21 Jun 2018 17:18:26 GMT

MV_ADD is a transforms.conf directive, not props. If you want to rerun a regular expression multiple times , you need to combine props and transforms.

I can't see your data, but I think something like the following would work:

#add these lines to your props.conf
[LDAP Format]
disabled = 0
REPORT-01-parseGroupMembers = parsegroupmembers
REPORT-02-parseComments = parsecomments

#add these to transforms.conf
[parsegroupmembers]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = groupmembers
REGEX = groupmembers\=([^\r\n]+)
FORMAT = groupmembers::$1
MV_ADD = true

[parsecomments]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = comments
REGEX = comment\=([^\r\n]+)
FORMAT = comments::$1
MV_ADD = true

Re: Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ?

johnjj7141 — Mon, 25 Jun 2018 18:17:40 GMT

Awesome....worked like a champ.... AND I now understand MV_ADD better! Thanks!