Splunk Search

Why am I having issues with two fields that hold multi-values in one time import of OpenLDAP Data ?

johnjj7141
Explorer

I am importing a dump from my openLDAP into splunk via on one-time "data-import" . The fields, O, OU, DN, MAIL, etc are one value fields and are no issue. I am having issues with two fields that hold multi-values. One multivalue (comment) starts with a date value.

ie:

comment=20180604123700|admin-name|some admin comment string here
comment=20180604125700|admin-name|some admin comment string here
comment=20180612105700|admin-name|some admin comment string here
comment=20180616021500|admin-name|some admin comment string here
...

The other mulitvalue is groupmembers:

groupmembers=group_1
groupmembers=group_2
groupmembers=group_3
groupmembers=group_4
...

My problem with the comments appears that splunk sees the "date" and stops processing after that. It wont treat the entire line as a value string which is what I need.

My problem with "groupmembers" is that splunk only takes the first value and ignores the rest. I have tried setting MV_ADD to "true" on the "advanced settings" but it does not change how splunk handles it. The same is true for comments....all I get is the first "date" and nothing else.

My props.conf file from the import:

[LDAP Format]
BREAK_ONLY_BEFORE = dn=
MAX_TIMESTAMP_LOOKAHEAD = 4096
NO_BINARY_CHECK = true
TIME_PREFIX = approvaldate=
description = Ldap Import Sourcetype
disabled = false
pulldown_type = true
MV_ADD = true
MAX_DAYS_AGO = 5475
category = Application

Any advice would be greatly appreciated. Thanks.

0 Karma
1 Solution

darrenfuller
Contributor

MV_ADD is a transforms.conf directive, not props. If you want to rerun a regular expression multiple times , you need to combine props and transforms.

I can't see your data, but I think something like the following would work:

#add these lines to your props.conf
[LDAP Format]
disabled = 0
REPORT-01-parseGroupMembers = parsegroupmembers
REPORT-02-parseComments = parsecomments

#add these to transforms.conf
[parsegroupmembers]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = groupmembers
REGEX = groupmembers\=([^\r\n]+)
FORMAT = groupmembers::$1
MV_ADD = true

[parsecomments]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = comments
REGEX = comment\=([^\r\n]+)
FORMAT = comments::$1
MV_ADD = true

View solution in original post

0 Karma

darrenfuller
Contributor

MV_ADD is a transforms.conf directive, not props. If you want to rerun a regular expression multiple times , you need to combine props and transforms.

I can't see your data, but I think something like the following would work:

#add these lines to your props.conf
[LDAP Format]
disabled = 0
REPORT-01-parseGroupMembers = parsegroupmembers
REPORT-02-parseComments = parsecomments

#add these to transforms.conf
[parsegroupmembers]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = groupmembers
REGEX = groupmembers\=([^\r\n]+)
FORMAT = groupmembers::$1
MV_ADD = true

[parsecomments]
disabled = 0
SOURCE_KEY = _raw
DEST_KEY = comments
REGEX = comment\=([^\r\n]+)
FORMAT = comments::$1
MV_ADD = true
0 Karma

johnjj7141
Explorer

Awesome....worked like a champ.... AND I now understand MV_ADD better! Thanks!

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...