Simple field transformation to split firstname lastname - help!

[BradOH]

Hello all, I've been working for the better part of a day trying to get a simple field transformation working without success. Here's the scenario.

We have a source CSV file which is pulled into Splunk via file input. One of the fields is formatted "Lastname Firstname" (with space).  The field name in the source file also has a space in the name (which could be the issue, who knows).  

I've added a field transformation to split into separate fields, as follows:

• Regular expression (?<Last_Name>^\S+)\s+(?<First_Name>.+$)
• Source key "Full Name" (enclosed in quotes in Splunk Web interface due to space in field name)

I tried two different options. The first was adding a reference to this field transformation in the source type definition, which caused the following errors to appear in Splunk.log

Regex 'extract_person_name' has no capturing groups, transform_name='EXTRACT-Person_Name'.

I've tested the regex, it works...

I also tried removing this reference in the source type and adding a field extraction definition referencing the transform, which appears to do nothing.

What am I missing?  I've googled and searched the documentation and forums without finding a solution...

[richgalloway · Accepted solution]

The EXTRACT attribute requires a regex value containing a named capture group.  The listed props don't have that so that's why the error is reported.

To extract a field using a transform, use either the REPORT (search time) or TRANSFORM (index time) attribute.

[BradOH]

Hey, finally figured it out.  After updating the attribute to REPORT as suggested and updating permission to GLOBAL on transformation, this eliminated the errors, but fields were still not extracting.

It appears the quotes I added around the original key field name (as the field in the CSV had a space) were not required.  Once I removed the quotes from the key field name, all works as expected...

Thanks for your assistance, learned a lot figuring this one out!

[BradOH]

Thanks, appears you've got me halfway there, as I'm getting a different error at least....

Invalid key-value parser, ignoring it, transform_name='extract_person_name'.

The transform is unchanged, I just updated the attribute in the source type definition (to REPORT-Person_Name)

[BradOH]

@richgalloway  That may have been a permission issue.  I've updated the field transformation to Global permissions, which seems to have eliminated all the errors in Splunk.log, but still not seeing the extracted fields in the index...

| search index=test-extract shows the original CSV extracted fields but not the firstname/lastnames which should be extracted.  Still missing something... 

[richgalloway]

Please share the full props.conf stanza for the sourcetype and the referenced transforms.conf stanzas.

[BradOH]

Thanks, here you go as I originally configured where the source type directly references the transform (which I had assumed would work...)

PROPS.CONF

[test-source:csv]
BREAK_ONLY_BEFORE_DATE = 
DATETIME_CONFIG = 
FIELD_DELIMITER = ,
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = "Event Date_Time"
TIME_FORMAT = %m/%d/%Y %H:%M
TZ = America/New_York
category = Structured
disabled = false
pulldown_type = 1
Extract-Person_Name = extract_person_name

TRANSFORMS.CONF

[extract_person_name]
CLEAN_KEYS = 0
REGEX = (?<Last_Name>^\S+)\s+(?<First_Name>.+$)
SOURCE_KEY = "Person Name"

[gcusello]

Hi @BradOH ,

where did you insert the transformation to split names: in a search or where?

if in search I'd use something like this:

<your_search>
| rename "Full Name" AS Full_Name
| rex field=Full_Name "?<Last_Name>^\S+)\s+(?<First_Name>.+$"

Ciao.

Giuseppe

[BradOH]

I'm trying to configure a field transformation so the date ingested from the CSV to our index will be in the proper format, rather than fiddling with the data inline in reports...

[gcusello]

Hi @BradOH ,

you can do this pre-processing the CSV before ingestion or at search time using my search.

I'm not sure that it's possible to apply the transformation at index time.

Ciao.

Giuseppe