Getting Data In

Field Names ending with underscore _

cramasta
Builder

Hi,

I have a sourcetype where i defined the field names in the transforms.conf

Transforms.conf

[my_parse]

DELIMS = "|"

FIELDS = "RecordID"| "Timestamp"| "ActivityID"| "Start"| "End"| "TransactionId"| "SessionId"| "Status"| "Description"| "SourceSystemId"| "Message"| "Operation"| "InstanceID"| "LastModified"

Everything was working fine and splunk was picking up the field names as they are listed in the transforms when i would do a search. However one day we decided that we were going to add a new column to the end of the raw data csv file that is forwarded by a splunk agent. So I stopped the agent, went to the indexer and deleted all the records for that particular sourcetype and then changed the transforms.conf entry to look like this

[my_parse]

DELIMS = "|"

FIELDS = "RecordID"| "Timestamp"| "ActivityID"| "Start"| "End"| "TransactionId"| "SessionId"| "Status"| "Description"| "SourceSystemId"| "Message"| "Operation"| "InstanceID"| "LastModified"| "MYMESSAGE"

Then I restarted Splunk on the indexer, turned back on the forwarder, and recreated the csv file with the new column at the end that was forwarded to the indexer. Now when I go into splunk all of my field names for this sourcetype, except for the new field name "MYMESSAGE" have an underscore at the end just like this RecordID_ Timestamp_

Anyone run into this issue before and know how to remove the underscores? Thanks J

Tags (2)
0 Karma
1 Solution

Hajime
Path Finder

Hello, cramasta.

I think your configuration is incorrect. Try this one:

transforms.conf

[my_parse]
DELIMS = "|"
FIELDS = "RecordID", "Timestamp", "ActivityID", "Start", "End", "TransactionId", "SessionId", "Status", "Description", "SourceSystemId", "Message", "Operation", "InstanceID", "LastModified", "MYMESSAGE"

View solution in original post

0 Karma

Hajime
Path Finder

Hello, cramasta.

I think your configuration is incorrect. Try this one:

transforms.conf

[my_parse]
DELIMS = "|"
FIELDS = "RecordID", "Timestamp", "ActivityID", "Start", "End", "TransactionId", "SessionId", "Status", "Description", "SourceSystemId", "Message", "Operation", "InstanceID", "LastModified", "MYMESSAGE"

0 Karma

jrodman
Splunk Employee
Splunk Employee

I don't know what's happening at the moment. Someone else at splunk may.

However you can probably work around the problem for the moment, by finding the auto-generated field extraction and modifying it (removing the underscores). I seem to recall that these end up in etc/apps/learned/local/transforms.conf

0 Karma

jrodman
Splunk Employee
Splunk Employee

Some characters are smashed to underscore at search time. It might be that they are other characters in the FIELD list in transforms.conf. Use Manager or splunk cmd btool transforms list to find them.

0 Karma

cramasta
Builder

For some reason they dont show up in the transforms anywhere that I have looked. In the mean time I just did a field extraction for each field and named it back to the name without the underscore. Still hoping for a better solution though.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...