Field Names ending with underscore _

[cramasta]

Hi,

I have a sourcetype where i defined the field names in the transforms.conf

Transforms.conf

[my_parse]

DELIMS = "|"

FIELDS = "RecordID"| "Timestamp"| "ActivityID"| "Start"| "End"| "TransactionId"| "SessionId"| "Status"| "Description"| "SourceSystemId"| "Message"| "Operation"| "InstanceID"| "LastModified"

Everything was working fine and splunk was picking up the field names as they are listed in the transforms when i would do a search. However one day we decided that we were going to add a new column to the end of the raw data csv file that is forwarded by a splunk agent. So I stopped the agent, went to the indexer and deleted all the records for that particular sourcetype and then changed the transforms.conf entry to look like this

[my_parse]

DELIMS = "|"

FIELDS = "RecordID"| "Timestamp"| "ActivityID"| "Start"| "End"| "TransactionId"| "SessionId"| "Status"| "Description"| "SourceSystemId"| "Message"| "Operation"| "InstanceID"| "LastModified"| "MYMESSAGE"

Then I restarted Splunk on the indexer, turned back on the forwarder, and recreated the csv file with the new column at the end that was forwarded to the indexer. Now when I go into splunk all of my field names for this sourcetype, except for the new field name "MYMESSAGE" have an underscore at the end just like this RecordID_ Timestamp_

Anyone run into this issue before and know how to remove the underscores? Thanks J

[Hajime · Accepted solution]

Hello, cramasta.

I think your configuration is incorrect. Try this one:

transforms.conf

[my_parse]
DELIMS = "|"
FIELDS = "RecordID", "Timestamp", "ActivityID", "Start", "End", "TransactionId", "SessionId", "Status", "Description", "SourceSystemId", "Message", "Operation", "InstanceID", "LastModified", "MYMESSAGE"

[jrodman]

I don't know what's happening at the moment. Someone else at splunk may.

However you can probably work around the problem for the moment, by finding the auto-generated field extraction and modifying it (removing the underscores). I seem to recall that these end up in etc/apps/learned/local/transforms.conf

[jrodman]

Some characters are smashed to underscore at search time. It might be that they are other characters in the FIELD list in transforms.conf. Use Manager or splunk cmd btool transforms list to find them.

[cramasta]

For some reason they dont show up in the transforms anywhere that I have looked. In the mean time I just did a field extraction for each field and named it back to the name without the underscore. Still hoping for a better solution though.