Why did the field name change while indexing csv file in splunk?

[sivaranjiniG]

I am trying to index a csv file by uploading it through splunk web.... while setting up sourcetype i could see all my field names become changed

For example:

field.project become field_project

Can somebody help me in this?

[damien_chillet · Accepted solution]

By default, Splunk replaces non alphanumeric characters in field names with underscore:

From http://docs.splunk.com/Documentation/Splunk/7.0.0/Knowledge/Configureadvancedextractionswithfieldtra...

 CLEAN_KEYS

Controls whether the system strips leading underscores and 0-9 characters from the field names it extracts. Key cleaning is the practice of replacing any non-alphanumeric characters in field names with underscores, as well as the removal of leading underscores and 0-9 characters from field names.

You could try set it to false for your sourcetype transforms.

[sivaranjiniG]

Sorry i m totally new to this....
Can you help me in creating this transforms.conf file. not sure what should be the stanza name

[damien_chillet]

Here is the test csv I am using:

field.name,field.value
field1,splunk
field2,answers

If you ingest the file using the sourcetype detected by Splunk (that will be csv here), it will use indexed extractions and replace "field.attr" with "field_attr". It is not possible to prevent Splunk from doing that at indexing time (as far as i know).

So you need to turn index extractions off. I used the following props.conf / transforms.conf:

props.conf

[test_abc]
SHOULD_LINEMERGE = false
disabled = false
INDEXED_EXTRACTIONS=
REPORT-no_clean_keys = test_abc_transforms
TRANSFORMS-drop_header = drop_header

transforms.conf

[test_abc_transforms]
CLEAN_KEYS = false
DELIMS=,
FIELDS=field.name,field.value

[drop_header]
REGEX = field\.name,field\.value
DEST_KEY = queue
FORMAT = nullQueue

test_abc_transforms extracts fields from your csv and tell Splunk not to clean the keys (replace "." with "_" in this case).

drop_header transform is to tell Splunk not to index the header of your CSV file.

Restart Splunk then ingest your file using Splunk CLI:

$SPLUNK_HOME/bin/splunk add oneshot path/to/your/csv/abc.csv -sourcetype test_abc -index main

It worked for me, let me know if you need further help.
Also i welcome anyone reading this solution to suggest any ameliorations, there may be better ways to achieve that 🙂

[sivaranjiniG]

Thanks So much its working 🙂

[robertlynch2020]

Hi

If you have some fields with and without the ., below is an example of how to get that to work.

However it only works going into an event index, it does not seem to work going into metrices.

[test_abc_transforms]
CLEAN_KEYS = false
DELIMS=,
FIELDS=degraded.threshold,down.threshold

[drop_header]
REGEX = metric_timestamp,metric_name,_value,degraded\.threshold,down\.threshold
DEST_KEY = queue
FORMAT = nullQueue

metric_timestamp,metric_name,_value,degraded.threshold,down.threshold
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300
1695201472,mx.process.cpu.utilization,1.373348018,30,300