Knowledge Management

Is there a way to remove/replace spaces in between fieldname?

snallam123
Path Finder

we are extracting fields with spaces in it using below transforms, Is there a way we can remove spaces in between fields from backend? There are 100's of fields with spaces. I tried with field alias amd it's hard to apply for each and every field. 

Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<field>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2

_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line:  test"

output I am getting:
"Process Id" =  6228

Is there a way we can change this to ProcessId=6228 or Process-Id=6228 ?


From UI i tried this, Can someone help me with backend trick

| makeresults
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test"
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)"
| rex mode=sed field=field "s/ /_/g"
| eval tmp=mvzip(field,value,"=")
| rename tmp as _raw
| kv
| table *

Labels (2)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

I'm not tried this myself, but can you try INGEST_EVAL + replace function within your Transforms?

0 Karma

snallam123
Path Finder

I tried with it but no success @isoutamo ,

Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<fieldNam>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2

_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025

<Ingest_eval_change_fields>

INGEST_EVAL = NewField=replace(fieldNam, "\s", "_")   -

When we did Ingest_eval_change_fields transforms FORMAT function in earlier transforms has already changed to field names so "fieldNam" no longer exists.

0 Karma

mayurr98
Super Champion
  • The SEDCMD script applies only to the _raw field at index time. With the regular expression transform, you can apply changes to other fields

https://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedata?_gl=1*16ogb9v*_ga*MTA1MzM4MzA...

you can try using evaluation functions as well 

 

 

| makeresults 
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test" 
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)" 
| mvexpand field 
| eval field1=replace(field,"\s","_")

 

 

see if you can use calculated fields if its not a multivalue field.

0 Karma

mayurr98
Super Champion

Hello,

you can achieve this using SEDCMD Scripts

https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/Anonymizedata#Example_of_substitution_using_...

Transforms.conf must be used for the extracted field, and SEDCMD for _raw.

See here for details. way at the bottom.

https://answers.splunk.com/answers/739964/need-sedcmd-help.html

 

0 Karma

snallam123
Path Finder

Thanks @mayurr98 ,

If i use SED it ll modify _raw data as well, Is it possible to change field name with modifying _raw data, I am thinking If i use SED to modify the _raw data it ll impact the ingestion speed.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever

Now On Demand Whether you're managing complex deployments or looking to future-proof your data ...

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...