we are extracting fields with spaces in it using below transforms, Is there a way we can remove spaces in between fields from backend? There are 100's of fields with spaces. I tried with field alias amd it's hard to apply for each and every field.
Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<field>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2
_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test"
output I am getting:
"Process Id" = 6228
Is there a way we can change this to ProcessId=6228 or Process-Id=6228 ?
From UI i tried this, Can someone help me with backend trick
| makeresults
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test"
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)"
| rex mode=sed field=field "s/ /_/g"
| eval tmp=mvzip(field,value,"=")
| rename tmp as _raw
| kv
| table *
I'm not tried this myself, but can you try INGEST_EVAL + replace function within your Transforms?
I tried with it but no success @isoutamo ,
Transforms:
[sourcetype]
SOURCE_KEY=_raw
REGEX=(?<fieldNam>[a-zA-Z ]+):(?<value>.+)
FORMAT=$1:$2
_raw:
"Process Create:
Utc Time: 2022-04-28 22:08:22.025
<Ingest_eval_change_fields>
INGEST_EVAL = NewField=replace(fieldNam, "\s", "_") -
When we did Ingest_eval_change_fields transforms FORMAT function in earlier transforms has already changed to field names so "fieldNam" no longer exists.
you can try using evaluation functions as well
| makeresults
| eval _raw="Process Create:true
Utc Time: 2022-04-28 22:08:22.025
Process Guid: {XYZ-bd56-5903-0000-0010e9d95e00}
Process Id: 6228
Image: chrome.exe
Command Line: test"
| rex field=_raw max_match=0 "(?<field>[a-zA-Z ]+):(?<value>.+)"
| mvexpand field
| eval field1=replace(field,"\s","_")
see if you can use calculated fields if its not a multivalue field.
Hello,
you can achieve this using SEDCMD Scripts
Transforms.conf must be used for the extracted field, and SEDCMD for _raw.
See here for details. way at the bottom.
https://answers.splunk.com/answers/739964/need-sedcmd-help.html
Thanks @mayurr98 ,
If i use SED it ll modify _raw data as well, Is it possible to change field name with modifying _raw data, I am thinking If i use SED to modify the _raw data it ll impact the ingestion speed.