How i can remove unused fields in extracted fields using "Delimiters" ?

[zksvc]

Hi all,

I’m extracting fields from an event using the Field Extractor with a pipe (|) delimiter for sourcetype=alert:access. The extractor produces many interim fields (field1, field2, … fieldN). I only need a subset of them and want to discard the rest.

Example (redacted) raw sample:

Detection: Suspicious Account Lockout Activity (T1110) _time: 1758185796 ComputerName:"MACHINE007.zake.com" | 666 | ... | mitre_technique_id:"T1110" | mitre_tactic_id:"TA0006" | RecordNumber:"307552707" | Raw_Message:"..."

From this tokenized output I only want to keep:

• field2 → host (example: MACHINE007.zake.com)

• field4 → locked_out_user_id (example: 666)

• field10 → mitre_technique_id (example: T1110)

• field12 → mitre_tactic_id (example: TA0006)

  
  All other generated fields should be ignored/removed.

  What is the recommended approach to achieve this?

Thanks in advance for your guidance!

[gcusello]

Hi @zksvc ,

you can modify the original events removing the unused fields using the SED_CMD command in props.conf.

For more infos see at https://docs.splunk.com/Documentation/Splunk/9.4.2/Admin/Propsconf or https://docs.splunk.com/Documentation/Splunk/9.2.1/Data/Anonymizedata

But in this way you modify the original events (I don't know if for compliance reasons yu can do this!) and you loose (because you cut them before indexing) part of the event and of the information.

If you haven't license consuption problems, I hint to leave these fields even if not used, because you could need to use them in a second time.

Ciao.

Giuseppe

[zksvc]

When i check in props.conf it only call REPORT-ikanbana without specific about the fields.

same as if i check in UI there is no configuration for specified fields

[gcusello · Accepted solution]

Hi @zksvc ,

you are speaking of fields extraction at search time, I was speaking, as you can read in the shared links, to remove the unwanted fields before indexing adding the SED_CMD command in props.conf.

This command must be added in the first full Splunk instance that logs pass through, in other words, in the first Heavy Forwarder (if present) or in the Indexers.

When you completed this action, you should also modify the fields extraction REPORT command because the record is changed for the previous SED_CMD command.

Ciao.

Giuseppe