Splunk Search

How to customize raw data into fields using regex before exporting to CSV?

zoeygirl45
Explorer

We are ingesting syslog logs. While doing a search and exporting to csv, we would like the raw data column to be split into readable fields. Do I need to use regex?

What can I post here so you can see the raw data?

aljohnson_splun
Splunk Employee
Splunk Employee
  • A more permanent solution would be using transforms.conf and props.conf. Docs here.
  • or use the field extractor ? Docs here
  • or maybe you just love that web gui ? You can manage extractions there as well. Here

Example:

Add a new error code field
This example shows how to create a new "error code" field by configuring a field extraction in props.conf. The field can be identified by the occurrence of device_id= followed by a word within brackets and a text string terminating with a colon. The field should be extracted from events related to the testlog source type.

In props.conf, add:

[testlog]
EXTRACT-errors = device_id=\[w+\](?<err_code>[^:]+)

Or get jiggy with that GUI:
alt text

0 Karma

rlough
Path Finder

You can try using the "erex" command that Splunk has. This command basically creates your rex call for you.

Here's how it works:

... YOUR SEARCH ... | erex FIELD_NAME examples="2345,3455,9283"

After your search, use the erex command by replacing FIELD_NAME with a name of your choice and then post real examples of the data that you want to be put in this field (you only need two or three examples). These examples of yours will replace the dummy data that I have above in between quotes. Your examples must be comma separated and surrounded by quotes.

After your search runs, the rex command that it creates for you can be found under the "Jobs" tab.

sk314
Builder

you could post a few lines from your logs itself...

esix_splunk
Splunk Employee
Splunk Employee

If this is syslog, and your timestamps and events are being recognized correctly, you can use the Field Extractor Utility to extract fields as you desire. And then using the fields and outputcsv commands, you can format how you want your CSV to look.

Other options as mentioned are using props and transforms to create your own extractions and fields.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...