All Apps and Add-ons

How to split a single file into multiple files based on its content header using regex in linux

ejmin
Path Finder

Hi to be able to understand more here's what my current file format

FOOD1 header1 header2 header3 header4 header5 header6
FOOD1 data1 data2 data3 data4 data5 data6
FOOD1 data1 data2 data3 data4 data5 data6
FOOD1 data1 data2 data3 data4 data5 data6
FOOD2 header7 header8 header9 header10 header11 header12
FOOD2 data7 data8 data9 data10 data11 data12
FOOD2 data7 data8 data9 data10 data11 data12
FOOD2 data7 data8 data9 data10 data11 data12

It contains 28 different headers in a single file... all I want is to separate it based on its contents using regex and linux. I used csplit command in linux but the separation of data divides it per event and saved it to one file so 1 raw event into 1 single file. The output I want is FOOD1 into 1 single file and also as FOOD2. The reason I want this to happen is to ingest the FOOD1 and FOOD2 into TSV format and to lessen its parsing function.

This 1 single file is just part of 4000+ files that will be ingest to splunk daily.

1 Solution

jkat54
SplunkTrust
SplunkTrust

grep FOOD1 /path/to/your/file > /path/to/newFood1File
grep FOOD2 /path/to/your/file > /path/to/newFood2File

...

View solution in original post

0 Karma

jkat54
SplunkTrust
SplunkTrust

grep FOOD1 /path/to/your/file > /path/to/newFood1File
grep FOOD2 /path/to/your/file > /path/to/newFood2File

...

0 Karma

ejmin
Path Finder

Thank you very much for your response. I used complicated script to separate one single file but your answer is much more efficient than my code. Thanks again

jkat54
SplunkTrust
SplunkTrust

It was my pleasure! Cheers!

0 Karma
Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...