topic Re: How can I parse 2 sets of CSVs in one file? in Getting Data In

How can I parse 2 sets of CSVs in one file?

hylam — Mon, 26 Oct 2015 11:02:00 GMT

How could I parse this?

section1String
field1,field2,field3
value1,value2,value3
value1,value2,value3
value1,value2,value3
section2String
field4,field5,field6,field7
value4,value5,value6,value7
value4,value5,value6,value7
value4,value5,value6,value7

The number of value lines is unknown. Please handle 4 cases
a) The header lines are known strings
b) The header lines satisfy a regex
c) The header lines are 1 line below a known string
d) The header lines are 1 line below a string satisfying a regex

Re: How can I parse 2 sets of CSVs in one file?

woodcock — Mon, 26 Oct 2015 15:27:53 GMT

Like this:

| inputcsv TwoInOne.csv | head 4 | outputcsv FirstHalf.csv
| inputcsv TwoInOne.csv | tail 4 | outputcsv SecondHalf.csv

| inputcsv FirstHalf.csv | append [|inputcsv SecondHalf.csv]

Re: How can I parse 2 sets of CSVs in one file?

hylam — Mon, 26 Oct 2015 15:35:01 GMT

plz see edit1

Re: How can I parse 2 sets of CSVs in one file?

hylam — Mon, 26 Oct 2015 15:39:12 GMT

your suggestion should work on fixed number of lines. i have edited the question. thx

Re: How can I parse 2 sets of CSVs in one file?

woodcock — Mon, 26 Oct 2015 16:11:56 GMT

The best thing to do is split the file on the outside of splunk.

Re: How can I parse 2 sets of CSVs in one file?

hylam — Mon, 26 Oct 2015 16:15:56 GMT

Are there any line numbers in splunk? Can a splunk search grep for "field[1-3]" and "field[4-7]" and expand a subsearch to "4"?

Re: How can I parse 2 sets of CSVs in one file?

woodcock — Mon, 26 Oct 2015 16:17:31 GMT

Like this:

| inputcsv TwoInOne.csv | rex "(?<filebreaker>SomRegExHere)" | streamstats current=t count(filebreaker) AS fileID | where fileID=0 outputcsv FirstHalf.csv
| inputcsv TwoInOne.csv | rex "(?<filebreaker>SomRegExHere)" | streamstats current=t count(filebreaker) AS fileID | where fileID=1 outputcsv SecondHalf.csv

| inputcsv FirstHalf.csv | append [|inputcsv SecondHalf.csv]

Note that this approach should work for any number of concatenated files, provided that filebreaker is the same.

Re: How can I parse 2 sets of CSVs in one file?

woodcock — Mon, 26 Oct 2015 16:24:26 GMT

There is an internal field called _serial which is a line-number. I will think more about this approach if my other answer does not work (let me know with followup comments).