Limit in field extraction rule length or buggy web surface?

[splunkbeginner2]

Hello,

tried to create a rule, to extract all fields of a csv table. Unfortunately the field extractor (the new App) causes problems after a certain length of the RegEx). The problem is, that the input length in the browser is limited. Is there something more behind it and I am just using the field extractions in the wrong way? Or is it again just another really annoying bug in the splunk web surface?

(Like the problems caused, when you want to edit a long saved search and have problems with the input window that each time jumps back to its unbelievable tiny size and you have to use an external editor like notepad to edit the query.. or die in scrolling. (Splunk gots beaten by notepad. Thats hard - and not really a pro to buy it..!))

Kind regards,
Xantor

[kristian_kolb]

Normally, you wouldn't need regex extractions for CSV data. Have you looked at the DELIMS and FIELDS (in transforms.conf) combination with a REPORT (in props.conf)?

props.conf
[your_sourcetype]
REPORT-blah = my_csv_extractor

transforms.conf
[my_csv_extractor]
DELIMS = ","
FIELDS = field1, field2, field3 etc etc

/K

[martin_mueller · Accepted solution]

Splunk comes with an understanding of CSV data out of the box. Take a look at http://docs.splunk.com/Documentation/Splunk/6.1.1/Data/Extractfieldsfromfileheadersatindextime for a start.

[splunkbeginner2]

Hey, I am sorry it took me some time until I could review this problem. For this case its a good solution for the basic problem.
I think I should be able to use a FIELD_HEADER_REGEX to filter out all Headers, even though they might be repeated every 200 lines, or?

[splunkbeginner2]

Well it looks, like this is just a limit in the webinterface. Nevertheless: Is there a better way for field extractions of the data? Or is this acceptable. We will get about 20MB/Day in this logs, and the csvtable has 24 columns and each event has a length of about 220 chars.