topic Re: Field Extraction in Getting Data In

Field Extraction

gjohnson — Fri, 04 Oct 2013 01:34:18 GMT

I am trying to extract a field from the following lines but the field extraction does not result in a Field. The sample data is a CSV download from a credit card site:

02/14/2013 Thu,,"PIZZA HUT # 023603 0MC LEAN VA ","GEORGE T JOHNSON","XXXX-XXXXXX-82003",,,28.75,,,,,,,
02/15/2013 Fri,,"BONSAI GRILL 1234560ARLINGTON VA ","GEORGE T JOHNSON","XXXX-XXXXXX-82003",,,32.21,,,,,,,
02/15/2013 Fri,,"EXXONMOBIL MCLEAN VA ","GEORGE T JOHNSON","XXXX-XXXXXX-82003",,,57.28,,,,,,,

The field I am looking for has the values of "28.75, 32.21, 57.28" values. I did an automated field extract by copying out the value, putting it in the sample and generating a regex - it successfully identifies the information I want (it is highlighted). I hit save then go back to searching, but the field doesn't extract.

I then tried copying the rex pattern that successfully identified the fields in the auto-generator (see the rex below) and I ran it right on the search bar - but it brought back a sparse return (there wasn't a price field for every line like there should be).

running this yields inconsistent results
... | rex "(?i)^(?:[^\-]*\-){3}\d+\",,,(?P[^,]+)"

Can anyone help me with the right regex for a field extraction that yields a "price" for every line in the file?

Re: Field Extraction

bizza — Fri, 04 Oct 2013 01:42:13 GMT

Hi gjohnson,
why you are not putting that csv in inputs.conf, defining a TRANSFORM in props.conf, defining a field DELIM on your transforms.conf and the assign the right field to the different column?

ciao

Re: Field Extraction

gjohnson — Fri, 04 Oct 2013 01:52:58 GMT

Sorry, I am not sure I know what you are saying. I did put the CSV in as a file input. I thought that the Field Extraction process would create a delimeter? Am I wrong?

Re: Field Extraction

bizza — Mon, 28 Sep 2020 14:53:30 GMT

something like this

inputs.conf

[monitor:///path/to/file.csv]
sourcetype = yoursourcetype
disabled = 0

props.conf

[sourcetype::yoursourcetype]
CHECK__FOR _HEADER=TRUE
SHOULD _LINEMERGE = false
TRANSFORM-transformfile = transf_csv

transforms.conf

[transf_csv]
DELIMS=","
FIELDS="field1", "field2", "field3", "field4", "fieldN"

Re: Field Extraction

somesoni2 — Fri, 04 Oct 2013 10:32:06 GMT

Could you please try below rex.

"(?i)^(?:[^,]*,){7}(?P[^,]+)"