I'm using curl in Spluk to download some data from an API and to build a lookup of the downloaded data. The data comes back as a single field value (curl_message). The first line is the effect field ...
See more...
I'm using curl in Spluk to download some data from an API and to build a lookup of the downloaded data. The data comes back as a single field value (curl_message). The first line is the effect field list for the lookup I am going to create and then there is data, of which one field MAY be multi-line. So in this example iNote iWine Type iUser Vintage Wine SortWine Locale Producer Varietal MasterVarietal Designation Vineyard Country Region SubRegion Appellation TastingDate Defective fAllowComments Views Name fHelpful fFavorite Rating EventLocation EventTitle iEvent EventDate EventEndDate TastingNotes fLikeIt CNotes CScore LikeVotes LikePercent Votes Comments cLabels
9537078 1519682 Red 94404 2012 Wolf Blass Black Label Wolf Blass Black Label Australia, South Australia Wolf Blass Cabernet-Shiraz Blend Red Blend Black Label Unknown Australia South Australia Unknown Unknown 11/27/2021 False True 17 username 0 93 True 4 92 2 1 0 0 237
9537066 2452851 White 94404 2014 Xanadu Chardonnay Reserve Margaret River Xanadu Chardonnay Reserve Margaret River Australia, Western Australia, South West Australia, Margaret River Xanadu Chardonnay Chardonnay Reserve Unknown Australia Western Australia South West Australia Margaret River 11/3/2021 False True 23 username 0 95 Seems to be improving. A perfect accompaniment to prosciutto True 6 92.8333333333333 6 1 0 0 35
9516281 2778467 White 94404 2016 Weingut Thörle Saulheimer Hölle Riesling trocken Thörle, Weingut Saulheimer Hölle Riesling trocken Germany, Rheinhessen Weingut Thörle Riesling Riesling trocken Saulheimer Hölle Germany Rheinhessen Unknown Unknown 11/28/2021 False True 135 username 0 93 Paired well with Barramundi and sweet potato fries
Colour: Pale gold
Nose: Medium
P: Lemon rind, peach, orange
S: cream
T: honey
Palate: Dry, high acidity, medium alcohol, full bodied, pronounced flavour, medium finish
P: pear, peach, lemon
S: bread
T: nutmeg, caramel
B: 1
L: .5
I: .5
C: 1
Very good wine True 5 91 3 1 0 0 45
9431031 3300231 Red 94404 2017 Girolamo Russo Etna 'a Rina Girolamo Russo Etna 'a Rina Italy, Sicily, Etna DOC Girolamo Russo Nerello Blend Nerello Mascalese 'a Rina Unknown Italy Sicily Unknown Etna DOC 10/2/2021 False True 65 username 0 93 True 50 90.2888888888889 28 0.964285714285714 0 0 378
9431030 3580970 Red 94404 2019 Swinging Bridge Shiraz William J. Swinging Bridge Shiraz William J. Australia, New South Wales, Central Ranges, Orange Swinging Bridge Shiraz Syrah William J. Unknown Australia New South Wales Central Ranges Orange 9/11/2021 False True 0 username 0 92 True 1 92 1 1 0 0 10
Primary Black cherry, liquorice, dried herbs, black pepper, black olive, blackberry
9431025 3157557 Red - Sparkling 94404 2008 Seppelt Shiraz Show Sparkling Great Western Seppelt Shiraz Show Sparkling Great Western Australia, Victoria, Western Victoria, Great Western Seppelt Shiraz Syrah Show Sparkling Unknown Australia Victoria Western Victoria Great Western 10/15/2021 False True 252 username 0 95 Deep ruby, pronounced nose, lots of jammy red and black fruits, bubbles washing inside the mouth, filling the mouth with flavour. The finish lingering forever. True 5 93.25 3 1 0 0 28 there is the header (starting iNote...) and then 6 data 'rows' that need to be expanded. I have used | makemv tokenizer="(.*)\n" curl_message but when the 'TastingNotes' field is multi-line, as in the line starting 9516281, then of course that fails to extract the multi-line value. In that case, the tasting note should be from (Paired well... to Very good wine) I have tried playing with rex and max_match=0. I know that a valid line starts with a (currently) 7 digit number, so I know I will never have that in the tasting note text, but I can't figure out what the correct regex might be. I got as far as (?s)((iNote|^\d{7}).*?)^\d{7} but don't know how to exclude the end match part, which is the start of the next entry and in any case that doesn't work as the tokenizer regex I don't want the data to go to an index, so I could write a scripted input that get the data and uses sed/awk to break out the events then REST API to create the lookup, but that seems like overkill.