topic Re: line_breaker help in Getting Data In

line_breaker help

renems — Mon, 25 Jan 2016 21:09:42 GMT

I'm struggling getting my data to break to events. A REST call gives me a csv in a long straight line, without any character to break. Any help how to fix this?

This is what it looks like now:

svc;Time_Stamp;Omgeving;name;Locatie;capacity;free_capacity;virtual_capacity;real_capacity;overalloc;type;Storage_ClassIVB201;2016/01/2518:00:13CET;OTA;B_IVB201_HUB196_1;Best;62603443306496;32838246203392;34303903793152;29762410301440;54;thin;BronsIVB800;2016/01/2518:00:15CET;PRD;B_IVB800_HUB193_1;Best;62602906435584;2785286291456;59817088641536;59817088641536;95;THICK;Brons

This is what would make me happy:

svc;Time_Stamp;Omgeving;name;Locatie;capacity;free_capacity;virtual_capacity;real_capacity;overalloc;type;Storage_Class
IVB201;2016/01/25 18:00:13 CET;OTA;B_IVB201_HUB196_1;Best;62603443306496;32838246203392;34303903793152;29762410301440;54;thin;Brons
IVB800;2016/01/25 18:00:15 CET;PRD;B_IVB800_HUB193_1;Best;62602906435584;2785286291456;59817088641536;59817088641536;95;THICK;Brons

Bonuspoints for the right way to get rid of the header record. 🙂

Re: line_breaker help

richgalloway — Mon, 25 Jan 2016 21:18:14 GMT

What REST call is giving you those results?

Re: line_breaker help

somesoni2 — Mon, 25 Jan 2016 21:28:43 GMT

Give this a try (assuming your value of svc field is 6 character long always)

[ <SOURCETYPE NAME> ]
SHOULD_LINEMERGE=false
disabled=false
LINE_BREAKER=([\r\n]*)(?=\w{6};\d+\/\d+)
TIME_FORMAT=%Y/%m/%d%H:%M:%S%Z
TIME_PREFIX=^\w{6};
SEDCMD-removeheader=s/^(svc;.*)//

Re: line_breaker help

renems — Mon, 25 Jan 2016 22:26:18 GMT

Thanks for the help. However, it's not the result I was hoping for. In fact, it doesn't seem to work.
Could it be because between the last word of the sentence, and the first one of the new one, there's no character in between? Any other suggestions?

Re: line_breaker help

renems — Mon, 25 Jan 2016 22:27:01 GMT

Yeah I know, it's not a very good one.

Re: line_breaker help

richgalloway — Tue, 26 Jan 2016 12:37:15 GMT

There could be something in your REST call that can be changed to improve your results, but it's impossible to know for sure without seeing the command.

Re: line_breaker help

alemarzu — Tue, 26 Jan 2016 13:56:48 GMT

Hi there Renems,

Try with this,

SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
SEDCMD=s/svc;.*?Storage_Class//g
LINE_BREAKER=([\r\n]*)(?:[A-Z]{1,}\d{1,})(?=;\d{4}\/\d{2}\/\d{4})