How to change DELIMS in transforms.conf to use multiple white space?

[DeronJensen]

We have a system that sends logs with extra white space, causing everything to be off by 1. When the day of month changes from 1 digit to 2 digits, the proxy device sends the log with:

The first example has 2 spaces: Nov 3 and the second has 1 space: Oct 31. When there are 2 spaces between the month and day, all of the fields are off by one.

Nov  3 16:27:47 10.1.126.214.100 2015-11-03 16:27:47 15 10.2.172.212 404 ...
Oct 31 23:59:59 10.1.126.99.100 2015-10-31 23:59:59 227 10.2.189.178 407 ...

My transforms.conf is as follows:

DELIMS = " "
FIELDS = month,date,time,dvc,date1,time-taken,duration,src,status,...

I was thinking about changing to a REGEX, but this seems to be a little more trouble. Is there an easy way to do this? I would like to do: DELIMS = "\s+" or possibly do something to say "skip empty values". I do not know if this is possible, or is there a simple REGEX that will work?

[gcato]

Hi DeronJensen,

I tested DELIMS with \s+ and confirmed it will not work. You'd need to use REGEX instead to use \s+ notation but then FIELDS will not work. A simpler way would be to normalise the incoming data so that multi-whitespaces become one. You can easily do this using the SEDCMD in props.conf. In this case the following should work,

[my_sourectype]
...
SEDCMD-single-whitepace = s/\s\s*/ /g

This will only work on new data. Hope this helps.