HALP! My *nix deployment server is propagating corrupted configuration files to my Windows clients!

[hexx]

I've noticed that the configuration files propagated by my Linux deployment server to my Windows deployment clients seem to be corrupted. When I open them with Notepad, everything is mashed up on a single line!

Example :

On my Linux deployment, the props.conf file I destined for my Windows forwarders looks like this :

[source::...\\MMEngine*.log]
sourcetype = mmengine_log
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}
SHOULD_LINEMERGE= false
DATETIME_CONFIG = \etc\system\local\datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 18
TIME_PREFIX = \d{4}\s

But once the file is deployed to the Windows forwarder, it looks like this when I open it with notepad :

[source::...\\MMEngine*.log]sourcetype = mmengine_logLINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}SHOULD_LINEMERGE= falseDATETIME_CONFIG = \etc\system\local\datetime.xmlMAX_TIMESTAMP_LOOKAHEAD = 18TIME_PREFIX = \d{4}\s

How can this be prevented?

[gkanapathy]

It may not be fun using Notepad, but it works fine in Splunk. You can still edit the files on Windows using Wordpad, which can handle the unix line ends just fine.

[hexx · Accepted solution]

This is likely to occur if the text files you are propagating to your Windows clients have been created and edited on a Linux box, and the reason is that Windows uses it's own newline character.

This difference can be exposed with "cat -etv". This command will expose "$" as the newline character used on *nix systems :

# cat -etv props.conf.linux
[source::...\\MMEngine*.log]$
sourcetype = mmengine_log$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}$
SHOULD_LINEMERGE= false$
DATETIME_CONFIG = \etc\system\local\datetime.xml$
MAX_TIMESTAMP_LOOKAHEAD = 18$
TIME_PREFIX = \d{4}\s$

...and will show "^M$" as the newline sequence used on Windows systems :

# cat -etv props.conf.windows
[source::...\\MMEngine*.log]^M$
sourcetype = mmengine_log^M$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}^M$
SHOULD_LINEMERGE= false^M$
DATETIME_CONFIG = \etc\system\local\datetime.xml^M$
MAX_TIMESTAMP_LOOKAHEAD = 18^M$
TIME_PREFIX = \d{4}\s^M$

The fix is a simple one : on your Linux deployment server, use the unix2dos command (http://en.wikipedia.org/wiki/Unix2dos) to convert the newline sequences of the configuration files destined to your Windows clients. This will make them readable in Notepad once they've been deployed.

Do note that Splunk (as well as most text editors other than Notepad) is "*nix-newline-aware" and will still honor the configuration parameters in files that appear mashed up on one line when opened in Notepad.

[piebob]

here is a related question/answer:
http://answers.splunk.com/questions/5731/line-endings-in-an-apps-config-files