Deployment Architecture

HALP! My *nix deployment server is propagating corrupted configuration files to my Windows clients!

hexx
Splunk Employee
Splunk Employee

I've noticed that the configuration files propagated by my Linux deployment server to my Windows deployment clients seem to be corrupted. When I open them with Notepad, everything is mashed up on a single line!

Example :

On my Linux deployment, the props.conf file I destined for my Windows forwarders looks like this :

[source::...\\MMEngine*.log]
sourcetype = mmengine_log
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}
SHOULD_LINEMERGE= false
DATETIME_CONFIG = \etc\system\local\datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 18
TIME_PREFIX = \d{4}\s

But once the file is deployed to the Windows forwarder, it looks like this when I open it with notepad :

[source::...\\MMEngine*.log]sourcetype = mmengine_logLINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}SHOULD_LINEMERGE= falseDATETIME_CONFIG = \etc\system\local\datetime.xmlMAX_TIMESTAMP_LOOKAHEAD = 18TIME_PREFIX = \d{4}\s

How can this be prevented?

1 Solution

hexx
Splunk Employee
Splunk Employee

This is likely to occur if the text files you are propagating to your Windows clients have been created and edited on a Linux box, and the reason is that Windows uses it's own newline character.

This difference can be exposed with "cat -etv". This command will expose "$" as the newline character used on *nix systems :

# cat -etv props.conf.linux
[source::...\\MMEngine*.log]$
sourcetype = mmengine_log$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}$
SHOULD_LINEMERGE= false$
DATETIME_CONFIG = \etc\system\local\datetime.xml$
MAX_TIMESTAMP_LOOKAHEAD = 18$
TIME_PREFIX = \d{4}\s$

...and will show "^M$" as the newline sequence used on Windows systems :

# cat -etv props.conf.windows
[source::...\\MMEngine*.log]^M$
sourcetype = mmengine_log^M$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}^M$
SHOULD_LINEMERGE= false^M$
DATETIME_CONFIG = \etc\system\local\datetime.xml^M$
MAX_TIMESTAMP_LOOKAHEAD = 18^M$
TIME_PREFIX = \d{4}\s^M$

The fix is a simple one : on your Linux deployment server, use the unix2dos command (http://en.wikipedia.org/wiki/Unix2dos) to convert the newline sequences of the configuration files destined to your Windows clients. This will make them readable in Notepad once they've been deployed.

Do note that Splunk (as well as most text editors other than Notepad) is "*nix-newline-aware" and will still honor the configuration parameters in files that appear mashed up on one line when opened in Notepad.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

It may not be fun using Notepad, but it works fine in Splunk. You can still edit the files on Windows using Wordpad, which can handle the unix line ends just fine.

hexx
Splunk Employee
Splunk Employee

This is likely to occur if the text files you are propagating to your Windows clients have been created and edited on a Linux box, and the reason is that Windows uses it's own newline character.

This difference can be exposed with "cat -etv". This command will expose "$" as the newline character used on *nix systems :

# cat -etv props.conf.linux
[source::...\\MMEngine*.log]$
sourcetype = mmengine_log$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}$
SHOULD_LINEMERGE= false$
DATETIME_CONFIG = \etc\system\local\datetime.xml$
MAX_TIMESTAMP_LOOKAHEAD = 18$
TIME_PREFIX = \d{4}\s$

...and will show "^M$" as the newline sequence used on Windows systems :

# cat -etv props.conf.windows
[source::...\\MMEngine*.log]^M$
sourcetype = mmengine_log^M$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}^M$
SHOULD_LINEMERGE= false^M$
DATETIME_CONFIG = \etc\system\local\datetime.xml^M$
MAX_TIMESTAMP_LOOKAHEAD = 18^M$
TIME_PREFIX = \d{4}\s^M$

The fix is a simple one : on your Linux deployment server, use the unix2dos command (http://en.wikipedia.org/wiki/Unix2dos) to convert the newline sequences of the configuration files destined to your Windows clients. This will make them readable in Notepad once they've been deployed.

Do note that Splunk (as well as most text editors other than Notepad) is "*nix-newline-aware" and will still honor the configuration parameters in files that appear mashed up on one line when opened in Notepad.

piebob
Splunk Employee
Splunk Employee
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...