Deployment Architecture

HALP! My *nix deployment server is propagating corrupted configuration files to my Windows clients!

hexx
Splunk Employee
Splunk Employee

I've noticed that the configuration files propagated by my Linux deployment server to my Windows deployment clients seem to be corrupted. When I open them with Notepad, everything is mashed up on a single line!

Example :

On my Linux deployment, the props.conf file I destined for my Windows forwarders looks like this :

[source::...\\MMEngine*.log]
sourcetype = mmengine_log
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}
SHOULD_LINEMERGE= false
DATETIME_CONFIG = \etc\system\local\datetime.xml
MAX_TIMESTAMP_LOOKAHEAD = 18
TIME_PREFIX = \d{4}\s

But once the file is deployed to the Windows forwarder, it looks like this when I open it with notepad :

[source::...\\MMEngine*.log]sourcetype = mmengine_logLINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}SHOULD_LINEMERGE= falseDATETIME_CONFIG = \etc\system\local\datetime.xmlMAX_TIMESTAMP_LOOKAHEAD = 18TIME_PREFIX = \d{4}\s

How can this be prevented?

1 Solution

hexx
Splunk Employee
Splunk Employee

This is likely to occur if the text files you are propagating to your Windows clients have been created and edited on a Linux box, and the reason is that Windows uses it's own newline character.

This difference can be exposed with "cat -etv". This command will expose "$" as the newline character used on *nix systems :

# cat -etv props.conf.linux
[source::...\\MMEngine*.log]$
sourcetype = mmengine_log$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}$
SHOULD_LINEMERGE= false$
DATETIME_CONFIG = \etc\system\local\datetime.xml$
MAX_TIMESTAMP_LOOKAHEAD = 18$
TIME_PREFIX = \d{4}\s$

...and will show "^M$" as the newline sequence used on Windows systems :

# cat -etv props.conf.windows
[source::...\\MMEngine*.log]^M$
sourcetype = mmengine_log^M$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}^M$
SHOULD_LINEMERGE= false^M$
DATETIME_CONFIG = \etc\system\local\datetime.xml^M$
MAX_TIMESTAMP_LOOKAHEAD = 18^M$
TIME_PREFIX = \d{4}\s^M$

The fix is a simple one : on your Linux deployment server, use the unix2dos command (http://en.wikipedia.org/wiki/Unix2dos) to convert the newline sequences of the configuration files destined to your Windows clients. This will make them readable in Notepad once they've been deployed.

Do note that Splunk (as well as most text editors other than Notepad) is "*nix-newline-aware" and will still honor the configuration parameters in files that appear mashed up on one line when opened in Notepad.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

It may not be fun using Notepad, but it works fine in Splunk. You can still edit the files on Windows using Wordpad, which can handle the unix line ends just fine.

hexx
Splunk Employee
Splunk Employee

This is likely to occur if the text files you are propagating to your Windows clients have been created and edited on a Linux box, and the reason is that Windows uses it's own newline character.

This difference can be exposed with "cat -etv". This command will expose "$" as the newline character used on *nix systems :

# cat -etv props.conf.linux
[source::...\\MMEngine*.log]$
sourcetype = mmengine_log$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}$
SHOULD_LINEMERGE= false$
DATETIME_CONFIG = \etc\system\local\datetime.xml$
MAX_TIMESTAMP_LOOKAHEAD = 18$
TIME_PREFIX = \d{4}\s$

...and will show "^M$" as the newline sequence used on Windows systems :

# cat -etv props.conf.windows
[source::...\\MMEngine*.log]^M$
sourcetype = mmengine_log^M$
LINE_BREAKER =([\r|\n])\d{4}\s\d{2}:\d{2}:\d{2}\.\d{3}^M$
SHOULD_LINEMERGE= false^M$
DATETIME_CONFIG = \etc\system\local\datetime.xml^M$
MAX_TIMESTAMP_LOOKAHEAD = 18^M$
TIME_PREFIX = \d{4}\s^M$

The fix is a simple one : on your Linux deployment server, use the unix2dos command (http://en.wikipedia.org/wiki/Unix2dos) to convert the newline sequences of the configuration files destined to your Windows clients. This will make them readable in Notepad once they've been deployed.

Do note that Splunk (as well as most text editors other than Notepad) is "*nix-newline-aware" and will still honor the configuration parameters in files that appear mashed up on one line when opened in Notepad.

piebob
Splunk Employee
Splunk Employee
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...