Archive

Transform to switch source field path separator from "/" to "\"

mjohanne
Explorer

I am trying to transform the source field from using Unix path separator (/) to Windows path separator (\).

For example:

from: /opt/cwtsplunk/SplunkSinkhole/mspcwtwvm030/D/Aqua/Server/Diags/ID_337/2010-09-28/TravelAdvisor/uci-2010-09-28-337.log  
to:   D:\Aqua\Server\Diags\ID_337\2010-09-28\TravelAdvisor\uci-2010-09-28-337.log

This transform:

  [reformat_sinkhole_source]
  SOURCE_KEY = MetaData:Source
  REGEX = (.*/D/)(.*)
  DEST_KEY = MetaData:Source
  FORMAT = source::D:/$2

transforms to: D:/Aqua/Server/Diags/ID_337/2010-09-28/TravelAdvisor/uci-2010-09-28-337.log

Close except for the path separator. Suggestions?

Lowell
Super Champion

I don't think there is a great way to do this. The problem is that you can't issue a repeating regex (sed-like) transformation on the source field. If this were some value within your raw event, then using SEDCMD-flip-slash = s/\//\\/g would do the trick, but this cannot be applied to the "source" field and therefore can't be used.

So the next best option would be to either have a fixed-length transformer. (In your example, you have a 7-directory path, but I'm assuming that not all of your files are setup that way.) Or setup a transformer that fixes up one slash at a time and call it a bunch of times. Here's what I mean.

props.conf:

[my_sourcetype]
TRANSFORMS-source = reformat_sinkhole_source, u2w, u2w, u2w, u2w, u2w, u2w, u2w

transforms.conf:

[reformat_sinkhole_source]
SOURCE_KEY = MetaData:Source
REGEX = /.*?/([A-Z])/(.*)$
DEST_KEY = MetaData:Source
FORMAT = source::$1:\$2

[u2w]
# Convert a unix-style path to a windows-style path.  Must call this transformer multiple times, once per slash
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = ^(.*?)/(.*)$
FORMAT = source::$1\$2

I've updated your reformat_sinkhole_source transformer slightly. It takes care of converting one of your slashes for you. It also pulls out the drive letter automatically. (This could get you into trouble if you have other single-letter entries in your path, so double check on that).

Keep in mind that you'll have to add one more "u2w" transformer for each directory that you have, or you'll end up with a half-converted path.


On the other hand, are the forward-slashes all that bad? I've always found the windows-style backslashes to be a pain. Especially since you have to escape them in the search language. For example, this search will not work:

source="D:\Aqua\Server\Diags\ID_337\2010-09-28\TravelAdvisor\uci-2010-09-28-337.log"

You have to change it to:

source="D:\\Aqua\\Server\\Diags\\ID_337\\2010-09-28\\TravelAdvisor\\uci-2010-09-28-337.log"

But however you want it to look. Best of luck.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I think you could use props.conf:

[mysourcetype]
TRANSFORMS-reformat-sinkhole = reformat_sinkhole_source
SEDCMD-reformat-sinkhole = s/\//\\/g

You may be able to use

SEDCMD-reformat-sinkhole = y/\//\\/g

but I'm not sure. And in transforms.conf I would change slightly to:

[reformat_sinkhole_source]
SOURCE_KEY = MetaData:Source
REGEX = .*?/D/(.*)
DEST_KEY = MetaData:Source
FORMAT = source::D:/$1

gkanapathy
Splunk Employee
Splunk Employee

I guess it doesn't. Never mind.

0 Karma

Lowell
Super Champion

Does SEDCMD work on the source field? I thought that only updates the raw content of the event.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!