topic Re: Regex & File Directory Path in Splunk Search

Regex & File Directory Path

behymejt2012 — Mon, 04 Nov 2013 13:44:53 GMT

Hi Everyone,

Need a little help with regexing out a portion of a directory path. The examples below are the current structure of paths that I have (the user account is always 9 digits long):

c:\users\123456789\desktop\

c:\users\123456789\documents\

c:\users\123456789\desktop\files

c:\users\987654321\pictures\

the goal is to remove the section "123456789" section from the directory.

Any help or guidance would be greatly appreciated.

Re: Regex & File Directory Path

behymejt2012 — Mon, 04 Nov 2013 13:45:47 GMT

Looks like the submission took out all the \

Re: Regex & File Directory Path

alacercogitatus — Mon, 04 Nov 2013 13:59:10 GMT

Not sure what you want to do with the section of numbers.But maybe something like this:

your_search |rex field=_raw "C:\\users\\(?<user_id>\d{9})\\" | stats count by user_id

Does that help?

Re: Regex & File Directory Path

martin_mueller — Mon, 04 Nov 2013 14:21:43 GMT

Is this the source field, and you want to anonymize the user id?

Re: Regex & File Directory Path

behymejt2012 — Mon, 04 Nov 2013 14:31:28 GMT

Not exactly. ideally it would be best to only have everything after user/123456789.

Re: Regex & File Directory Path

behymejt2012 — Mon, 04 Nov 2013 14:38:52 GMT

@martin_muller that would work as well. The goal is to hide/anonymize the users id

Re: Regex & File Directory Path

gfuente — Mon, 04 Nov 2013 14:39:09 GMT

Hello

Try this regex:

\w\:\\users\\\d+(?<yourfield>\\[^\r]*)$

Maybe you need to substitute the \r of the end of the rex by \n or \R

Regards

Re: Regex & File Directory Path

martin_mueller — Mon, 04 Nov 2013 14:45:55 GMT

In order to anonymize the source field you need to change index-time settings in props.conf and transforms.conf. Here's the relevant doc: http://docs.splunk.com/Documentation/Splunk/latest/Data/Overridedefaulthostassignments (Note: This works with the host field, but the approach is the same.

As for the specifics, you'll likely need a custom REGEX and FORMAT like this:

REGEX = ^source::(.*)/\d{9}(.*)$
FORMAT = source::$1$2

I didn't test these expressions.

Re: Regex & File Directory Path

alacercogitatus — Mon, 04 Nov 2013 15:06:04 GMT

So to remove data do this in props and transforms, requires restart.

props.conf [source::C:\\users\\*] TRANSFORMS-hide = hide_users_path

transforms.conf [hide_users_path] REGEX = source::C:\\users\\\d+\\(.*) FORMAT = source::$1 SOURCE_KEY = Metadata:Source DEST_KEY = Metadata:Source WRITE_META = TRUE