Extracting path and file name separately using regex

[telcosi]

Hi - we have some data that contains a hierarchy of folders and computer name that we want to extract.

The raw data looks like this:

Time=2014.06.12 11:04:03.772 EST, Agent=/root/SiteHome/FirstFolder/605c6f6a-145d051f, Id=68.2, Watchlist=JW1

and from the bit that starts with "Agent=" and ends with ", Id=" we would like to extract the folders and the computer ID into two different fields.

There can be any number (from 1 to any number - never zero) of folders and the computer ID can have any non-special (/ \ * , . " @) characters.

The folders are up to the last slash and the computer ID is from the last slash up to the comma

We would like 2 regex for use in the Field Extractor - can someone please help ?

Thanks

John

[s2_splunk · Accepted solution]

Your log format should already give you a field called 'Agent', so this should break that field into the two components (if I understood your goal):

rex field=Agent ".*\/(?<ComputerID>.*?)$" | rex field=Agent "^(?<Folder>.*)/"

or both fields in a single rex command, like so:

rex field=Agent "^(?<Folder>.*\/)(?<ComputerID>.*?)$" 

[s2_splunk]

No problem. Strange that you didn't see the computerID, I had tested it with your raw data sample to make sure it works. Oh well, glad you got it working!

[telcosi]

Thanks - get_agent and get_folder worked well. get_computer returned no values so I blundered my way around and found that this worked okay ([^/]*)$

Closing this with my thanks !

[s2_splunk]

NOTE: For some reason, it changes the 'A' of the second 'agent' in the first REGEX to lowercase when I comment. Make sure the case matches on all references to fields.

[s2_splunk]

Please read up on props/transforms; you are very likely not dumb, this stuff takes a little practice. 😉
This works for me:

1) props.conf
[mySourcetype]
REPORT-AgentFields = get_agent, get_folder, get_computer

2) transforms.conf

[get_agent]
REGEX = Agent=(?P[^,]+)
FORMAT = Agent::$1

[get_folder]
REGEX = ^(.*)/
SOURCE_KEY = Agent
FORMAT = folder::$1

[get_computer]
REGEX = ./(.?)$
SOURCE_KEY = Agent
FORMAT = computerID::$1

It probably can be simplified a bit more, but I hope you get the idea.

[telcosi]

Hi Again - I must be dumb but I cannot figure out how to do this extraction in the UI or props.conf. How do I convert the (working) rex commands to something recognised by Field Extractions in the UI or in props.conf ?

[s2_splunk]

You can attach the extraction to your sourcetype/source/host, either via the UI (Settings->Fields->Field Extractions) or by directly editing props.conf.

Details: http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Addfieldsatsearchtime

[Please accept answer if it solved your problem]

[telcosi]

That is great - how do we then persist that field definition in (or in the same way as) the field extractor ? I have tested it in the search bar but want to keep it available for all searches ? Thanks !