topic Fields with and without spaces in Splunk Search

Fields with and without spaces

kmattern — Tue, 04 Jun 2013 18:31:01 GMT

On my portal I have Solaris web logs from which I must extract file names that were downloaded by the end user. These files come from many different sources, individuals and companies. Sometimes the filenames have spaces in them and sometimes they do not. They are intermixed in the logs.

These filenames have spaces in them:

/krbox/resources/SW CH 32 0 2012-11-16.pdf
/krbox/system/platform/krbox304b/Attachment 4 111026M123B.pdf
/krbox/system/platform/krbox304b/Attachment 1 111026M121C-704.pdf
/krbox/system/platform/krbox304b/Attachment 6 Functional Test Logs.zip

While these do not:

/krbox/reference/publications/XARP/cnf/tb11-5895-1893-13.pdf
/krbox/system/platform/krbox304b/Attachment_7_D0206-001_KRBOX-304B_RS1Testing.pdf
/krbox/system/platform/krbox304b/Attachment_1_111026M121C-704.pdf

This is how they might look in the actual log:

/krbox/reference/publications/XARP/cnf/tb11-5895-1893-13.pdf
/krbox/resources/SW CH 32 0 2012-11-16.pdf
/krbox/system/platform/krbox304b/Attachment_7_D0206-001_KRBOX-304B_RS1Testing.pdf
/krbox/system/platform/krbox304b/Attachment 1 111026M121C-704.pdf
/krbox/system/platform/krbox304b/Attachment 6 Functional Test Logs.zip
/krbox/system/platform/krbox304b/Attachment_1_111026M121C-704.pdf
/krbox/system/platform/krbox304b/Attachment 4 111026M123B.pdf

I normally use the slash “/” as the delimiter in multi value fields to extract the filename. How can I extract the filenames regardless of the spaces?

Re: Fields with and without spaces

bmacias84 — Tue, 04 Jun 2013 20:41:10 GMT

Due to the complexity of the field you will have to use regex. You can use a transform, inline rex, or inline regex extract your field.

Below I start matching at end of line (end of field in this case) then create a not capture group for zip or pdf extentions then match everything expect for forward slash.

Example using rex:



…| rex field=<your_field> "(?<filename>[^/]+.(?:zip|pdf)$)"| table filename

Additional Reading:

Hope this help or gets you started.

Cheers,

Re: Fields with and without spaces

Gilberto_Castil — Tue, 04 Jun 2013 20:51:55 GMT

This extraction works at search time. In this case we capture anything at the end of the string ($) and the slash (/) delimiter.

sourcetype="answers-1370378440" | rex field=_raw "/(?< filename>[a-zA-Z0-9\s_\.-]+)$"

If the sample data is partial and you have the file name as a whole in a field, called filename, then you would use it like this:

sourcetype="answers-1370378440" | rex field=filename "/(?< filename>[a-zA-Z0-9\s_\.-]+)$"

And, of course you can automate this with an entry in props.conf:

[answers-1370378440]
EXTRACT-filename = /(?< filename>[a-zA-Z0-9\s_\.-]+)$

PS: There is a space in <filename> due to the markup.