moving rex to props and transforms not woking

[sajeshpp]

Hi,
I am monitoring print events from windows event logs using WinEventLog:Microsoft-Windows-PrintService/Operational.
inputs.conf
[WinEventLog:Microsoft-Windows-PrintService/Operational]
disabled = 0
start_from = oldest
current_only = 0
index = index1
sourcetype = winprint

The events getting indexed and fields are extracted as key value pair in logs.

06/19/2017 09:56:22 AM
LogName=Microsoft-Windows-PrintService/Operational
SourceName=Microsoft-Windows-PrintService
EventCode=307
EventType=4
Type=Information
ComputerName=hostname.net
User=NOT_TRANSLATED
Sid=sid
SidType=0
TaskCategory=Printing a document
OpCode=Spooler Operation Succeeded
RecordNumber=541
Keywords=Document Print Job, Classic Spooler Event

Message="Document 4, TRAVEL EXPENSE STATEMENT.pdf owned by user1 on \host1 was printed on P-1234567 through port p-1234567.server.com. Size in bytes: 1695734. Pages printed: 3. No user action is required."

I need to extract the values from Message filed and assigned to new fields and it is working using below search with rex command.

index = index1| rex field=Message "(?:\w+\s\d+,\s)(?.+)(?:\sowned\sby\s)(?\w+)(?:\son.+printed\son\s)(?\w+-\d+)(?:.+Size\s\w+\s\w+:\s)(?\d+)(?:.\s\w+\s\w+:\s)(?\d+)"

I need this field extraction to be done with props and transforms instead of rex.

props.conf
[winprint]
REPORT-field_ext = extfield

transforms.conf
[extfield]
SOURCE_KEY = Message
MV_ADD = true
REGEX = "(?:\w+\s\d+,\s)(?.+)(?:\sowned\sby\s)(?\w+)(?:\son.+printed\son\s)(?\w+-\d+)(?:.+Size\s\w+\s\w+:\s)(?\d+)(?:.\s\w+\s\w+:\s)(?\d+)"

None of the fields are extracting with this props and transforms but works with rex.

Can some one please suggest what is the issue with this configuration and help to resolve the issue.

Thanks,
Sajesh

[sbbadri]

Try this,

props.conf
[winprint]
REPORT-field_ext = extfield

transforms.conf
[extfield]
REGEX = Message=\"(\S+\s\d+\,\s\S+.)\sowned\sby\s(\w+)\son\s(\S+.)\sSize\s\S+\s\S+\s(\d+).\s+Pages\sprinted:\s(\d+).\s\S+.*.\"
FORMAT = docname::$1 user::$2 printer::$3 size::$4 pages::$5

[sajeshpp]

Tried these settings. but I cannot find the fields extracted.

[DalJeanis]

@sajeshpp - Be sure to mark your code as code. Otherwise, the interface will strip some tags out of it - words in angle brackets and so on < >

[sajeshpp · Accepted solution]

I suspect the issue with using transforms is, The filed "Message" that I am using as source key is extracted during search time. But you can use a field as source key if it is indexed field.

So, I got my requirement met by doing inline extraction instead of transforms.
I have used below settings in props to extract and it extracted required fields.

EXTRACT-printfields= Message="\w+\s\d+,\s(?.+)\sowned\sby\s(?\w+)\son.+printed\son\s(?\w+-\d+).+Size\s\w+\s\w+:\s(?\d+).\s\w+\s\w+:\s(?\d+)

[DalJeanis]

Sounds correct. Please accept your answer, so the question will show as answered.

[cpetterborg]

Why are you wanting to do the field extractions at index time? That defeats one of the key features of Splunk - the ability to apply different views on the same data because the field extractions happen at search time instead of index time.

If you want to extract the fields automatically, that is easy and much better than at index time. Use the Field Extraction Tool if you want, but since you have such a good regular expression already, just add the new field extraction from the Settings -> Fields -> Field extractions page.

If you use the Windows TA, it will do the field extractions at search time. By default, fields are not extracted at index time in Splunk - there is a reason for that - a very good one.

[sajeshpp]

As I am using REPORT- in props.conf, it will do search time field extraction not index time ?
I am already using Windows TA. But it does not extract these fields.

[sbbadri]

Try this,

props.conf
[winprint]
REPORT-field_ext = extfield

transforms.conf
[extfield]
REGEX = Message=\"(\S+\s\d+\,\s\S+.)\sowned\sby\s(\w+)\son\s(\S+.)\sSize\s\S+\s\S+\s(\d+).\s+Pages\sprinted:\s(\d+).\s\S+.*.\"
FORMAT = docname::$1 user::$2 printer::$3 size::$4 pages::$5

[WalshyB]

Regex does not require it to be wrapped in quotes, it may be thinking that is part of the string?

Also have you checked the metadata to ensure you are able to see the extraction?
local.meta
[]
access = read : [*], write : [admin,...]
export = system

[sajeshpp]

-bash-4.1$ cat transforms.conf
[extfield]
SOURCE_KEY = Message
MV_ADD = true
REGEX = (?:\w+\s\d+,\s)(?.+)(?:\sowned\sby\s)(?\w+)(?:\son.+printed\son\s)(?\w+-\d+)(?:.+Size\s\w+\s\w+:\s)(?\d+)(?:.\s\w+\s\w+:\s)(?\d+)

[adonio]

use the field extractor with gui.
check the props and transforms it created and then apply.
also, do you use the Splunk TA for Windows?

[sajeshpp]

I am using TA for windows. But that didn't help.
Tried field extractor with GUI. It did not create props and transforms instead it created inline field EXRACT entry in $SPLUNK_HOME/etc/users//search/local/props.conf.

EXTRACT-docname,user,printer,size,pages = \n(?:Message=")(?:\w+\s\d+,\s)(?.+)(?:\sowned\sby\s)(?\w+)(?:\son.+printed\son\s)(?\w+-\d+)(?:.+Size\s\w+\s\w+:\s)(?\d+)(?:.\s\w+\s\w+:\s)(?\d+)

The fields are extracting with this settings, but I am not sure if this the recommended method to use it.
Is there any way we can convert this into transforms ?

[sajeshpp]

I have removed the quotes from REGEX. still no luck.

here is my metadata.
local.meta
[]
access = read : [ * ], write : [ admin ]
export = system