Archive
Highlighted

Transform on extracted field

Explorer

Hi. I'm using OSSEC to send logs from a variety of sources by syslog to Splunk. I've created transforms to extract the fields, the main one of which gets extracted as "message". One of the log sources is ISA Server, and I'm trying to do some extractions on those fields. I've got a transform to change the "host" to the original host, not the OSSEC server. Everything has the same sourcetype, which I tried to change using a "[host::" entry in props.conf, but this failed because, I think, it's looking at the original host, not the transformed one.

I can get an extraction to work against the "_raw" field, by not against the "message" field. Is it possible to get an extract to work against a field extracted an search time? I've tried referencing the field as "message" and "field:message", but the extract fails. I can get a more accurate regex if I can reference this field.

Alternatively, and this might be better, is to use a delimiter transform rather than regex if I can get that to work on the "message" field, but I need to be able selectively target just the ISA logs.

Hope this makes some sense.

Thanks

Tags (2)
0 Karma
Highlighted

Re: Transform on extracted field

Legend

Your question makes sense - but a lot of us are not familiar with OSSEC or the ISA Server logs. A few lines of the log would be very helpful! Otherwise, only a subset of the folks on this forum will be able to answer your question.

Also, the relevant props.conf and transforms.conf stanzas would be helpful as well.

0 Karma
Highlighted

Re: Transform on extracted field

Motivator

I would first recommend checking the regex from the search bar to validate the field extraction works. Then, once working, you could move it to props and transforms.

i.e.

your search | rex field=message (?<my_field>my capture pattern here)
0 Karma
Highlighted

Re: Transform on extracted field

Explorer

Hi. my regex works fine, using rex and as a transform, if I run it on the "_raw" field. It runs fine using rex against the extracted "message" field. I just can't get it to run as a transform against the "message" field.

0 Karma
Highlighted

Re: Transform on extracted field

Legend

A-ha! What you are asking is to create a new field based on an existing field - when both fields are extracted at search time!

AFAIK, this is not possible in props.conf or transforms.conf - but you can do it with the rex command, as Rob illustrates above.

0 Karma
Highlighted

Re: Transform on extracted field

Explorer

Hi. OSSEC is a log server, which collects logs and forwards alerts using syslog. ISA logs look like this as they are forwarded, the "message" field is tab-delimited:

Feb 3 08:40:19 log-01 ossec: crit=3
id=108001 description="ISA Internet
access permitted" component="(isa01)
172.19.1.1->\ISALogs\ISALOG20140203WEB000.w3c",
classification=" proxy,allowed,",
src
ip="172.19.2.2", acct="anonymous",
message="172.19.2.2 anonymous ... 0x800
Allowed"

Sorry, ISA Logs are too long to post here in full ...

0 Karma
Highlighted

Re: Transform on extracted field

Explorer

In props.conf I had:

[ossec_alerts]
REPORT-ossec-isa = ossec-splunk-isa

in transforms.conf I tried a number of permutations, but what I want is:

[ossec-splunk-isa]
SOURCEKEY = field:message
REGEX = ^\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\t[^\t]+\t(?P.*?)\t\d{4}-\d{2}-\d{2}\t\d{2}:\d{2}:\d{2}\t[^\t]+\t[^\t]+\t(?P<r
host>[^\t]+)\t\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\t

thanks

0 Karma