topic Re: Transform on extracted field in Knowledge Management

Transform on extracted field

pixelseventy2 — Thu, 30 Jan 2014 16:22:10 GMT

Hi. I'm using OSSEC to send logs from a variety of sources by syslog to Splunk. I've created transforms to extract the fields, the main one of which gets extracted as "message". One of the log sources is ISA Server, and I'm trying to do some extractions on those fields. I've got a transform to change the "host" to the original host, not the OSSEC server. Everything has the same sourcetype, which I tried to change using a "[host::" entry in props.conf, but this failed because, I think, it's looking at the original host, not the transformed one.

I can get an extraction to work against the "_raw" field, by not against the "message" field. Is it possible to get an extract to work against a field extracted an search time? I've tried referencing the field as "message" and "field:message", but the extract fails. I can get a more accurate regex if I can reference this field.

Alternatively, and this might be better, is to use a delimiter transform rather than regex if I can get that to work on the "message" field, but I need to be able selectively target just the ISA logs.

Hope this makes some sense.

Thanks

Re: Transform on extracted field

lguinn2 — Thu, 30 Jan 2014 17:14:11 GMT

Your question makes sense - but a lot of us are not familiar with OSSEC or the ISA Server logs. A few lines of the log would be very helpful! Otherwise, only a subset of the folks on this forum will be able to answer your question.

Also, the relevant props.conf and transforms.conf stanzas would be helpful as well.

Re: Transform on extracted field

bandit — Fri, 31 Jan 2014 05:57:44 GMT

I would first recommend checking the regex from the search bar to validate the field extraction works. Then, once working, you could move it to props and transforms.

i.e.

your search | rex field=message (?<my_field>my capture pattern here)

Re: Transform on extracted field

pixelseventy2 — Wed, 05 Feb 2014 13:46:37 GMT

Hi. my regex works fine, using rex and as a transform, if I run it on the "_raw" field. It runs fine using rex against the extracted "message" field. I just can't get it to run as a transform against the "message" field.

Re: Transform on extracted field

pixelseventy2 — Mon, 28 Sep 2020 15:48:30 GMT

Hi. OSSEC is a log server, which collects logs and forwards alerts using syslog. ISA logs look like this as they are forwarded, the "message" field is tab-delimited:

Feb 3 08:40:19 log-01 ossec: crit=3
id=108001 description="ISA Internet
access permitted" component="(isa01)
172.19.1.1->\ISALogs\ISALOG_20140203_WEB_000.w3c",
classification=" proxy,allowed,",
src_ip="172.19.2.2", acct="anonymous",
message="172.19.2.2 anonymous ... 0x800
Allowed"

Sorry, ISA Logs are too long to post here in full ...

Re: Transform on extracted field

pixelseventy2 — Mon, 28 Sep 2020 15:48:33 GMT

In props.conf I had:

[ossec_alerts]
REPORT-ossec-isa = ossec-splunk-isa

in transforms.conf I tried a number of permutations, but what I want is:

[ossec-splunk-isa]
SOURCE_KEY = field:message
REGEX = ^\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\t[^\t]+\t(?P.*?)\t\d{4}-\d{2}-\d{2}\t\d{2}:\d{2}:\d{2}\t[^\t]+\t[^\t]+\t(?P[^\t]+)\t\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}\t

thanks

Re: Transform on extracted field

lguinn2 — Tue, 11 Feb 2014 23:57:51 GMT

A-ha! What you are asking is to create a new field based on an existing field - when both fields are extracted at search time!

AFAIK, this is not possible in props.conf or transforms.conf - but you can do it with the rex command, as Rob illustrates above.