Why isn't this transforming?

howyagoin — Sun, 10 Jul 2011 03:07:41 GMT

I've got some Active Directory logs which are CSV that I'm trying to split apart into appropriate fields. The header of the .csv.zip files reads:

RecordNumber,TimeGenerated,EventID,EventType,EventTypeName,EventCategory,EventCategoryName,SourceName,Strings,SID

And I've created an inputs.conf for the directory where the files are that reads:

[monitor:///logs/test]
disabled = false
crcSalt = <SOURCE>
followTail = 0
sourcetype = ad-logs
index = test
host = ad

In my props.conf I have:

[ad-logs]
CHECK_FOR_HEADER = false
TRANSFORMS-ad = ad-csv

I've tried this with and without the check-for-header defined..

And in transforms.conf:

[ad-csv]
DELIMS = ","
FIELDS = "RecordNumber","TimeGenerated","EventID","EventType","EventTypeName","EventCategory","EventCategoryName","SourceName","Strings","SID"

I've tried the transforms.conf with and without quoting the field names.

The only way that I've got this to work is if I import just a single file, not monitor a directory, but obviously that gets to be a bit annoying.

What am I missing, or what should I be trying that I've not yet tried? I've also tried it without the DELIMS in the transforms.conf...no joy.

Re: Why isn't this transforming?

gkanapathy — Sun, 10 Jul 2011 05:48:24 GMT

CSV/DELIMS/FIELDS extractions are search-time, not index-time. Therefore, it should be specified in props.conf with a REPORT-ad clause rather than a TRANSFORMS-ad clause.

Re: Why isn't this transforming?

howyagoin — Sun, 10 Jul 2011 09:36:30 GMT

Thanks, that seems to work. I was thrown off in the documentation by:

"Use the TRANSFORMS field extraction type to create index-time field extractions. Use the
REPORT or EXTRACT field extraction types to create search-time field extractions.
"

topic Re: Why isn't this transforming? in Getting Data In

Why isn't this transforming?

Re: Why isn't this transforming?

Re: Why isn't this transforming?