CSV Field export doesn't work

Christian — Fri, 07 Jan 2011 01:26:01 GMT

Hi all,

i know there are a few other questions with good answers about my topic but I still have my problems. This is my setup :

inputs.conf

[monitor:///data/proxy/archiv] 
disabled = false 
followTail = 0 
index = idx_proxy_pro 
sourcetype = proxy

props.conf

[proxy]
KV_MODE=none
CHECK_FOR_HEADER = false
SHOULD_LINEMERGE = false
TRANSFORMS-commentsToNull = commentsToNull
REPORT-proxy_kv_export = proxyKvExport

transforms.conf (with no linebreaks, this is just here)

[proxyKvExport] 
DELIMS = ","
FIELDS "dummy","tran_id","tran_time","client_ip","scanning_server_ip","domain_user_name",
"user_domain","user_id","protocol","url","file_name","policy_id",
"identification_policy_id","https_policy_id","kaspersky_virus_name",
"sophos_virus_name","mcafee_virus_name","tran_size","HTMLRepaired","activex_name",
"xray","action_gid","admin_group","cache_hit"

[commentsToNull]
REGEX = ^[#R]
DEST_KEY = queue
FORMAT = nullQueue

Here is e example of the Logfile (values were replaced) :
T,"4D1FECFE3A63930D6778","01/02/2011 04:11:58","111.111.111.111","1111.111.111.111","Username","Domain","16","HTTP","http://www.google.com","1.gif","229","215",,,,,"0",,,,"","none","
R,"4D1FECFE3A63930D6778","O"," ",0,1,1006,1006
R,"4D1FECFE3A63930D6778","O"," ",0,8,8004,8005
R,"4D1FECFE3A63930D6778","O"," ",0,19,19001,19001
R,"4D1FECFE3A63930D6778","I"," ",0,8,8004,8005
T,"4D1FECFE3A63930D6778","01/02/2011 04:11:58","111.111.111.111","1111.111.111.111","Username","Domain","16","HTTP","http://www.google.com","1.gif","229","215",,,,,"0",,,,"","none","
R,"4D1FECFE3A63930D6778","O"," ",0,1,1006,1006
R,"4D1FECFE3A63930D6778","O"," ",0,8,8004,8005
R,"4D1FECFE3A63930D6778","O"," ",0,19,19001,19001
R,"4D1FECFE3A63930D6778","I"," ",0,8,8004,8005
T,"4D1FECFE3A63930D6778","01/02/2011 04:11:58","111.111.111.111","1111.111.111.111","Username","Domain","16","HTTP","http://www.google.com","1.gif","229","215",,,,,"0",,,,"","none","
R,"4D1FECFE3A63930D6778","O"," ",0,1,1006,1006
R,"4D1FECFE3A63930D6778","O"," ",0,8,8004,8005
R,"4D1FECFE3A63930D6778","O"," ",0,19,19001,19001
R,"4D1FECFE3A63930D6778","I"," ",0,8,8004,8005

I would like to dismiss the Lines beginning with R and # for this i have the Transformation commentsToNull witch works fine. Only the proxyKvExport doesn't work and I have no idea why not.

Anyone a good hint ?

thanks christian

Re: CSV Field export doesn't work

Christian — Fri, 21 Jan 2011 21:06:21 GMT

Okay wiht the help of Diana (thank you very mutch :-)!) we found the solution for this problem. This might be interesting for others with the same problem. The thing is I' am working with a indexer and a search head. Because the KVExport is made during the search-time, the extraction informations had to be on the search head.

After putting the following Lines on the search head's transforms and props.conf everything worked as excepted.

transforms.conf:
[proxyKvExport]
DELIMS = ","
FIELDS "dummy","tran_id","tran_time","client_ip","scanning_server_ip","domain_user_name", "user_domain","user_id","protocol","url","file_name","policy_id", "identification_policy_id","https_policy_id","kaspersky_virus_name", "sophos_virus_name","mcafee_virus_name","tran_size","HTMLRepaired","activex_name", "xray","action_gid","admin_group","cache_hit"

props.conf :
[proxy]
REPORT-proxy_kv_export = proxyKvExport

Re: CSV Field export doesn't work

dvb — Sat, 22 Jan 2011 00:57:02 GMT

And here's the general description on which config has to go where: http://www.splunk.com/wiki/Where_do_I_configure_my_Splunk_settings%3F

topic Re: CSV Field export doesn't work in Getting Data In

CSV Field export doesn't work

Re: CSV Field export doesn't work

Re: CSV Field export doesn't work