McAfee TA 2.0 inputs issues

[abonuccelli_spl]

Hi,

I use McAfee EPO 4.x and would like to import data into Splunk using Splunk McAfee TA 2.0

however I am seeing errors like

2014-07-22 17:07:20.442 server01:ERROR:Scheduler - Error while reading stanza=[dbmon-tail://mcafee_epo_4_db/mcafee_epo_4_db_myinputl]: com.splunk.config.SplunkConfigurationException: Error validating dbmonTail for monitor=dbmon-tail://mcafee_epo_4_db/mcafee_epo_4_db_myinputl: The column "AutoID" is not present in the result set with query = SELECT CAST([EPOEvents].[ReceivedUTC] as varchar) as.......

I've tried few variations without using 'AS' in 'SELECT' statement, however the timestamps are coming in wrong!

how to fix this?

[mcronkrite]

This worked for me with this format of timestamp (Aug 3 2012 8:17AM)

[dbmon-tail://mcafee_epo/ta_mcafee_input]

host = 

index = mcafee

sourcetype = mcafee:epo

interval = * * * * *

output.format = kv

output.timestamp = 1

output.timestamp.column = timestamp

output.timestamp.format = yyyy-MM-dd HH:mm:ss

output.timestamp.parse.format = MMM dd yyyy HH:mmaa

query= SELECT TOP 10000

    [EPOEvents].[AutoID],

    [EPOEvents].[ThreatName] as [signature],

    [EPOEvents].[ThreatType] as [threat_type],

    [EPOEvents].[ThreatEventID] as [signature_id],

    [EPOEvents].[ThreatCategory] as [category],

    [EPOEvents].[ThreatSeverity] as [severity_id],

    [EPOEventFilterDesc].[Name] as [event_description],

    [EPOEvents].[ReceivedUTC] as [received_timestamp],

    [EPOEvents].[DetectedUTC] as [detected_timestamp],

    [EPOEvents].[TargetFileName] as [file_name],

    [EPOEvents].[AnalyzerDetectionMethod] as [detection_method],

    [EPOEvents].[ThreatActionTaken] as [vendor_action],

    [EPOEvents].[ThreatHandled] as [threat_handled],

    [EPOEvents].[TargetUserName] as [logon_user],

    [EPOComputerProperties].[UserName] as [user],

    [EPOComputerProperties].[DomainName] as [dest_nt_domain],

    [EPOEvents].[TargetHostName] as [dest_dns],

    [EPOEvents].[TargetHostName] as [dest_nt_host],

    [EPOComputerProperties].[IPHostName] as [fqdn],

    [dest_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),4,1))) ),

    [EPOComputerProperties].[SubnetMask] as [dest_netmask],

    [EPOComputerProperties].[NetAddress] as [dest_mac],

    [EPOComputerProperties].[OSType] as [os],

    [EPOComputerProperties].[OSServicePackVer] as [sp],

    [EPOComputerProperties].[OSVersion] as [os_version],

    [EPOComputerProperties].[OSBuildNum] as [os_build],

    [EPOComputerProperties].[TimeZone] as [timezone],

    [EPOEvents].[SourceHostName] as [src_dns],

    [src_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),4,1))) ),

    [EPOEvents].[SourceMAC] as [src_mac],

    [EPOEvents].[SourceProcessName] as [process],

    [EPOEvents].[SourceURL] as [url],

    [EPOEvents].[SourceUserName] as [logon_user],

    [EPOComputerProperties].[IsPortable] as [is_laptop],

    [EPOEvents].[AnalyzerName] as [product],

    [EPOEvents].[AnalyzerVersion] as [product_version],

    [EPOEvents].[AnalyzerEngineVersion] as [engine_version],

    [EPOEvents].[AnalyzerEngineVersion] as [dat_version],

    [EPOProdPropsView_VIRUSCAN].[datver] as [vse_dat_version],

    [EPOProdPropsView_VIRUSCAN].[enginever64] as [vse_engine64_version],

    [EPOProdPropsView_VIRUSCAN].[enginever] as [vse_engine_version],

    [EPOProdPropsView_VIRUSCAN].[hotfix] as [vse_hotfix],

    [EPOProdPropsView_VIRUSCAN].[productversion] as [vse_product_version],

    [EPOProdPropsView_VIRUSCAN].[servicepack] as [vse_sp],

    [EPOProdPropsView_ANTISPYWARE].[productversion] as [antispyware_version]

FROM [EPOEvents]

left join [EPOLeafNode] on [EPOEvents].[AgentGUID] = [EPOLeafNode].[AgentGUID]

left join [EPOProdPropsView_ANTISPYWARE] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_ANTISPYWARE].[LeafNodeID]

left join [EPOProdPropsView_VIRUSCAN] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_VIRUSCAN].[LeafNodeID]

left join [EPOComputerProperties] on [EPOLeafNode].[AutoID] = [EPOComputerProperties].[ParentID]

left join [EPOEventFilterDesc] on [EPOEvents].[ThreatEventID] = [EPOEventFilterDesc].[EventId]

and (EPOEventFilterDesc.Language='0409')

{{WHERE EPOEvents.$rising_column$ > ? }} ORDER BY [EPOEvents].[AutoID]

table = ta_mcafee_input

[abonuccelli_spl · Accepted solution]

This will be addressed in next maintenance release,

to fix this please replace the default

'output.timestamp.parse.format' and 'query' params in $SPLUNK_HOME/etc/apps/Splunk_TA_mcafee/default/inputs.conf
with

output.timestamp.parse.format = yyyy-MM-dd HH:mm:ss
query = SELECT CONVERT(VARCHAR(19), [EPOEvents].[ReceivedUTC], 120) as [timestamp], [EPOEvents].[AutoID], [EPOEvents].[ThreatName] as [signature], [EPOEvents].[ThreatType] as [threat_type], [EPOEvents].[ThreatEventID] as [signature_id], [EPOEvents].[ThreatCategory] as [category], [EPOEvents].[ThreatSeverity] as [severity_id], [EPOEventFilterDesc].[Name] as [event_description], [EPOEvents].[DetectedUTC] as [detected_timestamp], [EPOEvents].[TargetFileName] as [file_name], [EPOEvents].[AnalyzerDetectionMethod] as [detection_method], [EPOEvents].[ThreatActionTaken] as [action], [EPOEvents].[ThreatHandled] as [threat_handled], [EPOEvents].[TargetUserName] as [logon_user], [EPOComputerProperties].[UserName] as [user], [EPOComputerProperties].[DomainName] as [dest_nt_domain], [EPOEvents].[TargetHostName] as [dest_dns], [EPOEvents].[TargetHostName] as [dest_nt_host], [EPOComputerProperties].[IPHostName] as [fqdn], [dest_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOComputerProperties].[IPV4x] + 2147483648))),4,1))) ), [EPOComputerProperties].[SubnetMask] as [dest_netmask], [EPOComputerProperties].[NetAddress] as [dest_mac], [EPOComputerProperties].[OSType] as [os], [EPOComputerProperties].[OSServicePackVer] as [sp], [EPOComputerProperties].[OSVersion] as [os_version], [EPOComputerProperties].[OSBuildNum] as [os_build], [EPOComputerProperties].[TimeZone] as [timezone], [EPOEvents].[SourceHostName] as [src_dns], [src_ip] = ( convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),1,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),2,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),3,1)))+'.'+convert(varchar(3),convert(tinyint,substring(convert(varbinary(4),convert(bigint,([EPOEvents].[SourceIPV4] + 2147483648))),4,1))) ), [EPOEvents].[SourceMAC] as [src_mac], [EPOEvents].[SourceProcessName] as [process], [EPOEvents].[SourceURL] as [url], [EPOEvents].[SourceUserName] as [logon_user], [EPOComputerProperties].[IsPortable] as [is_laptop], [EPOEvents].[AnalyzerName] as [product], [EPOEvents].[AnalyzerVersion] as [product_version], [EPOEvents].[AnalyzerEngineVersion] as [engine_version], [EPOEvents].[AnalyzerEngineVersion] as [dat_version], [EPOProdPropsView_VIRUSCAN].[datver] as [vse_dat_version], [EPOProdPropsView_VIRUSCAN].[enginever64] as [vse_engine64_version], [EPOProdPropsView_VIRUSCAN].[enginever] as [vse_engine_version], [EPOProdPropsView_VIRUSCAN].[hotfix] as [vse_hotfix], [EPOProdPropsView_VIRUSCAN].[productversion] as [vse_product_version], [EPOProdPropsView_VIRUSCAN].[servicepack] as [vse_sp], [EPOProdPropsView_ANTISPYWARE].[productversion] as [antispyware_version] FROM [EPOEvents] left join [EPOLeafNode] on [EPOEvents].[AgentGUID] = [EPOLeafNode].[AgentGUID] left join [EPOProdPropsView_ANTISPYWARE] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_ANTISPYWARE].[LeafNodeID] left join [EPOProdPropsView_VIRUSCAN] on [EPOLeafNode].[AutoID] = [EPOProdPropsView_VIRUSCAN].[LeafNodeID] left join [EPOComputerProperties] on [EPOLeafNode].[AutoID] = [EPOComputerProperties].[ParentID] left join [EPOEventFilterDesc] on [EPOEvents].[ThreatEventID] = [EPOEventFilterDesc].[EventId] and (EPOEventFilterDesc.Language='0409') WHERE [EPOEvents].[AutoID] > 0 {{ AND [EPOEvents].$rising_column$ > ? }} ORDER BY [EPOEvents].[AutoID]

[abonuccelli_spl]

try this instead for the timestamp problem
http://answers.splunk.com/answers/145674/timestamp-issue-with-splunk-add-on-for-mcafee

[ysouchon]

Sorry to tell you, but it seems your answer doesn't work. The timestamp is still wrong (index time)...