I've been asked to update 'Imperva Database Audit Analysis' and I'm running into issues trying to update the Audit Dashboard. The sanitized data looks like this
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:51 GMT", "audit-policy":"["Policy - Policy_Name - Login/Logout"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"os_user", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"root-->user", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:50 GMT", "audit-policy":"["Global Policy - Login/Logout"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"os_user", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:51 GMT", "audit-policy":"["Policy - Login/Logout - SQL"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"\-->user", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:51 GMT", "audit-policy":"["Policy - Policy_Name - Login/Logout"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"os_user", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"root-->user", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:51 GMT", "audit-policy":"["Global Policy - Login/Logout"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"os_user", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Logout", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:50 GMT", "audit-policy":"["Policy - Login/Logout - SQL"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"${Event.struct.operations.objects.name}", "agent-name":"agent_name", "success":"True", "os-user-chain":"\-->user", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:51 GMT", "audit-policy":"["Global Policy - Login/Logout"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"os_user", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Logout", "schema-name":"schema_name", "object-name":"object_name", "agent-name":"agent_name", "success":"True", "os-user-chain":"", "db-name":"db_name" }
Nov 10 23:20:52 syslog_server {"header":"Imperva Inc.|SecureSphere|version|Audit|Audit.DAM|Informative|", "dest-ip":"dip_address", "db-user":"db_user", "source-ip":"sip_address", "real-time":"Nov 10 2020 23:20:50 GMT", "audit-policy":"["Policy - Login/Logout - SQL"]", "server-group":"server_group", "service-name":"service_name", "application-name":"application_name", "source-application":"source_application", "os-user":"", "host-name":"fqdn", "sql-error":"", "mx-ip":"mx_ip_address", "gw-ip":"gw_ip_address", "objects-list":"[]", "operation-name":"Login", "schema-name":"schema_name", "object-name":"object_name", "agent-name":"agent_name", "success":"True", "os-user-chain":"\-->user", "db-name":"db_nme" }
Since Splunk doesn't handle embedded [] and {} in json, I created this search to process the events.
index=my_index sourcetype=source:type
| rex field=_raw "(?<st_json>\{.*)"
| eval st_json_1=replace(st_json, "\"\[\]\"", "\"Null\"")
| eval st_json=replace(st_json_1, "\"\[", "")
| eval st_json_1=replace(st_json, "\]\"", "")
| eval st_json=replace(st_json_1, "\$\{", "")
| eval st_json_1=replace(st_json, "\}\",", "\",")
| spath input=st_json_1
| eval dest_ip_db_name= 'dest-ip'."\\".'db-name'
| chart count by dest_ip_db_name
| sort limit=10 -count
| rename dest_ip_db_name AS "Database Host \ Database Name" count AS "Number Of Events"
This works. When I move it to the dashboard I get the "Unexpected close tag". This is the query in the dashboard.
<query>index=my_index sourcetype=source:type | rex field=_raw "(?<st_json>\{.*)" | eval st_json_1=replace(st_json, "\"\[\]\"", "\"Null\"") | eval st_json=replace(st_json_1, "\"\[", "") | eval st_json_1=replace(st_json, "\]\"", "") | eval st_json=replace(st_json_1, "\$\{", "") | eval st_json_1=replace(st_json, "\}\",", "\",") | spath input=st_json_1 | eval dest_ip_db_name= 'dest-ip'."\\".'db-name' | chart count by dest_ip_db_name | sort limit=10 -count | rename dest_ip_db_name AS "Database Host \ Database Name" count AS "Number Of Events"</query>
I don't see anything that would cause the 'Unexpected close tag'. Is there an issue with doing the \ escapes in SimpleXML or something else that I'm not aware of?
TIA,
Joe
Duh! I didn't see it until I posted and saw that this line
rex field=_raw "(?<st_json>\{.*)"
is the cause. SimpleXML is treating the <st_json> as a JSON tag. I did a little more research and found a Splunk answer that said if using rex in a <query></query> you have to save the search and then call the saved search instead. That is what I did and it is working.
Duh! I didn't see it until I posted and saw that this line
rex field=_raw "(?<st_json>\{.*)"
is the cause. SimpleXML is treating the <st_json> as a JSON tag. I did a little more research and found a Splunk answer that said if using rex in a <query></query> you have to save the search and then call the saved search instead. That is what I did and it is working.