I want to manually add an event to an index, using collect seems to be the most straight forward method. I am asking for a method to use makeresults and eval to add field quotes like the native Aruba SNMP log format to send in raw format to an index
Background: We had a power outage at one of our sites. Report and Alert searches look for active user Wi-Fi sessions. Because the access points were offline, when users left for the day the Wi-Fi session end log events were not sent from Aruba to Splunk , which is causing false positive alerts.
The Aruba SNMP logs look like this:
timestamp=1723828026 notification_from_address = "172.20.0.69" notification_from_port = "34327" SNMPv2-SMI::mib-2.1.3.0 = "10679000" SNMPv2-SMI::snmpModules.1.1.4.1.0 = "1.3.6.1.4.1.14823.2.3.1.11.1.2.1219" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.60 = "0x07e808100a0706002d0700" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.51.0 = "192.168.50.54" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.52.0 = "0xd8be1f2f9c1a" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.3.0 = "0x2462ce8053b1" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.94.0 = "RAP1053a" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.28.0 = "0" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.59.0 = "0" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.103.0 = "2" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.136.0 = "11" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.137.0 = "1"
My search:
| makeresults
| eval timeStamp=now()
| eval logEvent="timestamp=1723830464 notification_from_address = \"172.20.0.17\" notification_from_port = \"43015\" SNMPv2-SMI::mib-2.1.3.0 = \"2063900\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.60 = \"0x07e8080e0d310f002d0700\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.51.0 = \"192.168.50.67\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.52.0 = \"0xd8be1f7d1076\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.3.0 = \"0x482f6b06b171\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.94.0 = \"AP7\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.28.0 = \"0\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.59.0 = \"0\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.103.0 = \"2\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.136.0 = \"10\" SNMPv2-SMI::enterprises.14823.2.3.1.11.1.1.137.0 = \"1\""
| collect index=aruba_snmp sourcetype=snmp_traps output_format=raw testmode=true
The search result looks like what I want but when sent in raw format the escape \ are visible. How do I obscure or remove the \ in raw format? Thank you for any help in advance.
That's because you're collecting the contents of the event in a field called logEvent. If you want to collect this as raw event, you obviously have to set the _raw field.
You are aware that using other sourcetype than stash (or stash_hec for output_format=hec) uses up your license?
You can also have issues with timestamps if you don't set _time properly before collecting (and generally you should set all default metadata fields)
That's because you're collecting the contents of the event in a field called logEvent. If you want to collect this as raw event, you obviously have to set the _raw field.
You are aware that using other sourcetype than stash (or stash_hec for output_format=hec) uses up your license?
You can also have issues with timestamps if you don't set _time properly before collecting (and generally you should set all default metadata fields)
I have the collect search working, eval _raw="field1","field2", ...
Conversion functions - Splunk Documentation
Thank you for pointing me in the right direction and well done @PickleRick
I was not aware of the licensing implications, thank you and I'll stay in compliance.
I mean you use up additional license amount for indexing additional data using the collect command unless you use the stash or stash_hec sourcetypes. So each events you firstly index into index A and then search, transform and collect into index B will cost you twice (roughly - depending on what you do with it in terms of processing before collecting) the license usage that it uses just be indexing it into index A. Whether you're within your license limits or not depends of course on the overall amount of ingested data and your license size.
Appreciate the clarification, I have 30%+ headroom with my license so a couple of onetime events should not be an issue.