I had to revert my VM from a snapshot back to Splunk 8.0.1 using Splunk Cloud Gateway instead of Secure Gateway.  It now works, I can register my device and check dashboards.   ... View more

  I can delete devices, I can somewhat register a device (error at the end of the process telling me to contact the admin).   Thankfully production doesn't use this, but seems shaky for a built in app. ... View more

Doing a quick beyond compare of the "garminclient.py" found in the add-on and "garminclient.py" found in the garminexprot-master from github. There are a number of other changes as well, but I'm thinking between the time of this add-on being made and the last update to garminexport, there has been a change with the REST APIs authentication.   Going to garminexport's Issues: https://github.com/petergardfjall/garminexport/issues June 12 Authentication failure   I'm tempted to just copy over the python into the TA but I'm sure I wreck something... ... View more

I ran into what might have been the same issue as you.  I went to var/log/ta_garmin_activities.log and it shows both what index=_internal showed: "Authenticating to Garmin Connect..." But then additional information "ERROR authentication failures: did you enter the valid credentials?"   I tripple checked creds.  I think there is an issue with the TA.  Googling about Garmin API issues it looks like Garmin changes things from time to time and it breaks many a github python project's authentication (this TA uses a python project from github to grab the .fit files). I'm not technical enough to try and fix this or build a TA from scratch.  Wish I was as its a real bummer this isn't working.  A big part of getting my watch was the thought I could Splunk it... ... View more

Additional info from one of the troubleshooting dashboards:     ... View more

Why don't you make the props.conf not for json? The eve json has a lot more data   This ELK dashboard is amazing and you need the additional fields to drive it https://github.com/robcowart/synesis_lite_suricata To be honest this seems to really give Splunk a run for the money https://www.youtube.com/watch?v=YA2tGrBQ4v0 Years ago I would have argued that Splunk has more support and documentation but I'm finding more and more tutorials geared towards ELK. ... View more

Sorry to4kawa, I'm not advanced enough to understand your answer. I changed the props to have your inputs, but not sure on what is and how to apply source="udp:514" | rex "(?<json>{.*})" | spath input=json   Example of the TA-pfsense props.conf: [pfsense] # timestamp fix to prevent default extraction issues when adding more apps TIME_PREFIX = ^ TIME_FORMAT = %b %d %H:%M:%S MAX_TIMESTAMP_LOOKAHEAD = 15 # rest is original code TRANSFORMS-pfsense_sourcetyper = pfsense_sourcetyper SHOULD_LINEMERGE = false # suspect the below 3 lines could be causing the time stamps being missing on some log types test 7.18.2020 # SEDCMD-event_cleaner = s/^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+\S+\.\S+\s+/\1/g # SEDCMD-event_cleaner2 = s/^(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+(\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s)+/\1/g # SEDCMD-event_cleaner3 = s/^\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s\S+\s(\S+\s)/\1/g [pfsense:filterlog] KV_MODE = none EXTRACT-ipv4_tcp = filterlog:\s(?<rule>[^,])*,(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offset>[^,]*) ,(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>tcp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<src_port>[^,]*),(?<d est_port>[^,]*),(?<payload_bytes>[^,]*),(?<vendor_tcp_flags>[^,]*),(?<sequence_number>[^,]*),(?<ack>[^,]*),(?<window>[^,]*),(?<urg>[^,]*),( ?<options>[^$]*)$ EXTRACT-ipv4_udp = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offset>[^,]*) ,(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>udp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<src_port>[^,]*),(?<d est_port>[^,]*),(?<payload_bytes>[^,]*) EXTRACT-ipv4_icmp_request = filterlog:\s(?<rule>[^,])*,(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reaso n>[^,]*),(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offs et>[^,]*),(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>icmp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<icmp_code> request),(?<echo_id>\d+),(?<echo_sequence>\d+) EXTRACT-ipv4_icmp_unreachport = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<r eason>[^,]*),(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?< offset>[^,]*),(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>icmp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<icmp_c ode>unreachport),(?<icmp_dest_ip>[^,]*),(?<unreachable_protocol_id>[^,]*),(?<unreachable_port_data>[^$]+)$ EXTRACT-ipv4_redirect_icmp_unreach_timeexceed = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest _int>[^,]*),(?<reason>[^,]*),(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*), (?<id>[^,]*),(?<offset>[^,]*),(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>icmp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip> [^,]*),(?<icmp_code>redirect|unreach|timexceed),(?<icmp_text>[^$]+)$ EXTRACT-ipv4_carp = filterlog:\s(?<rule>[^,])*,(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*) ,(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offset>[^,]* ),(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>carp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<carp_type>[^,]*),( ?<carp_ttl>[^,]*),(?<vhid>[^,]*),(?<version>[^,]*),(?<advbase>[^,]*),(?<advskew>[^,]*) EXTRACT-ipv4_igmp = filterlog:\s(?<rule>[^,])*,(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*) ,(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offset>[^,]* ),(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>igmp),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),datalength=(?<data_le ngth>\d+) EXTRACT-ipv4_pim = filterlog:\s(?<rule>[^,])*,(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>4),(?<tos>[^,]*),(?<ecn>[^,]*),(?<ttl>[^,]*),(?<id>[^,]*),(?<offset>[^,]*) ,(?<flags>[^,]*),(?<transport_id>[^,]*),(?<vendor_transport>pim),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),datalength=(?<data_leng th>\d+) EXTRACT-ipv6_icmp = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*) ,(?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>6),(?<class>[^,]*),(?<flow_label>[^,]*),(?<hop_limit>[^,]*),(?<vendor_tra nsport>ICMPv6),(?<transport_id>[^,]*),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*), EXTRACT-ipv6_tcp = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>6),(?<class>[^,]*),(?<flow_label>[^,]*),(?<hop_limit>[^,]*),(?<vendor_tran sport>TCP),(?<transport_id>[^,]*),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<src_port>[^,]*),(?<dest_port>[^,]*),(?<payload_byte s>[^,]*),(?<vendor_tcp_flags>[^,]*),(?<sequence_number>[^,]*),(?<ack>[^,]*),(?<window>[^,]*),(?<urg>[^,]*),(?<options>[^$]*)$ EXTRACT-ipv6_udp = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>6),(?<class>[^,]*),(?<flow_label>[^,]*),(?<hop_limit>[^,]*),(?<vendor_tran sport>UDP),(?<transport_id>[^,]*),(?<bytes>[^,]*),(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<src_port>[^,]*),(?<dest_port>[^,]*),(?<payload_byte s>[^,]*) EXTRACT-ipv6-hbh = filterlog:\s(?<rule>[^,]*),(?<sub_rule>[^,]*),(?<anchor>[^,]*),(?<tracker_id>[^,]*),(?<dest_int>[^,]*),(?<reason>[^,]*), (?<vendor_action>[^,]*),(?<vendor_direction>[^,]*),(?<ip_version>6),(?<class>[^,]*),(?<flow_label>[^,]*),(?<hop_limit>[^,]*),Options,\d+,\d +,(?<src_ip>[^,]*),(?<dest_ip>[^,]*),(?<extension_header>\w+), REPORT-vendor_tcp_flag = pfsense_vendor_tcp_flag EVAL-app = "pfsense:filterlog" EVAL-bytes_in = if(vendor_direction == "in", bytes,null) EVAL-bytes_out = if(vendor_direction == "out", bytes,null) EVAL-direction = case(vendor_direction == "in", "inbound", vendor_direction == "out", "outbound") FIELDALIAS-pfsense_filterlog_src=src_ip AS src FIELDALIAS-pfsense_filterlog_dest = dest_ip AS dest LOOKUP-filterlog_tcpflags = pfsense_filter_tcpflags vendor_tcp_flag OUTPUTNEW tcp_flag LOOKUP-filterlog_transport = pfsense_filter_transport vendor_transport OUTPUTNEW transport LOOKUP-filterlog_action = pfsense_filter_action vendor_action OUTPUTNEW action [pfsense:dhcpd] EXTRACT-ipv4_dhcp = (?<vendor_action>DHCPACK|DHCPREQUEST) (?:on|for) (?<dest_ip>\S+) (?:from|to) (?<src_mac>\S+) \(.*\) via (?<src_interfac e>\S+) EXTRACT-ipv6_dhcp_rebind = dhcpd: Rebind message from (?<src_mac>\S+) port (?<src_port>\d+), transaction ID (?<transaction_id>\S+) EXTRACT-ipv6_dhcp_reply = dhcpd: Sending Reply to (?<src_mac>\S+) port (?<src_port>\S+) EVAL-app = "pfsense:dhcpd" [pfsense:openvpn] EXTRACT-process_id = \w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}\s+\w+(\[(?<process_id>\d+)\])?: EXTRACT-authentication = openvpn: user '(?<user>[^']+)' (?<vendor_action>could not authenticate\.|authenticated) EXTRACT-user,src_ip,src_port = openvpn\[\d+\]: (?<user>\S+)\/(?<src_ip>\S+):(?<src_port>\d+) EXTRACT-src_ip,src_port = openvpn\[\d+\]:\s+(?<src_ip>\d+\.\d+\.\d+\.\d+):(?<src_port>\d+) EXTRACT-src_ip,src_port,user,reason = openvpn\[\S+\]: (?P<src_ip>[^:]+):(?P<src_port>\d+).*\[(?<user>\S+)\]\s+(?<reason>Peer Connection Ini tiated|Inactivity timeout) EVAL-app = "pfsense:openvpn" LOOKUP-openvpn_action = pfsense_openvpn_action vendor_action OUTPUTNEW action [pfsense:nginx] EXTRACT-pf_nginx = nginx: (?P<dest>[^ ]+) \-(?P<user>[^\s"]*) \- \[(?P<request_time>[^\]]+)[^"\n]*"(?P<http_method>\w+)\s+(?P<url>[^\s"]+)[ ^ \n]* (?P<http_proto>[^"]+)[^ \n]* (?P<status>[^ ]+)\s+(?P<bytes_out>\d+)[^"\n]*"(?P<http_referrer>[^"]+)(?:[^"\n]*"){2}(?P<http_user_agen t>[^"]+) EVAL-app = "pfsense:nginx" [pfsense:unbound] EXTRACT-queries = info: resolving (?P<query>\S{2,})\s(?P<query_type>\S+) EXTRACT-response = info: response for (?P<answer>\S+)\s(?P<record_type>\S+) EXTRACT-reply = info: reply from\s(\S+)\s(?P<src>(\d|.)+)#(?P<src_port>(\d+)) EVAL-app = "pfsense:unbound" # added due to Barnyard2 support dropped by Snort and Suricata 8.7.20 #[pfsense:suricata] #EXTRACT-classification = ^(?:[^:\n]*:){6}\s+(?P<classification>[^\]]+) #EXTRACT-generator_id = ^[^\[\n]*\[(?P<generator_id>\d+) #EXTRACT-signature_id = ^\[\d+:(?P<signature_id>\d+) #EXTRACT-sigrev_id = ^\[\d+:\d+:(?P<sigrev_id>\d+) #EXTRACT-description = ^[^\]\n]*\](?P<description>[^\[]+) #EXTRACT-priority = ^(?:[^:\n]*:){4}\s+(?P<priority>\d+) #EXTRACT-interface = ^[^<\n]*<(?P<interface>\w+) #EXTRACT-protocol = ^[^\{\n]*\{(?P<protocol>\w+) #EXTRACT-source_ip = ^[^\}\n]*\}\s+(?P<source_ip>[^:]+) #EXTRACT-source_port = ^(?:[^:\n]*:){3}(?P<source_port>\d+) #EXTRACT-destination_ip = ^[^\-\n]*\->\s+(?P<destination_ip>[^:]+) #EXTRACT-destination_port = ^[^\-\n]*\->\s+\d+\.\d+\.\d+\.\d+:(?P<destination_port>.+) # converted suricata to eve json for testing [pfsense:suricata] INDEXED_EXTRACTIONS = json KV_MODE = none ... View more

Another complexity, it looks like the suricata plugin for pfsense is sending both unified2 logs (for the alerts) and then eve json for the addition data to the same data input UDP.  I guess some transforms would be need to sourcetype the two differently to apply extractions to? In order to have eve json sent over syslog, send to system logs must be enabled:   So I get duplicates in two different formats:   Due to pfsense, I don't think there is a way around this (maybe in a config somewhere vs the GUI?)   So the challenge, tell Splunk to shed the unified2 format, keep the eve json format, pull fiends from the json format from without a props that has stanzas for the other pfsense sourcetypes such as filterlog, openvpn etc. Also, Splunk is cutting off the source, the line break is too soon I guess- Splunk vs. pfsense GUI.   ... View more