Unable to create sourcetype

pratapa — Wed, 29 Apr 2020 09:06:22 GMT

I am trying to create a souretype "meraki" on the GUI.

But it is saying "Sourcetype meraki already exists"

sourcetype meraki does not exist in the list of sourcetypes. What could be the problem. Why it is not allowing me to create sourcetype.

Earlier I created an index with name "meraki".

Re: Unable to create sourcetype

PavelP — Wed, 29 Apr 2020 12:10:35 GMT

Hello @pratapa,

do you have TA-meraki installed? Check under Manage Apps.

You can see meraki sourcetype definition with:

  $SPLUNK_HOME/bin/splunk btool props list --debug meraki

Re: Unable to create sourcetype

codebuilder — Wed, 29 Apr 2020 14:14:32 GMT

You can build a sourcetype from the web UI, but that interface does not actually install it. It's actually gone once you close or leave that page.

Once you've finished creating the sourcetype you'll need to copy the stanza text that is generated and paste it into props.conf then either cycle Splunk or use the deployer to push it out if it's going to be part of an app.

Re: Unable to create sourcetype

pratapa — Wed, 30 Sep 2020 05:21:22 GMT

Yes we installed TA-meraki.

Following is the output of $SPLUNK_HOME/bin/splunk btool props list --debug meraki

./splunk btool props list --debug meraki

/opt/splunk/etc/apps/TA-meraki/default/props.conf [meraki]
/opt/splunk/etc/system/default/props.conf ADD_EXTRA_TIME_FIELDS = True
/opt/splunk/etc/system/default/props.conf ANNOTATE_PUNCT = True
/opt/splunk/etc/system/default/props.conf AUTO_KV_JSON = true
/opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE =
/opt/splunk/etc/system/default/props.conf BREAK_ONLY_BEFORE_DATE = True
/opt/splunk/etc/system/default/props.conf CHARSET = UTF-8
/opt/splunk/etc/system/default/props.conf DATETIME_CONFIG = /etc/datetime.xml
/opt/splunk/etc/system/default/props.conf DEPTH_LIMIT = 1000

cat props.conf

[meraki]

This line is needed to be on the indexer or heavy forwarder

meraki includes their own date/time which is a unix timestamp, this transforms detect it and removes it

which saves you data

TRANSFORMS-meraki_date_clipper = meraki_date_clipper

KV_MODE = none

REPORT-dvc = meraki_dvc
REPORT-dvc2 = meraki_dvc2
REPORT-dvc_ip = meraki_dvc_ip
REPORT-dvc_ip2 = meraki_dvc_ip2
REPORT-content_filtering_generic = meraki_content_filtering_generic
REPORT-transport = meraki_transport
REPORT-url_protocol = meraki_url_protocol
REPORT-http_user_agent = meraki_http_user_agent
REPORT-src = meraki_src
REPORT-dst = meraki_dst
REPORT-http_method = meraki_http_method
REPORT-src_mac = meraki_src_mac
REPORT-user = meraki_user
REPORT-user2 = meraki_user2
REPORT-url = meraki_url
REPORT-url2 = meraki_url2
REPORT-category = meraki_category
REPORT-dest_port = meraki_dest_port
REPORT-dest_port2 = meraki_dest_port2
REPORT-src_port = meraki_src_port
REPORT-src_port2 = meraki_src_port2
REPORT-icmp_type = meraki_icmp_type
REPORT-meraki_action = meraki_action
REPORT-meraki_flows_action = meraki_flows_action
REPORT-meraki_priority = meraki_priority
REPORT-signature_id = meraki_signature_id
REPORT-signature = meraki_signature

Sets value for meraki_app; in REPORT first device that sets value overrides secondary queries

REPORT-meraki_1events_ad = meraki_events_ad
REPORT-meraki_2dhcp_conflict = meraki_dhcp_conflict
REPORT-meraki_3dhcp_lease_added = meraki_dhcp_lease_added
REPORT-meraki_4dhcp_lease_release = meraki_dhcp_lease_release
REPORT-meraki_5dhcp_lease_fail = meraki_dhcp_lease_fail
REPORT-meraki_6dhcp_lease_fail2 = meraki_dhcp_lease_fail2
REPORT-meraki_7port = meraki_port
REPORT-meraki_8authentication = meraki_authentication
REPORT-meraki_91wireless = meraki_events_wireless
REPORT-meraki_92app = meraki_app
REPORT-meraki_93app = meraki_app2

These handles the airmarshal_events

REPORT-air_signature = air_signature
REPORT-air_ssid = air_ssid
REPORT-air_bssid = air_bssid
REPORT-air_src_mac = air_src_mac
REPORT-air_dest_mac = air_dest_mac
REPORT-air_wired_mac = air_wired_mac
REPORT-air_client_mac = air_client_mac
REPORT-air_vlan_id = air_vlan_id
REPORT-air_channel = air_channel
REPORT-air_fc_type = air_fc_type
REPORT-air_fc_subtype = air_fc_subtype
REPORT-air_inter_arrival = air_inter_arrival
REPORT-air_dos_count = air_dos_count
REPORT-air_alarm_id = air_alarm_id
REPORT-air_state = air_state
REPORT-air_radio = air_radio
REPORT-air_packet = air_packet
REPORT-air_reason = air_reason
REPORT-air_rssi = air_rssi
REPORT-air_vap = air_vap
REPORT-air_client_ip = air_client_ip
REPORT-air_instigator = air_instigator
REPORT-air_duration = air_duration
REPORT-air_last_auth_ago = air_last_auth_ago
REPORT-air_is_wpa = air_is_wpa
REPORT-air_full_conn = air_full_conn
REPORT-air_ip_resp = air_ip_resp
REPORT-air_ip_src = air_ip_src
REPORT-air_http_resp = air_http_resp
REPORT-air_arp_resp = air_arp_resp
REPORT-air_arp_src = air_arp_src
REPORT-air_dns_server = air_dns_server
REPORT-air_dns_req_rtt = air_dns_req_rtt
REPORT-air_dns_resp = air_dns_resp
REPORT-air_dhcp_lease_completed = air_dhcp_lease_completed
REPORT-air_dhcp_ip = air_dhcp_ip
REPORT-air_dhcp_server = air_dhcp_server
REPORT-air_dhcp_server_mac = air_dhcp_server_mac
REPORT-air_dhcp_resp = air_dhcp_resp
REPORT-air_aid = air_aid
REPORT-air_info = air_info
REPORT-air_type = air_type
REPORT-meraki_wireless_action = meraki_wireless_action

FIELDALIAS-dest = dst AS dest
FIELDALIAS-src_ip = src AS src_ip
FIELDALIAS-srcip = src AS srcip
FIELDALIAS-dest_ip = dst AS dest_ip
FIELDALIAS-user_agent = http_user_agent AS user_agent
FIELDALIAS-ua = http_user_agent AS ua
FIELDALIAS-urlc = category AS urlc

FIELDALIAS-signature = category AS signature

FIELDALIAS-urlp = dest_port AS urlp
FIELDALIAS-client_ip = client_ip AS src_ip

EVAL-http_user_agent_length = len(http_user_agent)
EVAL-ids_type = if(meraki_app=="ids-alerts", "network", if(meraki_app=="events-airmarshal","wireless",null()))
EVAL-app = "meraki-".meraki_app
EVAL-url_length = len(url)
EVAL-response_time = sum(arp_resp+dhcp_resp+ip_resp)

CIM states that (src|dest|x)_mac should be lower case

docs.splunk.com/Documentation/CIM/latest/User/NetworkTraffic

EVAL-src_mac = lower(src_mac)
EVAL-dest_mac = lower(dest_mac)
EVAL-client_mac = lower(client_mac)
EVAL-cached = "0"
EVAL-lease_scope = if(len(lease_scope_subnet)=>1,src."/".lease_scope_subnet,null())
EVAL-signature = coalesce(dhcpsignature,category,signature)
EVAL-category = coalesce(category,signature)
EVAL-signature_id = coalesce(dhcpsignature_id,signature_id)
EVAL-meraki_action = coalesce(meraki_action,meraki_dhcp_action,meraki_wireless_action,meraki_airmarshal_action)
EVAL-meraki_priority = coalesce(meraki_port_priority,meraki_priority,meraki_dhcp_priority,meraki_ad_priority,meraki_url_priority)

LOOKUP-vendor_info_for_meraki = meraki_vendor_info_lookup sourcetype OUTPUT vendor,product,vendor_product
LOOKUP-action_for_meraki = meraki_action_lookup meraki_action OUTPUT action
LOOKUP-severity_for_meraki = meraki_severity_lookup meraki_priority OUTPUT severity
LOOKUP-icmp_code_for_meraki = meraki_icmp_code_lookup icmp_type OUTPUT icmp_code
LOOKUP-status_code_for_meraki = meraki_status_code_lookup meraki_app,meraki_action OUTPUT status_code,status,rule

What I need to do to create sourcetype meraki.

topic Unable to create sourcetype in Splunk Search

Unable to create sourcetype

Re: Unable to create sourcetype

Re: Unable to create sourcetype

Re: Unable to create sourcetype

cat props.conf

This line is needed to be on the indexer or heavy forwarder

meraki includes their own date/time which is a unix timestamp, this transforms detect it and removes it

which saves you data

Sets value for meraki_app; in REPORT first device that sets value overrides secondary queries

These handles the airmarshal_events

FIELDALIAS-signature = category AS signature

CIM states that (src|dest|x)_mac should be lower case

docs.splunk.com/Documentation/CIM/latest/User/NetworkTraffic