We have a simple alert with a Webook action assigned to it with an endpoint is OMI. Search: index=xyz TCP_ERROR appName="jojothedolphin" Alert: If number of results > 10 After the alert is triggered, field and values I want to send as my payload are stored in tokens: $trigger_date$ $trigger_time$ $alert.severity$ $job.resultCount$ But I am pulling my hair out trying to figure out how to access them and their value. I cannot get them to display in a table (or any other way which would then become my payload. Help! Damon ... View more

When I make a backup of a splunk server every few days, I just usually tarball the whole /opt/splunk dir. This works fine for recovery purposes when I may need it. But each backup is several GBs. So, I am looking for ways to trim down the backup copy while still retaining all I need for a 'full restore' of a server if every needed. I notice that the ~/var/run/searchpeers has a lot of files + can be large in size. Can I skip this dir when I make my backup or is it crucial to the rebuild/restore? cheers, Damon ... View more

layout: I have indexer cluster consisting of 3 indexers(ind-a, ind-b, ind-c). Replication and search factor is set to: 3. All works fine, no issues. Problem: But, I keep running out of disk space on ind-c only. Following are the steps I was thinking of applying but wanted to make sure this is the right approach. Concern: Is that the indexes.conf in ~/etc/slave-apps/_cluster/local will override the indexes.conf in etc/system/local, thus creating a loop and not resolving my issue. Can someone help with my flow Approach: 1. Select 2 large indexes that I can afford to lose since they are already replicated on ind-a & ind-B to free up disk space just on ind-c 2. Prevent any new data from being indexed to either index on ind-c Steps: 1. ssh ind-c 2. stop splunk 3. cd $SPLUNK_DB 4. cd big-index-dir-1 (80GB) 5. rm -rf ./frozen/* ./db/* ./colddb/* ./thawed/* 6. cd big-index-dir-2 (100GB) 7. rm -rf ./frozen/* ./db/* ./colddb/* ./thawed/* 8. du -sh $SPLUNK_DB (Plenty of space now) 9. Do not edit the ~/etc/slave-apps/_cluster/local/indexes.conf (this is controlled by the MN) 10. Instead do this: cd ~/etc/system/local 11. Create a new indexes.conf and insert: [big-index-dir-1] # turned off repFactor = 0 homePath = $SPLUNK_DB/big-index-dir-1/db coldPath = $SPLUNK_DB/big-index-dir-1/colddb thawedPath = $SPLUNK_DB/big-index-dir-1/thaweddb coldToFrozenDir = $SPLUNK_DB/big-index-dir-1/frozen-archivedb maxTotalDataSizeMB = 80000 frozenTimePeriodInSecs = 259200 [big-index-dir-2] # turned OFF repFactor = 0 homePath = $SPLUNK_DB/big-index-dir-2/db coldPath = $SPLUNK_DB/big-index-dir-2/colddb thawedPath = $SPLUNK_DB/big-index-dir-2/thaweddb coldToFrozenDir = $SPLUNK_DB/big-index-dir-2/frozen-archivedb maxTotalDataSizeMB = 100000 frozenTimePeriodInSecs = 259200 splunk start ... View more

Problem: I am stuck trying to apply props.conf to map my xml file. Read many docs + many attempts but can't figure out how to use the conf file. I placed my props.conf on my indexer. I want each iwuser entry/element. Also, do I need transforms.conf to make this work? Lastly, when I run my query I don't get any results in the table. A solid response/example would be much appreciated. xml file <nextid value="316"> </nextid> <iwuser name="usernamehere"> <email value="emailaddresshere"> </email> <preferredui value="ccstd"> </preferredui> <ismaster value="no"> </ismaster> </iwuser> <iwuser name="iwov"> <email value="sssss@gigi.com"> </email> <preferredui value="ccpro"> </preferredui> <ismaster value="yes"> </ismaster> </iwuser> <iwuser name="jojo" userdbid="ldap" tsid="33"> <display-name value="the dolphin"> </display-name> <email value="jojo.dolphin@gigi.com"> </email> <preferredui value="ccproonly"> </preferredui> <ismaster value="no"> </ismaster> </iwuser> props.conf [teamsite_xml] DATETIME_CONFIG = CURRENT KV_MODE = xml LINE_BREAKER = () MUST_BREAK_AFTER = \ NO_BINARY_CHECK = 1 SHOULD_LINEMERGE = false TRUNCATE = 0 pulldown_type = 1 FIELDALIAS-nextID = nextid{@value} as nextID FIELDALIAS-userName = iwuser{@name} as userName FIELDALIAS-userEmail = iwuser{@email} as userEmail FIELDALIAS-userUI = iwuser{@preferredui} as userUI FIELDALIAS-userMaster = iwuser{@ismaster} as userMaster Query index=dolphin sourcetype=teamsite_xml | table nextID userName userEmail userUI userMaster ... View more

I want to display a modified time-picker that shows only the following preset choices: Last 24 hours Last 3 days Last 7 days All other accordions will also be hidden. I have most of this working but I cannot figure out how to remove just: Last 15 minutes Last 60 minutes Last 4 hours Last 30 days and add: Last 3 days My current steps are: 1. vi /opt/splunk/etc/apps/search/appserver/static/dashboard.css 2. Add code below 3. http://svr:8000/en_US/_bump (to force a reload) 4. Refresh the browser dashboard page 5. Click timepicker to validate results: Last 15 minutes Last 60 minutes Last 4 hours Last 24 hours Last 7 days Last 30 days Here is the code: dashboard.css ` .SingleValueHolder { background-image: none; -moz-box-shadow: none; } SingleValue_0_1_0 .SingleValueHolder { background: none; text-align: left; font-weight: normal; width: auto; padding-top: 0; padding-left: 0; font-size: 14px; -moz-box-shadow: none; } /* Hide Majority of accordions / div[id^='relative_view'], div[id^='daterange_view'], div[id^='dateandtimerange_view'], div[id^='advanced_view'], div[id^='realtime_view'] { display: none; } / Within the Presets accoridan*/ /* Hide Real-time column choices / div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(1) { display:none; } / Hide the divider after the Real-time column / div[id^='presets_view'] > div.accordion-body > div > div:nth-child(2) { display:none; } / Hide the 1st column choices in the Relative column / div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(3) { display:none; } /** Display the 2nd column choices in the Relative column / / div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(4) { display:none; } **/ / Hide the divider after the Relative column / div[id^='presets_view'] > div.accordion-body > div > div:nth-child(5) { display:none; } / Hide Other column choices */ div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(6) { display:none; } ` ... View more

I want to display a modified Time Picker that shows ONLY the following preset choices: Last 24 hours Last 3 days Last 7 days I am running v6.4.1 My test process is: vi /opt/splunk/etc/apps/search/appserver/static/dashboard.css add css code (shown below) http://mysvr:8000/en_US/_bump (Bump it to reload) Goto browser and refresh the Dashboard to render new changes Check Timepicker widget to see results I have most of what I want working except it still shows all selections in that column: Last 15 minutes Last 60 minutes Last 4 hours Last 24 hours Last 7 days Last 30 days What code do i need to remove: Last 15 minutes Last 60 minutes Last 4 hours Last 30 days And to add: Last 3 days Here is what I have so far: 1) vi /opt/splunk/etc/apps/search/appserver/static/dashboard.css ` .SingleValueHolder { background-image: none; -moz-box-shadow: none; } SingleValue_0_1_0 .SingleValueHolder { background: none; text-align: left; font-weight: normal; width: auto; padding-top: 0; padding-left: 0; font-size: 14px; -moz-box-shadow: none; } /* Hide Majority of accordions */ div[id^='relative_view'], div[id^='daterange_view'], div[id^='dateandtimerange_view'], div[id^='advanced_view'], div[id^='realtime_view'] { display: none; } /* Within the Presets accordian*/ /* Hide Real-time column choices */ div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(1) { display:none; } /* Hide the divider after the Real-time column */ div[id^='presets_view'] > div.accordion-body > div > div:nth-child(2) { display:none; } /* Hide the 1st column choices in the Relative column */ div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(3) { display:none; } /* Keep/Display the 2nd column choices in the Relative column / /** div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(4) { display:none; } ***/ /* Hide the divider after the Relative column */ div[id^='presets_view'] > div.accordion-body > div > div:nth-child(5) { display:none; } /* Hide Other column choices */ div[id^='presets_view'] > div.accordion-body > div > ul:nth-child(6) { display:none; } ` ... View more

I want to add the Stop & Pause buttons to a dashboard for the user to click when needed (due to its expensive search). I am running Splunk 6.4.1 I am using the Bluecoat ProxySG add-on v3.5.0 and its pre-built panels incorporated into a new Dashboard. The add-on and panels are working fine, showing data, no issues. My current attempt and various files touched: Customize the pre-built panels that come with the add-on to match our env: cd $SPLUNK_HOME/etc/apps/Splunk_TA_bluecoat-proxysg/local cp -r ../default/data . (Make copy of out-of-the-box pre-built panels for modifications) vi ./data/ui/panels/*.xml Insert "index=myindexname" into the already provided tag to match our environment. easy. Created a new Dashboard, adding the 4 pre-built bluecoat panels to it, saved, rendered, anddata is displayed, all looks good so far. Add stop & pause buttons to DB Need help with steps here * ... View more

Problem I am trying to index Blue Coat proxySG log file (BCPSG) with no success. I suspect something wrong with the mapping in transforms.conf & props.conf - but I don't understand how to correlate it to the *.conf entries. I see that the BCPSG log file has a "Fields" definition (shown below) but I don't know how to match up the fields. Also, looks like BC changes its formatting on each version. So far, I am successful in getting rsyslog to bring in the data from BC, b/c I have it config'd to write to disk on the HF (See config file below). But I can't get it to get indexed from there. Below shows how i use tcp but i've also tried the file path in inputs.conf and still no luck. The BC device and log file The BC device is sending data out port: 514 to the Splunk HF successfully and I can store it to disk on the HF This is the header of the BC log file #Software: SGOS 6.7.3.6#015 #Version: 1.0#015 #Fields: date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bluecoat-application-name x-bluecoat-application-operation x-bluecoat-application-groups cs-threat-risk x-bluecoat-transaction-uuid x-icap-reqmod-header(X-ICAP-Metadata) x-icap-respmod-header(X-ICAP-Metadata)#015 On the Heavy Forwarder (HF) /etc/rsyslog.conf $ModLoad imuxsock $ModLoad imjournal $ModLoad imtcp $WorkDirectory /var/lib/rsyslog $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat $IncludeConfig /etc/rsyslog.d/*.conf $OmitLocalLogging on $IMJournalStateFile imjournal.state *.info;mail.none;authpriv.none;cron.none /var/log/messages authpriv.* /var/log/secure mail.* -/var/log/maillog cron.* /var/log/cron *.emerg :omusrmsg:* uucp,news.crit /var/log/spooler local7.* /var/log/boot.log $template splunklog, "/data/splunk_rsyslog/%HOSTNAME%/%PROGRAMNAME%.log" $template splunkmsg, "%rawmsg%\n" $template bluecoatlog, "/data/splunk_rsyslog/%fromhost%/%$year%-%$month%-%$day%-%$hour%-syslog.log" $RuleSet bluecoat $DirOwner root $DirGroup splunk $FileOwner root $FileGroup splunk $DirCreateMode 0750 $FileCreateMode 0640 *.* ?bluecoatlog;splunkmsg $InputTCPServerBindRuleset bluecoat $InputTCPServerRun 514 /opt/splunk/etc/apps/Splunk_TA_bluecoat-proxysg/local inputs.conf [tcp://514] source = tcp.bluecoat sourcetype = bluecoat:proxysg:access:syslog disabled = false transforms.conf I added this stanza to support 6.7.x format `[auto_kv_for_bluecoat_v6_7_x] REGEX = (?:"([^"]+)"|(\S+))\s+(?:"(\d{1,2}:\d{1,2}:\d{1,2})"|(\d{1,2}:\d{1,2}:\d{1,2}))\s+(?:"(\d+)"|(\d+))\s+(?:"(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})"|(\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3}))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s+(?:"([^"]+)"|(\S+))\s*$ FORMAT = date::$1 date::$2 time::$3 time::$4 time_taken::$5 time_taken::$6 c_ip::$7 c_ip::$8 cs_username::$9 cs_username::$10 cs_auth_group::$11 cs_auth_group::$12 x_exception_id::$13 x_exception_id::$14 sc_filter_result::$15 sc_filter_result::$16 cs_categories::$17 cs_categories::$18 cs_Referer::$19 cs_Referer::$20 sc_status::$21 sc_status::$22 s_action::$23 s_action::$24 cs_method::$25 cs_method::$26 rs_Content_Type::$27 rs_Content_Type::$28 cs_uri_scheme::$29 cs_uri_scheme::$30 cs_host::$31 cs_host::$32 cs_uri_port::$33 cs_uri_port::$34 cs_uri_path::$35 cs_uri_path::$36 cs_uri_query::$37 cs_uri_query::$38 cs_uri_extension::$39 cs_uri_extension::$40 cs_User_Agent::$41 cs_User_Agent::$42 s_ip::$43 s_ip::$44 sc_bytes::$45 sc_bytes::$46 cs_bytes::$47 cs_bytes::$48 x_virus_id::$49 x_virus_id::$50 x_bluecoat_application_name::$51 x_bluecoat_application_name::$52 x_bluecoat_application_operation::$53 x_bluecoat_application_operation::$54 FORMAT = date::$1 date::$2 time::$3 time::$4 time_taken::$5 time_taken::$6 c_ip::$7 c_ip::$8 cs_username::$9 cs_username::$10 cs_auth_group::$11 cs_auth_group::$12 s-supplier-name::$13 s-supplier-name::$14 s-supplier-ip::$15 s-supplier-ip::$16 s-supplier-country::$17 s-supplier-country::$18 s-supplier-failures::$19 s-supplier-failures::$20 x-exception-id::$21 x-exception-id::$22 sc-filter-result::$23 sc-filter-result::$24 cs-categories::$25 cs-categories::$26 cs_Referer::$27 cs_Referer::$28 sc_status::$29 sc_status::$30 s-action::$31 s-action::$32 cs-method::$33 cs-method::$34 rs_Content_Type::$35 rs_Content_Type::$36 cs_uri_scheme::$37 cs_uri_scheme::$38 cs_host::$39 cs_host::$40 cs_uri_port::$41 cs_uri_port::$42 cs_uri_path::$43 cs_uri_path::$44 cs_uri_query::$45 cs_uri_query::$46 cs_uri_extension::$47 cs_uri_extension::$48 cs_User_Agent::$49 cs_User_Agent::$50 s_ip::$51 s_ip::$52 sc_bytes::$53 sc_bytes::$54 cs_bytes::$55 cs_bytes::$56 x_virus_id::$57 x_virus_id::$58 x_bluecoat_application_name::$59 x_bluecoat_application_name::$60 x_bluecoat_application_operation::$61 x_bluecoat_application_operation::$62 cs_threat_risk::$63 cs_threat_risk::$64` props.conf [bluecoat:proxysg:access:syslog] pulldown_type = true category = Network & Security description = Data from Blue Coat ProxySG in W3C ELFF format thru syslog KV_MODE = none SHOULD_LINEMERGE = false MAX_DAYS_AGO = 10951 TRUNCATE = 20000 REPORT-auto_kv_for_bluecoat_v5 = auto_kv_for_bluecoat_v5_3_3 REPORT-auto_kv_for_bluecoat_v6 = auto_kv_for_bluecoat_v6_7_x,auto_kv_for_bluecoat_v6_5_x ... View more