About tscroggins

tscroggins · ‎04-17-2022

@delly_fofie You should handle that special case in whatever way makes sense for your data. My example used epoch values. If your data uses signed 32-bit epoch values, you might treat the latest value as 2147483648 (one more than the max epoch value): <eval token="other_time_tok_latest">if($job.latestTime$!="", strptime($job.latestTime$, "%Y-%m-%dT%H:%M:%S.%3N%z"), 2147483648)</eval> The solution you use is dependent on your source data. You may need to further modify the example.

tscroggins · ‎04-16-2022

@RockWarriorP The user interface is not meant to be modified, and the fields provided by the alerts/fire_alerts/{name} endpoint are document here: <https://docs.splunk.com/Documentation/Splunk/8.2.6/RESTREF/RESTsearch#alerts.2Ffired_alerts.2F.7Bname.7D>: actions Any additional alert actions triggered by this alert. alert_type Indicates if the alert was historical or real-time. digest_mode expiration_time_rendered savedsearch_name Name of the saved search that triggered the alert. severity Indicates the severity level of an alert. Severity level ranges from Info, Low, Medium, High, and Critical. Default is Medium. Severity levels are informational in purpose and have no additional functionality. sid The search ID of the search that triggered the alert. trigger_time The time the alert was triggered. trigger_time_rendered triggered_alerts Have you looked at Splunk IT Essentials Work, Splunk IT Service Intelligence, or Splunk Enterprise Security as more robust solutions, depending on your use cases? You might also consider developing your own alert dashboards or using a third-party solution like Alert Manager (not an endorsement).

tscroggins · ‎04-16-2022

@Jaylon Can you provide more context? The search you provided isn't a functional standalone search in any version of Splunk.

tscroggins · ‎04-16-2022

@msg4sunil Have you tried this? timeformat="%m/%d/%Y:%H:%M:%S%Z" starttime="07/04/2021:09:48:00Z" endtime="07/04/2021:09:48:59Z" Or this: timeformat="%Y-%m-%dT%H:%M:%S.%3N%Z" starttime="2021-07-04T09:48:00.000Z" endtime="2021-07-04T09:48:59.999Z" But you probably want this: timeformat="%Y-%m-%dT%H:%M%Z" starttime="2021-07-04T09:48Z" endtime="2021-07-04T09:49Z" unless you explicitly have millisecond precision and want to use that as an upper bound. Time ranges should be read as starttime (or earliest) >= T0 and endtime (or latest) < T1. Date and time format variables are documented at <https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Commontimeformatvariables>.

tscroggins · ‎04-16-2022

@lalitha What values for dv_state to do you see after running this search? index="servicenow" sourcetype="snow:sc_task" AND sys_class_name="sc_task" | fillnull "UnAssigned" dv_assigned_to | stats latest(*) as * by dv_number | stats count by dv_state

tscroggins · ‎04-16-2022

@timsheets13 Does the "notable" index exist?

tscroggins · ‎04-16-2022

@Aziz94 Which version of Splunk Enterprise Security are you running? Splunk Enterprise Security 7.0 introduces the Executive Summary and SOC Operations dashboards and the Mean Time to Triage and Mean Time to Resolution metrics. Whether you use Splunk Enterprise Security 7.0 or not, you can download the app from Splunkbase and read the metric searches for inspiration. (It may not be appropriate to copy and paste them here.)

tscroggins · ‎04-16-2022

@Poojitha If you're installing the RPM, the preinstall scriptlet does this: if [ -x "$SPLUNK_HOME/bin/splunk" ] ; then echo "This looks like an upgrade of an existing Splunk Server. Attempting to stop the installed Splunk Server..." $SPLUNK_HOME/bin/splunk stop fi The installer doesn't confirm whether the stop was successful. Does your Ansible playbook include a step to stop Splunk before you attempt to upgrade in place? After stopping Splunk, you should confirm 1) Splunk stopped successfully and 2) $SPLUNK_HOME/var/run/splunk/splunkd.pid was removed. You can both confirm Splunk is stopped and automatically remove the .pid file if present by running '$SPLUNK_HOME/bin/splunk status'.

tscroggins · ‎04-16-2022

@gheribhai1234 By "records," do you mean Splunk events or JSON array elements? An answer using spath or native JSON key-value mode depends on the context. If you're okay with an approximate value for character length, you can try a regex approach. This assumes all values of aValues are strings and includes whitespace, quotes, and commas. It's not a replacement for a JSON parser. index=myIndex | rex "(?s)aValues\\s*:\\s*\\[\\s*(?<aValues>['\"].*?['\"])\\s*\\]" | stats sum(eval(len(aValues))) as len_aValues

tscroggins · ‎04-16-2022

@muebel btool includes a "check" command, which I believe does simple .conf.spec validation, similar to Splunk startup. AppInspect includes various checks. The official Visual Studio Code Extension for Splunk includes .conf linting. I've not used it, so I can't comment on its quality or accuracy. How deep down the lint rabbit hole do you plan to go? It's perhaps too late to break PC-lint's continuously advertised software record. 😉 (I do miss Dr. Dobb's Journal.)

tscroggins · ‎04-16-2022

@delly_fofie We can use the first option proposed by @niketn in Running one of two searches based on time picker selection: <form> <label>delly_fofie_time_filters</label> <search> <query>| makeresults</query> <earliest>$other_time_tok.earliest$</earliest> <latest>$other_time_tok.latest$</latest> <done> <eval token="other_time_tok_earliest">strptime($job.earliestTime$, "%Y-%m-%dT%H:%M:%S.%3N%z")</eval> <eval token="other_time_tok_latest">strptime($job.latestTime$, "%Y-%m-%dT%H:%M:%S.%3N%z")</eval> </done> </search> <fieldset submitButton="false"> <input type="time" token="_time_tok"> <label>Time</label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> <input type="time" token="other_time_tok"> <label>Other Time</label> <default> <earliest>-24h@h</earliest> <latest>now</latest> </default> </input> </fieldset> <row> <panel> <table> <search> <query>| makeresults | addinfo | eval other_time_tok_earliest=$other_time_tok_earliest$, other_time_tok_latest=$other_time_tok_latest$</query> <earliest>$_time_tok.earliest$</earliest> <latest>$_time_tok.latest$</latest> </search> </table> </panel> </row> </form> Note that I've updated the code for Splunk Enterprise 8.2. Simple XML and Splunk Web use JavaScript to parse eval elements. The values of job.earliestTime and job.latestTime have formats that may change between Splunk versions, and strptime() format specifiers are converted internally to Moment.js specifiers. In this Simple XML example, the first time input with token _time_tok is referenced by the table search earliest and latest values. The second time input with token other_time_tok is referenced by a global search that uses Splunk's job engine to parse the values and store the earliest and latest epoch values in other_time_tok_earliest and other_time_tok_latest, respectively. You can then reference the other_time_tok_* tokens in any search or other dashboard element. I've used them with an eval command to display their values, but you can use them anywhere you'd like, including in search (implied or explicit) and where commands.

tscroggins · ‎01-22-2022

@btshivanand I keep the flowcharts published at https://wiki.splunk.com/Community:HowIndexingWorks handy as a reference. Look at: index=_internal source=*metrics.log* host=<yourhost> blocked=true parsingQueue itself could be full, but as @isoutamo noted, the most likely cause is a blocked output queue, which in turn blocks indexQueue, typingQueue, aggQueue, and parsingQueue on the forwarder. If an output queue is blocked, check the receiver using the same process. If indexQueue is blocked on an indexer, make sure 1) you have enough free disk space for hot data and 2) your storage is fast enough to keep up with your aggregate ingest rate amortized across your indexer or cluster. If your heavy forwarder has multiple outputs, a single blocked output will eventually block indexQueue, which will in turn block all outputs in the pipeline. This is especially common with syslog outputs, which have a fixed queue length / buffer size.

tscroggins · ‎01-19-2022

@a_m_s This works as long as File and User never contain a hyphen. There's no 100% safe method of concatenating and later separating strings for grouping without storing and comparing the original values, but if you understand your dictionary, you can choose values that reduce the probability of a collision. The values provided in the original question may just be examples.

tscroggins · ‎01-17-2022

@dokaas_2 You've adapted this better than I have! I was looking for ways to define and group FFT output by specific features, e.g. src-dest tuples. What general form did your base search take?

tscroggins · ‎01-17-2022

@dokaas_2 You can extend Splunk Machine Learning Toolkit to include the FFT algorithm. The following is an example adapted from https://www.ritchievink.com/blog/2017/04/23/understanding-the-fourier-transform-by-example/. First, let's generate the sample data: | makeresults count=500 | streamstats count as t | eval t=exact(t/1000)-0.001, s=sin(40*2*pi()*t)+0.5*sin(90*2*pi()*t) | table t s We should have signals with frequencies of 40 and 90 cycles. Next, let's add our algorithm stanza to $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/local/algos.conf: [FFT] Restart Splunk to enable the algorithm. Next, let's write the algorithm interface in $SPLUNK_HOME/etc/apps/Splunk_ML_Toolkit/bin/algos/FFT.py. This is just an example with no input validation: #!/usr/bin/env python import numpy as np import pandas as pd from base import BaseAlgo class FFT(BaseAlgo): def __init__(self, options): # Option checking & initializations here pass def fit(self, df, options): # Fit an estimator to df, a pandas DataFrame of the search results s = df[self.target_variable] t = df[self.feature_variables] fft = np.fft.fft(s) T = t[t.columns[0]][1] - t[t.columns[0]][0] N = fft.size freq = np.linspace(0, 1 / T, N)[:N // 2] amp = np.abs(fft)[:N //2 ] * 1 / N df = pd.DataFrame({'Frequency': freq, 'Amplitude': amp}, columns=['Frequency', 'Amplitude']) return df Finally, let's try the algorithm with the fit command: | makeresults count=500 | streamstats count as t | eval t=exact(t/1000)-0.001, s=sin(40*2*pi()*t)+0.5*sin(90*2*pi()*t) | table t s | fit FFT s from t Signals were detected at 40 and 90 cycles with the amplitudes (halved) shown. If you have a sample data set, we can test it directly.

tscroggins · ‎01-17-2022

@aquinojason To easily summarize values over time, you can use the timechart command: index=xxxxxxx sourcetype="xxxxxx" EXPRSSN=IBM4D* | timechart span=1d avg(MIPS) ```or max(MIPS), p90(MIPS), etc.``` Core Splunk does not include a linear trendline command, but you can create one yourself using SPL. See https://wiki.splunk.com/Community:Plotting_a_linear_trendline for an old example. Splunk Machine Learning Toolkit does include a linear regression algorithm for the fit command: index=xxxxxxx sourcetype="xxxxxx" EXPRSSN=IBM4D* | timechart span=1d avg(MIPS) as MIPS | fit LinearRegression MIPS from _time You can visualize your data as an area chart and then configure predicted(MIPS) as an overlay to show a linear trend. Here's an example using Splunk introspection events: | tstats max(data.normalized_pct_cpu) as pct_cpu where index=_introspection host=splunk by _time span=10s | fit LinearRegression pct_cpu from _time I'm not sure what the red line in your chart represents. If you want to add a moving average to your chart, you can use the trendline command: | tstats max(data.normalized_pct_cpu) as pct_cpu where index=_introspection host=splunk by _time span=10s | trendline sma6(pct_cpu) | fit LinearRegression pct_cpu from _time

tscroggins · ‎01-17-2022

@chenyt Yes, the configuration you define on the search head--users, roles, etc.--will be pushed to the indexers in a bundle used during the search. Different search heads can define different authentication and authorization settings. Splunk security is decentralized. While you can and should define strict authorization settings on your indexers through configuration deployed by your cluster manager, your users should access the environment through the search head.

tscroggins · ‎01-17-2022

@ravinayan_acc If we assume you're using Splunk-provided operating system add-ons with performance inputs enabled on default indexes, and your lookup file contains one Server and Application entry per row, you can use this: tag=performance tag=storage [| inputlookup Server_details.csv where Application="app name" | table Server | rename Serer as host ] | stats latest(storage_used_percent) by host mount The tags will limit search results to storage metrics. The subsearch will limit search results to your application servers by application name.

tscroggins · ‎01-17-2022

@jip31 The earliest and latest values are available as tokens: <drilldown> <set token="earliest_tok">$earliest$</set> <set token="latest_tok">$latest$</set> </drilldown> You can use these tokens in your other visualization searches: <search> <query>index=toto sourcetype=tutu ezconf=$ezconf$ | eval time = strftime(_time, "%d-%m-%y %H:%M") | sort - time | table time hang</query> <earliest>$earliest_tok$</earliest> <latest>$latest_tok$</latest> </search> You can initialize earliest_tok and latest_tok to default values using a time input or a combination of <init> and <set> tags.

tscroggins · ‎01-17-2022

@chenyt You should send REST API requests to your search head: https://yourhostname:8089. In most cases, REST API access to the your indexers should be limited to other Splunk instances. Your search head provides the authentication and authorization configuration necessary to control access to your data.

tscroggins · ‎01-17-2022

@syedikramulla The date_wday field may already be extracted from your events, but but be mindful of time zone differences between the raw data and your Splunk settings: index="XYZ" sourcetype=Logs (LOGLEVEL=ERROR OR LOGLEVEL=FATAL OR LOGLEVEL=INFO) date_wday!=saturday date_wday!=sunday | timechart span=1d count If date_wday isn't present or if you don't trust the vale: index="XYZ" sourcetype=Logs (LOGLEVEL=ERROR OR LOGLEVEL=FATAL OR LOGLEVEL=INFO) | eval Day=strftime(_time, "%w") | where Day>0 AND Day<6 | timechart span=1d count

tscroggins · ‎01-16-2022

@piukr The Simple XML <eval> tag implements a subset of eval functions in JavaScript. Internally, multivalued results are JavaScript arrays. They are collapsed to comma-delimited strings when returned as values. See $SPLUNK_HOME/share/splunk/search_mrsparkle/exposed/js/util/eval/functions/multivalue.js for the list of supported functions: split() mvappend() mvcount() mvfind() mvindex() mvjoin() mvdedup() mvsort() mvrange() mvzip() mvfilter() Unfortunately, mvmap() is not supported. Props to the dashboards and visualizations team for supporting as much as they did!

tscroggins · ‎01-16-2022

@klim The fields appear to be added by the summary index processor when the stash file is written to disk. Is there a particular reason you do not want the data to appear in _raw?

tscroggins · ‎01-16-2022

@anilbabuk If you have a data model, you can use the pivot command: | pivot My_Data_Model My_Dataset count(My_Dataset) SPLITROW File SPLITROW User SPLITCOL Date | sort User File | foreach */*/* [ eval <<FIELD>>=nullif('<<FIELD>>', 0) ] If you do not have a data model, you'll need to manipulate the output of chart, stats, etc. to split columns by multiple row fields: ```base search here``` | stats count by File User Date | eval {Date}=count | stats max(*/*/*) as */*/* by File User | sort User File

tscroggins · ‎01-16-2022

@kevinsteeee The host's routing table determines the default destination for non-local traffic. Typically, a host with multiple interfaces would have a single default route associated with the highest priority interface. In this example, non-local traffic would traverse eth0 and have source address 192.168.100.100: $ route -n Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.100.1 0.0.0.0 UG 100 0 0 eth0 192.168.100.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.101.0 0.0.0.0 255.255.255.0 U 101 0 0 eth1 192.168.102.0 0.0.0.0 255.255.255.0 U 102 0 0 eth2 192.168.103.0 0.0.0.0 255.255.255.0 U 103 0 0 eth3 If you have multiple addresses associated with one interface, you should still have one default route: $ route -n Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.100.1 0.0.0.0 UG 100 0 0 eth0 192.168.100.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.101.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.102.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.103.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 If, however, you have multiple default routes, they may be selected in a round-robin fashion, and your source address would vary between .100.100, .101.100, .102.100, and .103.100. $ route -n Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.100.1 0.0.0.0 UG 100 0 0 eth0 0.0.0.0 192.168.101.1 0.0.0.0 UG 100 0 0 eth0 0.0.0.0 192.168.102.1 0.0.0.0 UG 100 0 0 eth0 0.0.0.0 192.168.103.1 0.0.0.0 UG 100 0 0 eth0 192.168.100.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.101.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.102.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 192.168.103.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0 Routing table or policy management varies by operating environment.

Posts	538
Solutions	96
Karma Given	67
Karma Received	185
Member Since	‎08-13-2014

Online Status	Offline
Date Last Visited	Sunday

Chart command generates duplicate bins when span a...

Re: Multiple Time filters on a Dashboard

Re: Is there a way that I can modify the categorie...

Re: Why does timechart not work?

Re: How to search in UTC and of the format yyyy-mm...

Re: How to get not closed Incidents/Tasks/Changenu...

Re: Not getting Notables

Re: How to create a Dashboard that will show SLA f...

Re: Error while installing splunk universal forwar...

Re: How to write Array Within Array for Million re...

Re: Script to lint and validate splunk config?

Re: Multiple Time filters on a Dashboard

Re: Indexing delays due the monitor input cannot p...

Re: Group by multiple attribute and get count per ...

Re: Detecting Beaconing Using Fourier Transform (F...

Re: Detecting Beaconing Using Fourier Transform (F...

Re: Adding a Linear Trendline and customizing char...

Re: Which instance should I send REST API to?

Re: lookup CSV file and then search server disk pe...

Re: help on timechart click value

Re: Which instance should I send REST API to?

Re: Display table base on the if condition

Re: Using mvmap in eval token

Re: Removing field from _raw field generated by co...

Re: Group by multiple attribute and get count per ...

Re: On what basis does the Universal Forwarder det...