Hi @silverKi, The maxDataSize for your hot buckets is 1 MB. Your friend's setting appears to be higher (5 MB). To add to what's already been written, you're writing (compressed) data at different rates: Friend: ~720 bytes per second You: ~19 bytes per second This will influence the size of the warm bucket after it rolls from hot when either maxDataSize (1 MB in your case) or the default maxHotSpanSecs value of 90 days has been exceeded. Hot buckets can also roll to warm when Splunk is restarted or when triggered manually. That probably isn't happening here, but it's worth noting. ... View more

Looks good! Just be mindful of the difference between the plus (+) operator and the dot (.) operator. Plus concatenates strings and adds numbers, but dot concatenates both strings and numbers as strings. If you're unsure of the order of operations or the variable value in other contexts, you can wrap the variables in the tostring() function. It's interesting that Splunk also assumed a leading space would work in the SC4S transforms: [metadata_time] SOURCE_KEY = _time REGEX = (.*) FORMAT = t="$1$0 DEST_KEY = _raw @PickleRick's suggestion to escape the space with i.e. a blackslash just evaluates to a literal backslash followed by a space. ... View more

Hi @BrianLam, You can retrieve the search results using the search/v2/jobs/{search_id}/results endpoint. See https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTsearch#search.2Fv2.2Fjobs.2F.7Bsearch_id.7D.2Fresults. The search_id value is specific to the instance of the search that generated the alert. It's a simple GET request. The default output mode is XML. If you want JSON output, pass the output_mode query parameter as part of the GET request: https://splunk:8089/services/search/v2/jobs/scheduler__user__app__xxx_at_xxx_xxx/results?output_mode=json   ... View more

Hi @Jado95, Is your question specific to Splunk Add-on for Cisco ASA or Cisco ASA itself? The message format is defined by Cisco ASA, and the add-on implementation should agree with Cisco ASA documentation at https://www.cisco.com/c/en/us/td/docs/security/asa/syslog/b_syslog/syslog-messages-302003-to-342008.html: 302013 ... If inbound is specified, the original control connection was initiated from the outside. For example, for FTP, all data transfer channels are inbound if the original control channel is inbound. If outbound is specified, the original control connection was initiated from the inside. ... 302015 ... If inbound is specified, then the original control connection is initiated from the outside. For example, for UDP, all data transfer channels are inbound if the original control channel is inbound. If outbound is specified, then the original control connection is initiated from the inside. The corresponding teardown events, 302014 and 302106, do not specify a direction, so without prior knowledge, the field extraction can't know which address is the initiator. If needed, you can correlate the events by the session_id field. This example is slow and ugly; it's only meant to demonstrate the correlation: | eventstats values(direction) as direction by session_id | eval src_ip_tmp=src_ip, dest_ip_tmp=dest_ip, src_ip=if(lower(vendor_action)=="teardown" && lower(direction)=="outbound", dest_ip_tmp, src_ip_tmp), dest_ip=if(lower(vendor_action)=="teardown" && lower(direction)=="outbound", src_ip_tmp, dest_ip_tmp) | fields - src_ip_tmp dest_ip_tmp   ... View more

I would use list() instead of values() to prevent removal of duplicates and wrap the product in exact() to prevent rounding errors: | makeresults format=csv data="value_a 0.44 0.25 0.67 0.44" | stats list(value_a) as value_a | eval "product(value_a)"=1 | foreach value_a mode=multivalue [ eval "product(value_a)"=exact('product(value_a)' * <<ITEM>>) ] | table "product(value_a)"  => product(value_a) 0.032428 ... View more

Hi @madhav_dholakia, In Simple XML, you can generate categorical choropleth maps with color-coded city boundaries: In Dashboard Studio, however, choropleth maps are limited to numerical distributions, e.g. OpenIssues by city, and the geospatial lookup geometry isn't always interpreted correctly: In both examples, I've used mapping data published by the Office for National Statistics at https://geoportal.statistics.gov.uk. Search for BDY_TCITY DEC_2015 to download the corresponding KML file. As a compromise, you can use a marker map to display color-coded markers at city centers by latitude and longitude: Here's the source: { "visualizations": { "viz_KxsdmDQb": { "type": "splunk.map", "options": { "center": [ 52.560559999999924, -1.4702799999984109 ], "zoom": 6, "layers": [ { "type": "marker", "dataColors": "> primary | seriesByName('Status') | matchValue(colorMatchConfig)", "choroplethOpacity": 0.75, "additionalTooltipFields": [ "Status", "StoreID", "City", "OpenIssues" ], "latitude": "> primary | seriesByName('lat')", "longitude": "> primary | seriesByName('lon')" } ] }, "context": { "colorMatchConfig": [ { "match": "Dormant/Green", "value": "#118832" }, { "match": "Warning/Amber", "value": "#cba700" }, { "match": "Critical/Red", "value": "#d41f1f" } ] }, "dataSources": { "primary": "ds_CtvaIPJ3" } } }, "dataSources": { "ds_CtvaIPJ3": { "type": "ds.search", "options": { "query": "| makeresults format=csv data=\"StoreID,City,OpenIssues,Status,lat,lon\r\nStore 1,London,3,Critical/Red,51.507222,-0.1275\r\nStore 2,York,2,Warning/Amber,53.96,-1.08\r\nStore 3,Bristol,0,Dormant/Green,51.453611,-2.5975\r\nStore 4,Liverpool,1,Warning/Amber,53.407222,-2.991667\" \r\n| table StoreID City OpenIssues Status lat lon", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "name": "Choropleth map search" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": {} } } } }, "inputs": {}, "layout": { "type": "absolute", "options": { "width": 918, "height": 500, "display": "auto" }, "structure": [ { "item": "viz_KxsdmDQb", "type": "block", "position": { "x": 0, "y": 0, "w": 918, "h": 500 } } ], "globalInputs": [] }, "description": "", "title": "eaw_store_status_ds" } As a static workaround, the Choropleth SVG visualization allows you to upload a custom image, e.g. a stylized map of England and Wales, and define custom SVG boundaries and categorical colors. The Dashboard Studio documentation includes a basic tutorial at https://docs.splunk.com/Documentation/Splunk/latest/DashStudio/mapsChorSVG. ... View more

Hi @CyberWolf, There's a tendency among practitioners to bin time into buckets rounded to the nearest time interval, e.g. 1 hour: 00:00, 01:00, 02:00, etc.; however, this results in counting errors. Instead, count using a rolling window in ascending _time order:   index=VPN_Something status=failure | stats count by _time user | streamstats time_window=1h sum(count) as failure_count by user | where failure_count>5   Since you're only interested in a 1-hour window, your search time range only needs span the last hour plus any allowance for ingest lag and a buffer to accommodate your scheduling interval. See https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Streamstats for information on scaling streamstats. If you're using accelerated data models or indexed fields or if your raw events are structured in key-value pairs separated by minor breakers, you can use tstats to greatly improve the performance of the search. If you have the capacity, you might also consider a real-time search that counts events as they're indexed, although the results may be incorrect relative to your requirements. If you have Splunk Enterprise Security, look at the "Access - Excessive Failed Logins - Rule" correlation search. For reference, it's a real-time search scheduled every 5 minutes (*/5 * * * *), with earliest=rt-65m@m and latest=rt-5m@m:   | from datamodel:"Authentication"."Failed_Authentication" | stats values("tag") as "tag",dc("user") as "user_count",dc("dest") as "dest_count",count by "app","src" | where 'count'>=6   In the Authentication data model, app would be something like vpn, and src would be a device identifier. As @isoutamo wrote, there are many approximate solutions to this problem. The correct solution depends on your requirements and your tolerance for counting errors. ... View more

--if your base search includes N1, N2, N3, ... Nn, you can add additional logic to generate the N-value dynamically:   | appendpipe [| search Category=N* | stats count sum(*) as * | eval Category="N".tostring(count+1) | fields - count ]   Or if you want to sub all Category values:   | appendpipe [| rex field=Category "(?<Category>[^\d]+)" | stats count sum(*) as * by Category | eval Category=Category.tostring(count+1) | fields - count ]   => Category A B C D E F N1 1 2 4 2 4 1 N2 0 5 4 3 5 7 M1 1 0 1 0 4 3 M2 1 1 3 5 0 1 U1 0 4 6 5 4 3 M3 2 1 4 5 4 4 N3 1 7 8 5 9 8 U2 0 4 6 5 4 3 But note that you'll need to add custom sorting logic if you want something other than the default sort order. ... View more

You can put whatever you want in the label argument. It sets the value of the field specified by labelfield in the totals row: | addcoltotals labelfield=Category label="Total" | addcoltotals labelfield=Category label="SUM[n1..nN]" | addcoltotals labelfield=Category label="Life, the Universe and Everything" ... View more

Hi @MachaMilkshake, The addcoltotals command should do exactly what you need: | addcoltotals labelfield=Category label=N3 ... View more

Hi @ranjith4, What is the aggregate throughput of all sources? If you're unsure, what is the peak daily ingest of all sources? Splunk Universal Forwarder uses very conservative default queue sizes and a throughput limit of 256 KBps. As a starting point, you can disable the throughput limit in $SPLUNK_HOME/etc/system/local/limits.conf: [thruput] maxKBps = 0 If the forwarder is still not delivering data as quickly as it arrives, we can adjust output queue sizes based on your throughput (see Little's Law). As @PickleRick noted, the forwarder may be switching to an effectively single-threaded batch mode when reading files larger than 20 MB. Increase the min_batch_size_bytes setting in limits.conf to a value larger than your largest daily file or some other arbitrarily large value [default] # 1 GB min_batch_size_bytes = 1073741824 If throughput is still an issue, you can enable additional parallel processing with the server.conf parallelIngestionPipelines setting, but I wouldn't do that until after tuning other settings. ... View more

You can use the regex command to filter by a regular expression, but it's slower and more cumbersome than just combining TERM() functions in a search predicate. As alternatives, you can extract and normalize a mac field at index time with a combination of transforms or you can create a single-field data model that acts as a secondary time series index. For the latter, create a search-time field extraction using a transform with MV_ADD = true to capture strings that look like MAC addresses matching your 48-bit patterns (xx-xx-xx-xx-xx-xx, xx:xx:xx:xx:xx:xx, and xxxx.xxxx.xxxx). For example, using source type mac_addr: # props.conf [mac_addr] REPORT-raw_mac = raw_mac # transforms.conf [raw_mac] CLEAN_KEYS = 0 MV_ADD = 1 REGEX = (?<raw_mac>(?<![-.:])\b(?:[0-9A-Fa-f]{2}(?:(?(2)(?:\2)|([-:]?))[0-9A-Fa-f]{2}){5}|[0-9A-Fa-f]{4}(?:(\.)[0-9A-Fa-f]{4}){2})\b(?!\2|\3)) Create a subsequent calculated (eval) field that removes separators: # props.conf [mac_addr] REPORT-raw_mac = raw_mac EVAL-mac = mvdedup(mvmap(raw_mac, replace(raw_mac, "[-.:]", ""))) Then, define and accelerate a data model with a single dataset and field: # datamodels.conf [my_mac_datamodel] acceleration = true # 1 month, for example acceleration.earliest_time = -1mon acceleration.hunk.dfs_block_size = 0 # data/models/my_mac_datamodel.xml { "modelName": "my_mac_datamodel", "displayName": "my_mac_datamodel", "description": "", "objectSummary": { "Event-Based": 0, "Transaction-Based": 0, "Search-Based": 1 }, "objects": [ { "objectName": "my_mac_dataset", "displayName": "my_mac_dataset", "parentName": "BaseSearch", "comment": "", "fields": [ { "fieldName": "mac", "owner": "my_mac_dataset", "type": "string", "fieldSearch": "mac=*", "required": true, "multivalue": false, "hidden": false, "editable": true, "displayName": "mac", "comment": "" } ], "calculations": [], "constraints": [], "lineage": "my_mac_dataset", "baseSearch": "index=main sourcetype=mac_addr" } ], "objectNameList": [ "my_mac_dataset" ] } All of the above can be added to a search head using SplunkWeb settings in the following order: Define shared field transformation. Define shared field extraction. Define shared calculated field. Define shared data model. Finally, use the datamodel command to optimize the search: | datamodel summariesonly=t my_mac_datamodel my_mac_dataset flat | search mac=12EA5F7211AB Note that some undocumented conditions (source type renaming?) may force Splunk to disable the optimizations used by the datamodel command when distributing the search, in which case it will be no faster than a regular search of the extracted mac field. If it's working correctly, the search log should include an optimized search with a READ_SUMMARY directive as well as various ReadSummaryDirective log entries. The datamodel command with the flat argument will return the raw events and the undecorated mac field values, but no other extractions will be performed. ... View more

Hi @inventsekar, 1) As I recall, I only generated the lookup CSV file for testing in Tamil. An all-language lookup might be size prohibitive. The best SPL-based workaround to count all graphemes seems to be an eval expression using the \X regular expression token to match Unicode sequences. The simplest expression was: | eval count=len(replace(field, "\\X", "x")) 2) The external lookup allowed programmatic access to Python modules or any other library/program if called from an arbitrary script. The example returned a Unicode character category, but the subsequent counting solution wasn't comprehensive. In Bash, calculating the number of characters may be as simple as: echo ${#field} => 58 but this suffers the same problem as our earlier efforts by not taking into account marks and code sequences used to generate graphemes. Is Perl better? perl -CS -lnE 'say length' <<<${field} => 58 As before, the length is incorrect. I'm not a Perl expert, but see https://perldoc.perl.org/perlunicode: The only time that Perl considers a sequence of individual code points as a single logical character is in the \X construct .... That leads us to: perl -CS -lnE 's/\X/x/g; say length' <<<${field} => 37 There may be a better native Perl, Python, etc. solution, but calling an external program is more expensive than the equivalent SPL. 3) If you only need to count graphemes, I would use the eval command. What other use cases did you have in mind? ... View more

Hi @siva_kumar0147, No, I only use makeresults to generate sample data. The logic from the sort command down drives the visualization. ... View more

This was a fun thread! I upvoted https://ideas.splunk.com/ideas/EID-I-2176. ... View more

Hi @zerocoolspain, I would use separate but similar radio inputs. Each radio input has its own set of tokens; however, updating a radio input also updates the global trobots token. The currently selected trobots1 and trobots2 values are preserved across changes to the tintervalo token. <form version="1.1" theme="light"> <label>intervalo</label> <init> <unset token="trobots"></unset> </init> <fieldset> <input type="dropdown" token="tintervalo" searchWhenChanged="true"> <label>Intervalo</label> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusoultimasemana">Última semana completa</choice> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusoultimomes">Último mes completo</choice> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusoultimotrimestre">Último trimestre completo</choice> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusoultimoaño">Último año completo</choice> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusomescurso">Mes en curso</choice> <choice value="|loadjob savedsearch=&quot;q71139x:vap:precalculoVAPusoañoencurso">Año en curso</choice> <choice value="7">Otros</choice> <change> <condition match="'tintervalo'==7"> <set token="show_trobots1">true</set> <unset token="show_trobots2"></unset> <set token="trobots">$trobots1$</set> </condition> <condition match="'tintervalo'!=7"> <unset token="show_trobots1"></unset> <set token="show_trobots2"></set> <set token="trobots">$trobots2$</set> </condition> </change> </input> <input type="radio" token="trobots1" depends="$show_trobots1$" id="inputRadioRI1" searchWhenChanged="true"> <label>Robots</label> <choice value="| eval delete=delete">Yes</choice> <choice value="`filter_robots` `filter_robots_ip`">No</choice> <initialValue>`filter_robots` `filter_robots_ip`</initialValue> <change> <set token="trobots">$trobots1$</set> </change> </input> <input type="radio" token="trobots2" depends="$show_trobots2$" id="inputRadioRI2" searchWhenChanged="true"> <label>Robots</label> <choice value="conBots">Yes</choice> <choice value="sinBots">No</choice> <initialValue>sinBots</initialValue> <change> <set token="trobots">$trobots2$</set> </change> </input> </fieldset> <row> <html> <table> <tr> <td><b>tintervalo:</b></td><td>$tintervalo$</td> </tr> <tr> <td><b>trobots1:</b></td><td>$trobots1$</td> </tr> <tr> <td><b>trobots2:</b></td><td>$trobots2$</td> </tr> <tr> <td><b>trobots:</b></td><td>$trobots$</td> </tr> </table> </html> </row> </form>   ... View more

Hi @venal, Experience querying and pivoting relational data provides a solid foundation for analyzing data in Splunk; however, Splunk data is stored as a time series and streamed to clients as events. "Splunk SPL for SQL Users" is a good starting point, although it can lead to bad habits if not supplemented by other documentation and formal training. If you're a database administrator with combined Microsoft SQL Server and Microsoft Windows experience, you'll find many parallels between how a Microsoft SQL Server failover cluster is managed and how a simple Splunk Enterprise indexing cluster is managed; however, the product architectures diverge very quickly. Experience with Microsoft Windows and a server-oriented Linux distribution, e.g. Red Hat Enterprise Linux, are important for supporting Splunk Enterprise on-premises or in a self-managed hybrid/cloud environment. If your target is Splunk Cloud, operating system experience is still required to effectively support Splunk Universal Forwarder on endpoints/servers and Splunk Enterprise configured as a heavy forwarder. Splunk administrators install, configure, and maintain all aspects of Splunk Enterprise or configure and maintain customer-accessible aspects of Splunk Cloud. A Splunk "developer" can be a content developer (searches, reports, and dashboards), an app developer (add-ons and technology integrations), a back-end developer (primarily web services), a front-end developer (Splunk SDK and third-party clients), or any combination of the four. Splunk certification paths are published at https://www.splunk.com/en_us/training/certification.html. The Splunk Success Framework describes the typical roles associated with Splunk deployments and provides corresponding learning paths: https://lantern.splunk.com/Splunk_Success_Framework/People_Management/Setting_roles_and_responsibilities?mt-learningpath=ssfpeople ... View more

As an additional exercise, we can compare diff with combinations of inputlookup. The following searches should return the same results: A | set diff [| inputlookup test.csv ] [| inputlookup test2.csv ] B | inputlookup test.csv where NOT [| inputlookup test2.csv ] | inputlookup append=t test2.csv where NOT [| inputlookup test.csv ] ... View more

Hi @munang, The set command and the join command perform overlapping but different functions. set diff returns the symmetric difference of the subsearches: I.e. set diff returns all events in either subsearch A or subsearch B but not both: A url A_field https://www.splunk.com/ A_value https://www.appdynamics.com/ A_value   B url A_field https://www.appdynamics.com/ A_value https://www.cisco.com/ A_value   diff url A_field https://www.splunk.com/ A_value https://www.cisco.com/ A_value   Both join type=left and join type=outer perform a left outer join by joining all fields in all events in the base search with all fields from the first (default: max=1) matching event in the subsearch: I.e.: A url A_field https://www.splunk.com/ A_value https://www.appdynamics.com/ A_value   B url B_field https://www.appdynamics.com/ B_value1 https://www.appdynamics.com/ B_value2   join url A_field B_field https://www.splunk.com/ A_value (null) https://www.appdynamics.com/ A_value B_value1   As written, your join search is equivalent to join type=inner. The where command removes all events from the base search that were not joined to an event in the subsearch. To return the difference using the join command, the command would need to support a full outer join, and it does not. ... View more

Hi @danielbb, As an alternative in Simple XML, you can stack visualizations vertically by including more than one visualization in a panel:   <row> <!-- first row --> <panel> </panel> </row> <row> <panel> <!-- second row, first column --> </panel> <panel> <chart> <!-- second row, second column, first row --> </chart> <chart> <!-- second row, second column, second row --> </chart> <chart> <!-- second row, second column, third row --> </chart> </panel </row>   Some visualizations may require an empty spacer, e.g. an <html/> section, to defeat the auto-layout code. Here's an extended example using an inverse normal macro (not shown) to generate a bit of random data:   <dashboard version="1.1" theme="light"> <label>danielbb_layout</label> <search id="base"> <query>| gentimes start=11/29/2024:00:00:00 end=11/30/2024:00:00:00 increment=1m | eval _time=starttime | fields + _time | sort 0 - _time | eval x=`norminv("`rand()`*(0.9999999999999999-0.0000000000000001)+0.0000000000000001", 1.3, 10)`</query> </search> <search id="stats" base="base"> <query>| stats avg(x) as u stdev(x) as s</query> </search> <row> <panel> <html> <h1>Some Random Data</h1> </html> </panel> </row> <row> <panel> <chart> <title>Histogram</title> <search base="base"> <query>| chart count over x span=0.5</query> </search> <option name="charting.legend.placement">none</option> <option name="height">600</option> </chart> </panel> <panel> <chart> <title>Samples</title> <search base="base"></search> <option name="charting.legend.placement">none</option> </chart> <single> <title>Mean</title> <search base="stats"> <query>| fields u</query> </search> <option name="numberPrecision">0.000</option> </single> <html/> <single> <title>Standard Deviation</title> <search base="stats"> <query>| fields s</query> </search> <option name="numberPrecision">0.000</option> </single> </panel> </row> </dashboard>   ... View more

Hi @robertlynch2020, I'm not aware of a native method, although you could override the Splunk bar's behavior with a browser extension, assuming you control the browser. My preference is to customize the launcher or provide a default app with customized navigation menus. ... View more

Hi @siva_kumar0147, The simplest solution is to use the Timeline visualization. You'll need to calculation durations in milliseconds between transitions: | makeresults format=csv data="_time,direction,polarization 1732782870,TX,L 1732782870,RX,R 1732781700,TX,R 1732781700,RX,L" | sort 0 - _time + direction | eval polarization=case(polarization=="L", "LHCP", polarization=="R", "RHCP") | streamstats global=f window=2 first(_time) as end_time by direction | addinfo | eval duration=if(end_time==_time, 1000*(info_max_time-_time), 1000*(end_time-_time)) | table _time direction polarization duration   ... View more