Hi @antoniolamonica, Data model root search datasets start with a base search. Endpoint.Processes is: (`cim_Endpoint_indexes`) tag=process tag=report | eval process_integrity_level=lower(process_integrity_level) This search is expanded by Splunk to include the contents of the cim_EndPoint_indexes macros and all event types that match tag=process and tag=report. To compare like for like searches, start with the same base search: (`cim_Endpoint_indexes`) tag=process tag=report earliest=-4h@h latest=-2h@h | eval process_integrity_level=lower(process_integrity_level) | stats count values(process_id) as process_id by dest and construct a similar tstats search: | tstats summariesonly=f count values(Processes.process_id) as process_id from datamodel=Endpoint.Processes where earliest=-4h@h latest=-2h@h by Processes.dest The underlying searches should be similar. Optimization may vary. You can verify the SPL in the job inspector: Job > Inspect Job > search.log and the UnifiedSearch component's log output. When summariesonly=f, the searches have similar behavior. When summariesonly=t, the data model search only looks at indexed field values. This is similar to using field::value for indexed fields and TERM() for indexed terms in a normal search. ... View more

Hi @madhav_dholakia, Can you provide more context for the defaults object and a small sample dashboard that doesn't save correctly? Is defaults defined at the top level of the dashboard? I.e.: { "visualizations": { ... }, "dataSources": { ... }, "defaults": { ... }, ... }   ... View more

Hi @splunk_user_99, Which version of MLTK do you have installed? The underlying API uses a simple payload: {"name":"my_model","app":"search"} where name is the value entered in New Main Model Title and app is derived from the value selected in Destination App. The app list is loaded when Operationalize is clicked and sorted alphabetically by display name. On submit, the request payload is checked to verify that it contains only the 'app' and 'name' keys. Do you have the same issue in a sandboxed (private, incognito, etc.) browser session with extensions disabled? ... View more

Here's a straightforward hack that uses a zero width space as a padded value prefix to determine a cell's status. For example, a status of Unknown is one zero width space. The SPL uses the urldecode() eval function to convert URL-encoded UTF-8 characters to strings. <table id="table2"> <search> <query>| makeresults format=csv data=" _time,HOSTNAME,PROJECTNAME,JOBNAME,INVOCATIONID,RUNSTARTTIMESTAMP,RUNENDTIMESTAMP,RUNMAJORSTATUS,RUNMINORSTATUS,RUNTYPENAME 2025-01-20 04:38:04.142,AEW1052ETLLD2,AQUAVISTA_UAT,Jx_104_SALES_ORDER_HEADER_FILE,HES,2025-01-19 20:18:25.0,,STA,RUN,Run 2025-01-19 04:38:04.142,AEW1052ETLLD2,AQUAVISTA_UAT,Jx_104_SALES_ORDER_HEADER_FILE,HES,2025-01-18 20:18:25.0,2025-01-18 20:18:29.0,FIN,FWF,Run 2025-01-18 04:38:04.142,AEW1052ETLLD2,AQUAVISTA_UAT,Jx_104_SALES_ORDER_HEADER_FILE,HES,2025-01-17 20:18:25.0,2025-01-17 20:18:29.0,FIN,FOK,Run 2025-01-17 04:38:04.142,AEW1052ETLLD2,AQUAVISTA_UAT,Jx_104_SALES_ORDER_HEADER_FILE,HES,2025-01-16 20:18:25.0,2025-01-16 20:18:29.0,FIN,FWW,Run 2025-01-16 04:38:04.142,AEW1052ETLLD2,AQUAVISTA_UAT,Jx_104_SALES_ORDER_HEADER_FILE,HES,2025-01-15 20:18:25.0,2025-01-15 20:18:29.0,FIN,HUH,Run " ``` use zero width space as pad ``` | eval status_unknown=urldecode("%E2%80%8B") | eval status_success=urldecode("%E2%80%8B%E2%80%8B") | eval status_failure=urldecode("%E2%80%8B%E2%80%8B%E2%80%8B") | eval status_warning=urldecode("%E2%80%8B%E2%80%8B%E2%80%8B%E2%80%8B") | eval status_running=urldecode("%E2%80%8B%E2%80%8B%E2%80%8B%E2%80%8B%E2%80%8B") | eval _time=strptime(_time, "%Y-%m-%d %H:%M:%S.%Q") | search PROJECTNAME="*" INVOCATIONID="*" RUNMAJORSTATUS="*" RUNMINORSTATUS="*" | eval status=case(RUNMAJORSTATUS="FIN" AND RUNMINORSTATUS="FWW", status_warning, RUNMAJORSTATUS="FIN" AND RUNMINORSTATUS="FOK", status_success, RUNMAJORSTATUS="FIN" AND RUNMINORSTATUS="FWF", status_failure, RUNMAJORSTATUS="STA" AND RUNMINORSTATUS="RUN", status_running, 1=1, status_unknown) | eval tmp=JOBNAME."|".INVOCATIONID | eval date=strftime(strptime(RUNSTARTTIMESTAMP, "%Y-%m-%d %H:%M:%S.%Q"), "%Y-%m-%d") | eval value=status.if(status==status_unknown, "Unknown", "start time: ".coalesce(strftime(strptime(RUNSTARTTIMESTAMP, "%Y-%m-%d %H:%M:%S.%Q"), "%H:%M"), "").urldecode("%0a").if(status==status_running, "Running", "end time: ".coalesce(strftime(strptime(RUNENDTIMESTAMP, "%Y-%m-%d %H:%M:%S.%Q"), "%H:%M"), ""))) | xyseries tmp date value | eval tmp=split(tmp, "|"), Job=mvindex(tmp, 0), Country=mvindex(tmp, 1) | fields - tmp | table Job Country *</query> </search> <option name="drilldown">none</option> <option name="wrap">true</option> <format type="color"> <colorPalette type="expression">case(match(value, "^\\u200b{1}[^\\u200b]"), "#D3D3D3", match(value, "^\\u200b{2}[^\\u200b]"), "#90EE90", match(value, "^\\u200b{3}[^\\u200b]"), "#F0807F", match(value, "^\\u200b{4}[^\\u200b]"), "#FEEB3C", match(value, "^\\u200b{5}[^\\u200b]"), "#ADD9E6")</colorPalette> </format> </table>   ... View more

Now you're Splunking! Watch out for approximation errors! 2 * 3 * 4 = 24 exp(ln(2) + ln(3) + ln(4)) ~= 23.999999999999993 ... View more

Hi @welcomerrr, You can often get creative with alternative SPL without restoring to custom commands. See https://community.splunk.com/t5/Splunk-Search/Product-of-a-Column/m-p/707749 for a recent example of calculating a product over a list. Depending on the algorithm your hypothetical function implements, a solution may also be possible using a combination of an existing aggregation function and a nested eval. ... View more

Hi @silverKi, The maxDataSize for your hot buckets is 1 MB. Your friend's setting appears to be higher (5 MB). To add to what's already been written, you're writing (compressed) data at different rates: Friend: ~720 bytes per second You: ~19 bytes per second This will influence the size of the warm bucket after it rolls from hot when either maxDataSize (1 MB in your case) or the default maxHotSpanSecs value of 90 days has been exceeded. Hot buckets can also roll to warm when Splunk is restarted or when triggered manually. That probably isn't happening here, but it's worth noting. ... View more

Looks good! Just be mindful of the difference between the plus (+) operator and the dot (.) operator. Plus concatenates strings and adds numbers, but dot concatenates both strings and numbers as strings. If you're unsure of the order of operations or the variable value in other contexts, you can wrap the variables in the tostring() function. It's interesting that Splunk also assumed a leading space would work in the SC4S transforms: [metadata_time] SOURCE_KEY = _time REGEX = (.*) FORMAT = t="$1$0 DEST_KEY = _raw @PickleRick's suggestion to escape the space with i.e. a blackslash just evaluates to a literal backslash followed by a space. ... View more

Hi @BrianLam, You can retrieve the search results using the search/v2/jobs/{search_id}/results endpoint. See https://docs.splunk.com/Documentation/Splunk/latest/RESTREF/RESTsearch#search.2Fv2.2Fjobs.2F.7Bsearch_id.7D.2Fresults. The search_id value is specific to the instance of the search that generated the alert. It's a simple GET request. The default output mode is XML. If you want JSON output, pass the output_mode query parameter as part of the GET request: https://splunk:8089/services/search/v2/jobs/scheduler__user__app__xxx_at_xxx_xxx/results?output_mode=json   ... View more

Hi @Jado95, Is your question specific to Splunk Add-on for Cisco ASA or Cisco ASA itself? The message format is defined by Cisco ASA, and the add-on implementation should agree with Cisco ASA documentation at https://www.cisco.com/c/en/us/td/docs/security/asa/syslog/b_syslog/syslog-messages-302003-to-342008.html: 302013 ... If inbound is specified, the original control connection was initiated from the outside. For example, for FTP, all data transfer channels are inbound if the original control channel is inbound. If outbound is specified, the original control connection was initiated from the inside. ... 302015 ... If inbound is specified, then the original control connection is initiated from the outside. For example, for UDP, all data transfer channels are inbound if the original control channel is inbound. If outbound is specified, then the original control connection is initiated from the inside. The corresponding teardown events, 302014 and 302106, do not specify a direction, so without prior knowledge, the field extraction can't know which address is the initiator. If needed, you can correlate the events by the session_id field. This example is slow and ugly; it's only meant to demonstrate the correlation: | eventstats values(direction) as direction by session_id | eval src_ip_tmp=src_ip, dest_ip_tmp=dest_ip, src_ip=if(lower(vendor_action)=="teardown" && lower(direction)=="outbound", dest_ip_tmp, src_ip_tmp), dest_ip=if(lower(vendor_action)=="teardown" && lower(direction)=="outbound", src_ip_tmp, dest_ip_tmp) | fields - src_ip_tmp dest_ip_tmp   ... View more

I would use list() instead of values() to prevent removal of duplicates and wrap the product in exact() to prevent rounding errors: | makeresults format=csv data="value_a 0.44 0.25 0.67 0.44" | stats list(value_a) as value_a | eval "product(value_a)"=1 | foreach value_a mode=multivalue [ eval "product(value_a)"=exact('product(value_a)' * <<ITEM>>) ] | table "product(value_a)"  => product(value_a) 0.032428 ... View more

Hi @madhav_dholakia, In Simple XML, you can generate categorical choropleth maps with color-coded city boundaries: In Dashboard Studio, however, choropleth maps are limited to numerical distributions, e.g. OpenIssues by city, and the geospatial lookup geometry isn't always interpreted correctly: In both examples, I've used mapping data published by the Office for National Statistics at https://geoportal.statistics.gov.uk. Search for BDY_TCITY DEC_2015 to download the corresponding KML file. As a compromise, you can use a marker map to display color-coded markers at city centers by latitude and longitude: Here's the source: { "visualizations": { "viz_KxsdmDQb": { "type": "splunk.map", "options": { "center": [ 52.560559999999924, -1.4702799999984109 ], "zoom": 6, "layers": [ { "type": "marker", "dataColors": "> primary | seriesByName('Status') | matchValue(colorMatchConfig)", "choroplethOpacity": 0.75, "additionalTooltipFields": [ "Status", "StoreID", "City", "OpenIssues" ], "latitude": "> primary | seriesByName('lat')", "longitude": "> primary | seriesByName('lon')" } ] }, "context": { "colorMatchConfig": [ { "match": "Dormant/Green", "value": "#118832" }, { "match": "Warning/Amber", "value": "#cba700" }, { "match": "Critical/Red", "value": "#d41f1f" } ] }, "dataSources": { "primary": "ds_CtvaIPJ3" } } }, "dataSources": { "ds_CtvaIPJ3": { "type": "ds.search", "options": { "query": "| makeresults format=csv data=\"StoreID,City,OpenIssues,Status,lat,lon\r\nStore 1,London,3,Critical/Red,51.507222,-0.1275\r\nStore 2,York,2,Warning/Amber,53.96,-1.08\r\nStore 3,Bristol,0,Dormant/Green,51.453611,-2.5975\r\nStore 4,Liverpool,1,Warning/Amber,53.407222,-2.991667\" \r\n| table StoreID City OpenIssues Status lat lon", "queryParameters": { "earliest": "-24h@h", "latest": "now" } }, "name": "Choropleth map search" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": {} } } } }, "inputs": {}, "layout": { "type": "absolute", "options": { "width": 918, "height": 500, "display": "auto" }, "structure": [ { "item": "viz_KxsdmDQb", "type": "block", "position": { "x": 0, "y": 0, "w": 918, "h": 500 } } ], "globalInputs": [] }, "description": "", "title": "eaw_store_status_ds" } As a static workaround, the Choropleth SVG visualization allows you to upload a custom image, e.g. a stylized map of England and Wales, and define custom SVG boundaries and categorical colors. The Dashboard Studio documentation includes a basic tutorial at https://docs.splunk.com/Documentation/Splunk/latest/DashStudio/mapsChorSVG. ... View more

Hi @CyberWolf, There's a tendency among practitioners to bin time into buckets rounded to the nearest time interval, e.g. 1 hour: 00:00, 01:00, 02:00, etc.; however, this results in counting errors. Instead, count using a rolling window in ascending _time order:   index=VPN_Something status=failure | stats count by _time user | streamstats time_window=1h sum(count) as failure_count by user | where failure_count>5   Since you're only interested in a 1-hour window, your search time range only needs span the last hour plus any allowance for ingest lag and a buffer to accommodate your scheduling interval. See https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Streamstats for information on scaling streamstats. If you're using accelerated data models or indexed fields or if your raw events are structured in key-value pairs separated by minor breakers, you can use tstats to greatly improve the performance of the search. If you have the capacity, you might also consider a real-time search that counts events as they're indexed, although the results may be incorrect relative to your requirements. If you have Splunk Enterprise Security, look at the "Access - Excessive Failed Logins - Rule" correlation search. For reference, it's a real-time search scheduled every 5 minutes (*/5 * * * *), with earliest=rt-65m@m and latest=rt-5m@m:   | from datamodel:"Authentication"."Failed_Authentication" | stats values("tag") as "tag",dc("user") as "user_count",dc("dest") as "dest_count",count by "app","src" | where 'count'>=6   In the Authentication data model, app would be something like vpn, and src would be a device identifier. As @isoutamo wrote, there are many approximate solutions to this problem. The correct solution depends on your requirements and your tolerance for counting errors. ... View more

--if your base search includes N1, N2, N3, ... Nn, you can add additional logic to generate the N-value dynamically:   | appendpipe [| search Category=N* | stats count sum(*) as * | eval Category="N".tostring(count+1) | fields - count ]   Or if you want to sub all Category values:   | appendpipe [| rex field=Category "(?<Category>[^\d]+)" | stats count sum(*) as * by Category | eval Category=Category.tostring(count+1) | fields - count ]   => Category A B C D E F N1 1 2 4 2 4 1 N2 0 5 4 3 5 7 M1 1 0 1 0 4 3 M2 1 1 3 5 0 1 U1 0 4 6 5 4 3 M3 2 1 4 5 4 4 N3 1 7 8 5 9 8 U2 0 4 6 5 4 3 But note that you'll need to add custom sorting logic if you want something other than the default sort order. ... View more

You can put whatever you want in the label argument. It sets the value of the field specified by labelfield in the totals row: | addcoltotals labelfield=Category label="Total" | addcoltotals labelfield=Category label="SUM[n1..nN]" | addcoltotals labelfield=Category label="Life, the Universe and Everything" ... View more

Hi @MachaMilkshake, The addcoltotals command should do exactly what you need: | addcoltotals labelfield=Category label=N3 ... View more

Hi @ranjith4, What is the aggregate throughput of all sources? If you're unsure, what is the peak daily ingest of all sources? Splunk Universal Forwarder uses very conservative default queue sizes and a throughput limit of 256 KBps. As a starting point, you can disable the throughput limit in $SPLUNK_HOME/etc/system/local/limits.conf: [thruput] maxKBps = 0 If the forwarder is still not delivering data as quickly as it arrives, we can adjust output queue sizes based on your throughput (see Little's Law). As @PickleRick noted, the forwarder may be switching to an effectively single-threaded batch mode when reading files larger than 20 MB. Increase the min_batch_size_bytes setting in limits.conf to a value larger than your largest daily file or some other arbitrarily large value [default] # 1 GB min_batch_size_bytes = 1073741824 If throughput is still an issue, you can enable additional parallel processing with the server.conf parallelIngestionPipelines setting, but I wouldn't do that until after tuning other settings. ... View more