Hi @Emre , you should create some field extractions using regexes from the message field. If you can share a sample of your data in text format (not screenshot), we can help you. Ciao. Giuseppe ... View more

MAX_DAYS_AGO doesn't cut it. It will make Splunk still index the events but it will assume that the timestamp parsed from the event was wrong so it would just assume another timestamp (whatever that would effectively be). @Andre_ What you could to to prevent some data from being indexed could be to add whitelists/blacklists in inputs matching certain timestamp values. That's a relatively ugly solution and is not something to be kept forever but for a start - might be the way to go. Just be careful because you're doing it differently when you're windows logs as "classic" and differently while it's XML. ... View more

@sverdhan  Try below with clients, | tstats count WHERE index=* by index sourcetype | rex field=index max_match=0 "(?<clients>\w+)(?<sensitivity>_private|_public)" | lookup appserverdomainmapping.csv clients OUTPUT NewIndex, Domain, Sourcetype | eval NewIndex=NewIndex.sensitivity | table clients, sensitivity, Domain, Sourcetype, NewIndex If you do not need to add clients, and to just display lookup fields you can use appendcols | tstats count WHERE index=* by index sourcetype | rex field=index max_match=0 "(?<clients>\w+)(?<sensitivity>_private|_public)" | appendcols [| inputlookup appserverdomainmapping.csv | fields Domain, Sourcetype, NewIndex] | eval NewIndex=NewIndex.sensitivity | table clients, sensitivity, Domain, Sourcetype, NewIndex Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks! ... View more

You are likely correct in that the UF does not read the log file twice, as it reads it initially the first time for a batch ingestion and then never again when monitoring. The log file does not update its modification time or size as it's never closed by the application. I believe the CRC would change when the oldest events are overwritten, as that occurs at the top of the file. But as pointed out by you and others, this is not desirable behaviour.  So, assuming none of the checks for change in the log file works. Do you have any ideas on how I can make the UF open and read the file, or what mechanisms that prevents it? As stated in the initial post I have already tried a few things, are there any more tricks?   ... View more

When you send valid json object you can query any data what it contains. You could utilize those keys or use just any words which it has. ... View more

Hi @avikc100 , please, next time, send the search in text mode (using the Insert/Edit Code Sample button) so you can mask the sensitive data and we can use it. At first, don't use search or where after the main search, but put all the conditions as left as possible, possibly in the main search: index="webmethods_prd" source="/apps/WebMethods*/IntegrationServer/instances/default/logs/MISC.log" MISC_dynamicPrice mainframePrice!=discountPrice | stats count BY mainframePrice discountPrice accountNumber itemId  otherwise you could add the dc function to identify the different values: index="webmethods_prd" source="/apps/WebMethods*/IntegrationServer/instances/default/logs/MISC.log" MISC_dynamicPrice | stats dc(mainframePrice) AS mainframePrice_count dc(discountPrice) AS discountPrice_count first(mainframePrice) AS first_mainframePrice first(discountPrice) AS first_discountPrice last(mainframePrice) AS last_mainframePrice last(discountPrice) AS last_discountPrice BY accountNumber itemId | where mainframePrice_count>1 OR discountPrice_count>1 | fields - *_count Ciao. Giuseppe   ... View more

I would suggest a slightly optimal version that does not use the subsearch index=abc | stats count by source | inputlookup append=t my_source_lookup.csv | fillnull count | stats sum(count) AS total BY source   ... View more

Hi @sawwinnaung , good for you, see next time! Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated by all the contributors 😉 ... View more

@Trevorator  I dont think there is any Splunk setting to enable automatic retry or continuation for .tsidx file operations. The only way to ensure all data is accelerated and .tsidx files are preserved is to maintain a healthy infrastructure and address any resource limitations. Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos/Karma. Thanks! ... View more

Try changing the timezone in your time format TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N%z ("Z" as you have used it is just a character constant - which is used n some date formats) Time variables  ... View more

As I always say, do not treat structured data like text.  split is not the tool for this job.  @gcusello suggests INDEXED_EXTRACTION and a way to set up extraction in props.conf.  Short of these, you can also use multikv | multikv   ... View more

I see, but with this I am not able to see the data as I want, Iike I want to see the hostname on map and when point out on host it shows average CPU and memory utlization ... View more

in splunkd.log , of HF2 I could see these  06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_host_thruput, ingest_pipe=2, series="lmpsplablr001", kbps=4.247, eps=24.613, kb=131.668, ev=763, avg_age=2.279, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_index_thruput, ingest_pipe=2, series="_internal", kbps=2.206, eps=13.032, kb=68.396, ev=404, avg_age=2.233, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_index_thruput, ingest_pipe=2, series="_metrics", kbps=2.041, eps=11.581, kb=63.272, ev=359, avg_age=2.331, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_source_thruput, ingest_pipe=2, series="/mnt/splunk/splunk/var/log/splunk/audit.log", kbps=0.000, eps=0.032, kb=0.000, ev=1, avg_age=0.000, max_age=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_source_thruput, ingest_pipe=2, series="/mnt/splunk/splunk/var/log/splunk/metrics.log", kbps=4.082, eps=23.355, kb=126.545, ev=724, avg_age=2.312, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_source_thruput, ingest_pipe=2, series="/mnt/splunk/splunk/var/log/splunk/splunkd_access.log", kbps=0.165, eps=1.226, kb=5.123, ev=38, avg_age=1.711, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_sourcetype_thruput, ingest_pipe=2, series="splunk_audit", kbps=0.000, eps=0.032, kb=0.000, ev=1, avg_age=0.000, max_age=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_sourcetype_thruput, ingest_pipe=2, series="splunk_metrics_log", kbps=2.041, eps=11.677, kb=63.272, ev=362, avg_age=2.312, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_sourcetype_thruput, ingest_pipe=2, series="splunkd", kbps=2.041, eps=11.677, kb=63.272, ev=362, avg_age=2.312, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=per_sourcetype_thruput, ingest_pipe=2, series="splunkd_access", kbps=0.165, eps=1.226, kb=5.123, ev=38, avg_age=1.711, max_age=3 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=tcpout_my_syslog_group, max_size=512000, current_size=0, largest_size=0, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=tcpout_primary_indexers, max_size=512000, current_size=0, largest_size=219966, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=aggqueue, max_size_kb=2560000, current_size_kb=0, current_size=0, largest_size=260, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=indexqueue, max_size_kb=2560000, current_size_kb=0, current_size=0, largest_size=8, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=nullqueue, max_size_kb=2560000, current_size_kb=0, current_size=0, largest_size=1, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=parsingqueue, max_size_kb=2560000, current_size_kb=0, current_size=0, largest_size=2, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=syslog_system, max_size_kb=97, current_size_kb=0, current_size=0, largest_size=1, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=syslog_system2, max_size_kb=97, current_size_kb=0, current_size=0, largest_size=1, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=queue, ingest_pipe=2, name=typingqueue, max_size_kb=2560000, current_size_kb=0, current_size=0, largest_size=257, smallest_size=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=syslog_connections, ingest_pipe=2, syslog_system2:x.x.x.x:514:x.x.x.x:514, sourcePort=8089, destIp=x.x.x.x, destPort=514, _tcp_Bps=0.00, _tcp_KBps=0.00, _tcp_avg_thruput=3.71, _tcp_Kprocessed=2266, _tcp_eps=0.00 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=syslog_connections, ingest_pipe=2, syslog_system:y.y.y.y:514:y.y.y.y:514, sourcePort=8089, destIp=y.y.y.y, destPort=514, _tcp_Bps=0.00, _tcp_KBps=0.00, _tcp_avg_thruput=0.35, _tcp_Kprocessed=213, _tcp_eps=0.00 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=tcpout_connections, ingest_pipe=2, name=primary_indexers:z.z.z.z0:9997:0:0, sourcePort=8089, destIp=z.z.z.z0, destPort=9997, _tcp_Bps=1545.33, _tcp_KBps=1.51, _tcp_avg_thruput=1.51, _tcp_Kprocessed=45, _tcp_eps=0.50, kb=44.94 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=thruput, ingest_pipe=2, name=index_thruput, instantaneous_kbps=0.000, instantaneous_eps=0.000, average_kbps=0.000, total_k_processed=0.000, kb=0.000, ev=0 06-13-2025 12:34:19.086 +0800 INFO Metrics - group=thruput, ingest_pipe=2, name=thruput, instantaneous_kbps=4.505, instantaneous_eps=26.000, average_kbps=10.456, total_k_processed=6705.000, kb=139.648, ev=806, load_average=1.500   any abnormalities in this entries ? I suspect that issue is with HF 2 only .. when HF 2 stopped , everyother things works fine ..if HF2 service initiated, it start utilize 1GB to 50GB of memory only  out of 130GB , then the HF1 start use memory and log ingestion getting stopped .. especially from syslog server ( large log volume input) -> this index getting affected first  Hence increasing memory in HF 2 not helpful here ... View more

HI @TestUser  I think as @gcusello has stated here, there isnt such a tool or capability within Splunk currently that would allow this, but its possibly something that with enough information could be built into an app.  It would rely on a couple of key bits of information though, such as what the usecases for the dashboard are (e.g. what is it you want to visualise, and for whom etc) and also if the data is in a predictable (or ideally CIM compliant) format. e.g. can you reference fields reliably knowing their content (type) and names etc. It might help if you could share a little more about what you are trying to achieve. 🌟 Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing ... View more

Hi @Shakeer_Spl , as I said, there isn't a version of Splunk Full Stack, there are two versions of Splunk on premise: Splunk Enterprise Splunk Universal Forwarder. The full Stack is only Splunk Enterprise. For both the products there are many version (the last released is 9.4.3) and you can find it at https://www.splunk.com/en_us/download/splunk-enterprise.html . let us know if we can help you more, or, please, accept one answer for the other people of Community. Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated by all the contributors 😉   ... View more

Are you sure that e.g. some of your source nodes are not HF instead of UF? This is valid also form collecting node not only any nodes between UF and Indexers! Basically your configuration seems to be ok. Of course you could modify those REGEX little bit efficient than those are currently, but it's another story. As other already said the most obvious reason is that you have HF somewhere before your indexer (splunk enterprise node). You can check this on source node and all other nodes (check those from outputs.conf) $SPLUNK_HOME/bin/splunk version  This should show in UF Splunk Universal Forwarder 9.4.0 (build 6b4ebe426ca6) or in HF/indexer  Splunk 9.4.1 (build e3bdab203ac8) Just check which path it is running and replace SPLUNK_HOME with it.  ... View more

| rest splunk_server_group=dmc_group_license_master /services/licenser/groups | search is_active=1 | eval stack_id=stack_ids | mvexpand stack_id | join type=outer stack_id splunk_server [ rest splunk_server_group=dmc_group_license_master /services/licenser/pools] | fields splunk_server, stack_id, effective_quota, used_bytes | stats sum(used_bytes) as used_bytes max(effective_quota) as stack_quota by stack_id | eval usedGB=round(used_bytes/1024/1024/1024, 3),totalGB=round(stack_quota/1024/1024/1024, 3), percentage=round((used_bytes / stack_quota) * 100, 2) | eval alert_level=case(percentage >= 90, "Critical: >90%",percentage >= 80, "Warning: >80%",percentage >=70, "Info: >70%",true(), null()) | where isnotnull(alert_level) | rename stack_id AS Instance, percentage AS "License quota used (%)",usedGB AS "License quota used (GB)",totalGB AS "Total license quota (GB)",alert_level AS "Alert Level" ... View more

Hi @mcfly227 , good for you, see next time! Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated 😉 ... View more

First thing to debug inputs is usually, after verifying the config checking the output of splunk list monitor and splunk list inputstatus   ... View more

Hi @ayomotukoya  Can I check which method you are using to ingest the SNMP data? Is it using Splunk Connect for SNMP? (https://splunk.github.io/splunk-connect-for-snmp/main/) - If so this sends the data to Splunk with HEC so would require a different approach to this problem. Or is this monitoring local files which are saved to disk from SNMP? If so, is this on a Universal Forwarder (UF) or heavy forwarder (HF)? 🌟 Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing ... View more

Hi @kn450 , open a case to Splunk Support! there isn't any other solution! I had an experience in UBA installation and only when installed by Splunk PS it runned! Ciao. Giuseppe ... View more

Hi @Praz_123 , could you share a sample of your logs in text format? Ciao. Giuseppe ... View more

This is actually similar to another question I responded to recently at https://community.splunk.com/t5/Dashboards-Visualizations/Dashboard-Studio-time-range-input/m-p/745721#M58657 This is the snippet which calculated the time string from the time picker: | makeresults | eval earliest=$global_time.earliest|s$, latest=$global_time.latest|s$ | eval earliest_epoch = IF(match(earliest,"[0-9]T[0-9]"),strptime(earliest, "%Y-%m-%dT%H:%M:%S.%3N%Z"),earliest), latest_epoch = IF(match(latest,"[0-9]T[0-9]"),strptime(latest, "%Y-%m-%dT%H:%M:%S.%3N%Z"),latest)   @livehybrid wrote: Hi @abhishekP  This is an interesting one. When selecting a relative time window the earliest/latest are values like "-1d@d" which are valid for the earliest/latest field in a search - however when you select specific dates/between dates etc then it returns the full date string such as "2025-05-07T18:47:22.565Z" Such a value is not supported by the earliest/latest field in a Splunk search, to get around this I have put together a table off the side of the display with a search which converts dates into epoch where required. you can then use "$timetoken:result.earliest_epoch$" and "$timetoken:result.latest_epoch$" as tokens in your other searches like this:   Below is the full JSON of the dashboard so you can have a play around with it - hopefully this helps! { "title": "testing", "description": "", "inputs": { "input_global_trp": { "options": { "defaultValue": "-24h@h,now", "token": "global_time" }, "title": "Global Time Range", "type": "input.timerange" } }, "defaults": { "dataSources": { "ds.search": { "options": { "queryParameters": { "earliest": "$global_time.earliest$", "latest": "$global_time.latest$" } } } } }, "visualizations": { "viz_2FDRkepv": { "dataSources": { "primary": "ds_IPGx8Y5Y" }, "options": {}, "type": "splunk.events" }, "viz_V1oldcrB": { "options": { "markdown": "earliest: $global_time.earliest$ \nlatest: $global_time.latest$ \nearliest_epoch: $timetoken:result.earliest_epoch$ \nlatest_epoch:$timetoken:result.latest_epoch$" }, "type": "splunk.markdown" }, "viz_bhZcZ5Cz": { "containerOptions": {}, "context": {}, "dataSources": { "primary": "ds_KXR2SF6V" }, "options": {}, "showLastUpdated": false, "showProgressBar": false, "type": "splunk.table" } }, "dataSources": { "ds_IPGx8Y5Y": { "name": "timetoken", "options": { "enableSmartSources": true, "query": "| makeresults \n| eval earliest=$global_time.earliest|s$, latest=$global_time.latest|s$\n| eval earliest_epoch = IF(match(earliest,\"[0-9]T[0-9]\"),strptime(earliest, \"%Y-%m-%dT%H:%M:%S.%3N%Z\"),earliest), latest_epoch = IF(match(latest,\"[0-9]T[0-9]\"),strptime(latest, \"%Y-%m-%dT%H:%M:%S.%3N%Z\"),latest)" }, "type": "ds.search" }, "ds_KXR2SF6V": { "name": "Search_1", "options": { "query": "index=_internal earliest=$timetoken:result.earliest_epoch$ latest=$timetoken:result.latest_epoch$\n| stats count by host" }, "type": "ds.search" } }, "layout": { "globalInputs": [ "input_global_trp" ], "layoutDefinitions": { "layout_1": { "options": { "display": "auto", "height": 960, "width": 1440 }, "structure": [ { "item": "viz_V1oldcrB", "position": { "h": 80, "w": 310, "x": 20, "y": 20 }, "type": "block" }, { "item": "viz_2FDRkepv", "position": { "h": 260, "w": 460, "x": 1500, "y": 20 }, "type": "block" }, { "item": "viz_bhZcZ5Cz", "position": { "h": 380, "w": 1420, "x": 10, "y": 140 }, "type": "block" } ], "type": "absolute" } }, "tabs": { "items": [ { "label": "New tab", "layoutId": "layout_1" } ] } } } 🌟 Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing   ... View more

Hi @spm807 , as I said, try using throttling in your alerts, it's the solution to your problem. Ciao. Giuseppe ... View more

The deployment client disabled similar issues was facing during the lab setup with 1 sh, 2 idx, 1 mc and 1 uf..  After all the trail which did not work, I placed the deploymentclinet.conf file in two location below which worked.  1-->/opt/splunk/etc/apps/all_deployment client 2-->/opt/splunk/etc/syste/local stanza below [deployment-client] disabled =  false clientName = example(mc,idx,uf/hf) # Change the targetUri to point to deployment server targetUri = deploymentserver.splunk.mycompany.com:8089           ... View more