About StefanW

StefanW · ‎04-12-2021

Hello, since daylight savings time is active we have a time offset for our events. For example, we use das splunk stream addon to ingest netflow data. Within the Events, the timestamp is configured "2021-04-13T05:32:31Z". For my understanding with Z for zulu (UTC) But if i search for events my _time is 07:32:31. two hours later.. Our timezone is Europe/Berlin. How can i get this fixed? In the sourcetype of stream_netflow is the timestamp configured to auto. The OS time from the indexer/search head or universal forwarder are correct to CEST and the time is also correct. We have several other sourcestypes where the time offset is around 1 or 2 hours.

StefanW · ‎04-08-2021

Hello, we use mstats to visualize the _value. But for cpu perfmon values there is a number with 10 or more decimals after the pound like 45.736484739 For normal events we can use the eval command with round but for metrics this should be not working. How can we round the metrics values in the charts? | mstats avg(_value) prestats=t where metric_name="Processor.%_User_Time" AND index=* span=1m AND host=WI219SQL-P1 by metric_name | timechart avg(_value) as "AVG" span=60m

StefanW · ‎04-05-2021

ok, i solved it with wildcards [monitor:///var/syslog/logavaya/*/*-*.kuechen.de.log]

eddieddieddie · ‎03-28-2021

Hi Stefan, I've faced the same problem trying to deploy the Splunk App for Infrastructure to Windows via a deployment server - Splunk provide no "all Splunk" way of doing it. I think they expect you will be using another tool to deploy your forwarders and config (something like Ansible or SCCM I suppose). However I worked out another way to deploy this config which does use the deployment server and then set the metadata on a per-machine basis via a scripted input with the "interval" configured to -1 (run only once). Create a new app on your deployment server - for example "Windows-Base-Monitoring". Under this create two folders: 'local' and 'bin'. In the bin folder create a PowerShell script to open the inputs.conf file, look for stanza's titled "[perfmon://...]" and then update the _meta field with info specific to that host. The following script does this and inserts a _meta line containing the hosts OS version, IP address, hostname. ## InjectMetaConfigs.ps1 # Collect the hosts meta data essentials: # extract os & ip info $os_info = Get-CimInstance Win32_OperatingSystem | Select-Object Caption, Version $ip_info = Test-Connection -ComputerName $env:computername -count 1 | Select-Object IPV4Address # Add OS & ip info as dimension # If these dimensions are to be updated, user has to rerun this script $metadata = "_meta = os::" + "`"" + $os_info.Caption + "`"" $metadata = $metadata + " os_version::" + $os_info.Version $metadata = $metadata + " ip::" + "`"" + $ip_info.IPV4Address.IPAddressToString + "`"" $metadata = $metadata + " entity_type::Windows_Host" # Path to the Perfmon collection TA app: # ($SplunkHome is defined by Splunk before this script is run.) $perfmonApp = $SplunkHome + "/etc/apps/Windows-Base-Monitoring" # The path to the input.conf file to be edited. $inputsConfLoc = $perfmonApp + "/local/inputs.conf" $regex = '(?<=\[perfmon\:\/\/\w+\])' $inputfile = Get-Content ($inputsConfLoc) -Raw $outputfile = [regex]::replace($inputfile, '(?<=\[perfmon\:\/\/\w+\])(\r\n|\n)(?!_meta)', ($1 + "`n" + $metadata +"`n")) Set-Content -path $inputsConfLoc -Value $outputfile Now you can go ahead and create an inputs.conf file in the local folder with all the Perfmon configs set for everything you want to collect - however without the "_meta" lines as these will be set by the PowerScript after it's been deployed to the universal forwarders. In this file also define a PowerShell scripted input which runs the above script once only (interval = -1). For example: # inputs.conf # The following powershell script is run just once per restart (internal = -1) and if it finds the Perfmon stanza's below # doesn't contain '_meta' statements immidately after the [perfmon..] line adds it, completing the system info for the # it's running from. [powershell://InjectMetaConfig] script = . "$SplunkHome\etc\apps\Windows-Base-Monitoring\bin\InjectMetaConfigs.ps1" interval = -1 disabled = false ### Perfom monitoring ## CPU [perfmon://CPU] counters = % C1 Time;% C2 Time;% Idle Time;% Processor Time;% User Time;% Privileged Time;% Reserved Time;% Interrupt Time; disabled = false instances = _Total mode = single interval = 60 object = Processor useEnglishOnly = true sourcetype = PerfmonMetrics:CPU index = metrics ## Memory [perfmon://Memory] counters = Cache Bytes;% Committed Bytes In Use;Page Reads/sec;Pages Input/sec;Pages Output/sec;Committed Bytes;Available Bytes; disabled = false interval = 60 object = Memory useEnglishOnly=true sourcetype = PerfmonMetrics:Memory mode = single index = metrics When you deploy this app to your universal forwarders the deployment server handles sending the configuration and PowerShell script out. The UF then restarts itself and reads the new inputs.conf config, as it does this it runs the PowerShell script which updates the inputs.conf file with the _meta lines. You might need to restart the UF's after this has happened, however in my experience I discovered the inputs.conf is updated before the Perfmon monitoring starts. Hopefully this provides some inspiration to fixing your deployment! Eddie

StefanW · ‎02-07-2021

Hi, i am trying out the unix and linux addon on my linux machines. I enabled the iostat_metric monitor an see high io values. Often about 100 till 2000 iops. But thats wrong. Its test machine and also vcenter monitoring shows only 1-5 iops and after enabling collectd monitoring, collectd also shows low iops. If i run the iostat_metric.sh script manually, it shows also some time those high iops. Running iostat manually, the iops are low and right. Is there a known bug in parsing the output from iostat? Does someknow see same behavior.

StefanW · ‎02-02-2021

Hello, i read hundreds of articels, but its not working well. i try to gather data through telegraf from my snmp devices or other linux devices. First i install on a linux device telegraf 1.17. Create a simple input file [[inputs.diskio]] and a file output [[outputs.file]] ## Files to write to, "stdout" is a specially handled file. files = ["stdout", "/var/snmplog/metrics.out"] data_format = "splunkmetric" splunkmetric_hec_routing = false On the universal forwarder where my telegraf is running on, i create a inputs.conf stanza for the metrics.out file [monitor:///var/snmplog/*.out] disabled = false index = telegraf sourcetype = telegraf and in my company app a props.conf stanza for the sourcetype telegraf [telegraf] category = Metrics description = Telegraf Metrics pulldown_type = 1 DATETIME_CONFIG = NO_BINARY_CHECK = true SHOULD_LINEMERGE = true disabled = false INDEXED_EXTRACTIONS = json KV_MODE = none TIMESTAMP_FIELDS = time TIME_FORMAT = %s.%3N LINE_BREAKER = ([\r\n]+) I tried to create the index "telegraf" as event index and also as "metrics" index. What is the right type of index for telegraf sending as splunkmetrics? Running with metrics index type i dont see any events. Running with event type index, i see events, but no fields are extracted and i have a big event with hundreds of values. {"_value":103672,"metric_name":"diskio.weighted_io_time","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.writes","name":"loop2","time":1612282330}{"_value":47713280,"metric_name":"diskio.read_bytes","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.write_bytes","name":"loop2","time":1612282330}{"_value":114060,"metric_name":"diskio.read_time","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.write_time","name":"loop2","time":1612282330}{"_value":3872,"metric_name":"diskio.io_time","name":"loop2","time":1612282330}{"_value":39751,"metric_name":"diskio.reads","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.merged_reads","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.merged_writes","name":"loop2","time":1612282330}{"_value":0,"metric_name":"diskio.writes","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.write_bytes","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.write_time","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.io_time","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.merged_reads","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.merged_writes","name":"loop3","time":1612282330}{"_value":20,"metric_name":"diskio.reads","name":"loop3","time":1612282330}{"_value":32768,"metric_name":"diskio.read_bytes","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.read_time","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.weighted_io_time","name":"loop3","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"loop3","time":1612282330}{"_value":13287963,"metric_name":"diskio.reads","name":"sda","time":1612282330}{"_value":26131311,"metric_name":"diskio.writes","name":"sda","time":1612282330}{"_value":315109941248,"metric_name":"diskio.read_bytes","name":"sda","time":1612282330}{"_value":636415049728,"metric_name":"diskio.write_bytes","name":"sda","time":1612282330}{"_value":53542148,"metric_name":"diskio.write_time","name":"sda","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"sda","time":1612282330}{"_value":83744736,"metric_name":"diskio.read_time","name":"sda","time":1612282330}{"_value":30181608,"metric_name":"diskio.io_time","name":"sda","time":1612282330}{"_value":137261312,"metric_name":"diskio.weighted_io_time","name":"sda","time":1612282330}{"_value":142855,"metric_name":"diskio.merged_reads","name":"sda","time":1612282330}{"_value":36168488,"metric_name":"diskio.merged_writes","name":"sda","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"sda1","time":1612282330}{"_value":0,"metric_name":"diskio.writes","name":"sda1","time":1612282330}{"_value":48879616,"metric_name":"diskio.read_bytes","name":"sda1","time":1612282330}{"_value":3652,"metric_name":"diskio.read_time","name":"sda1","time":1612282330}{"_value":0,"metric_name":"diskio.write_time","name":"sda1","time":1612282330}{"_value":3652,"metric_name":"diskio.weighted_io_time","name":"sda1","time":1612282330}{"_value":1743,"metric_name":"diskio.reads","name":"sda1","time":1612282330}{"_value":0,"metric_name":"diskio.write_bytes","name":"sda1","time":1612282330}{"_value":3652,"metric_name":"diskio.io_time","name":"sda1","time":1612282330}{"_value":0,"metric_name":"diskio.merged_reads","name":"sda1","time":1612282330}{"_value":0,"metric_name":"diskio.merged_writes","name":"sda1","time":1612282330}{"_value":636415049728,"metric_name":"diskio.write_bytes","name":"sda2","time":1612282330}{"_value":53542148,"metric_name":"diskio.write_time","name":"sda2","time":1612282330}{"_value":30176468,"metric_name":"diskio.io_time","name":"sda2","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"sda2","time":1612282330}{"_value":142855,"metric_name":"diskio.merged_reads","name":"sda2","time":1612282330}{"_value":36168488,"metric_name":"diskio.merged_writes","name":"sda2","time":1612282330}{"_value":13282111,"metric_name":"diskio.reads","name":"sda2","time":1612282330}{"_value":314988112896,"metric_name":"diskio.read_bytes","name":"sda2","time":1612282330}{"_value":83727040,"metric_name":"diskio.read_time","name":"sda2","time":1612282330}{"_value":137243956,"metric_name":"diskio.weighted_io_time","name":"sda2","time":1612282330}{"_value":26131311,"metric_name":"diskio.writes","name":"sda2","time":1612282330}{"_value":0,"metric_name":"diskio.merged_writes","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.write_bytes","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.merged_reads","name":"loop0","time":1612282330}{"_value":17583104,"metric_name":"diskio.read_bytes","name":"loop0","time":1612282330}{"_value":42544,"metric_name":"diskio.read_time","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.write_time","name":"loop0","time":1612282330}{"_value":1432,"metric_name":"diskio.io_time","name":"loop0","time":1612282330}{"_value":36456,"metric_name":"diskio.weighted_io_time","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"loop0","time":1612282330}{"_value":12320,"metric_name":"diskio.reads","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.writes","name":"loop0","time":1612282330}{"_value":0,"metric_name":"diskio.write_bytes","name":"loop1","time":1612282330}{"_value":162976,"metric_name":"diskio.read_time","name":"loop1","time":1612282330}{"_value":0,"metric_name":"diskio.iops_in_progress","name":"loop1","time":1612282330}{"_value":0,"metric_name":"diskio.merged_reads","name":"loop1","time":1612282330}{"_value":0,"metric_name":"diskio.writes","name":"loop1","time":1612282330}{"_value":36114432,"metric_name":"diskio.read_bytes","name":"loop1","time":1612282330}{"_value":5880,"metric_name":"diskio.io_time","name":"loop1","time":1612282330}{"_value":139632,"metric_name":"diskio.weighted_io_time","name":"loop1","time":1612282330}{"_value":0,"metric_name":"diskio.merged_writes","name":"loop1","time":1612282330}{"_value":26595,"metric_name":"diskio.reads","name":"loop1","time":1612282330}{"_value":0,"metric_name":"diskio.write_time","name":"loop1","time":1612282330} What goes wrong? What is missing? Can someone help me? Thanks best regards Stefan

StefanW · ‎11-29-2020

Hab den Fehler gefunden. Warum auch immer fehlten so Berechtigungen für das default Team. Ich habe dann die Berechtigungen für das default Team neu ausgeführt mittels /bin/splunk cmd python itsi_reset_default_team.py Und danach ist der Button auch da.

isoutamo · ‎11-22-2020

Hi i haven’t seen any real officially supported way to do this. With most apps probably the easiest way is to use your own git repository where you keep official codes and own branch for your own changes/modifications. When a new version has published then just merge your own and latest versions to your new deployable version. r. Ismo

StefanW · ‎10-31-2020

i found my mistake. I wrote in my inputs.log the sourcetype=ias. But i have not deployed the app with the same inputs.conf to my search head. so splunk cannot show the entries. Strange... but ok. After copying the app to my searchhead/indexer i see the log entrys.

StefanW · ‎10-07-2020

We have a small splunk server up and running. More a proof of concept, but we believe its the right solution for our future. At the moment we bought the vmware app., so we can work with the data. The performance metrics of the esx server is aggregated from the vcenter and not the detailed metrics from the host itself. So if we would find problems or peaks, its not possible at the moment. I think the ITSI app would show us more detailed informations, but the price is to heavy. We only need 10% of the itsi solution.

StefanW · ‎09-27-2020

Hello, I am new to Splunk and we want to monitor our VMware environment. We installed the VMware app and add-on from Splunk to get data in. This is working, but the reports and dashboards are not what we need. It's too much and we only want some simple values. I tried to build my own searches, but it's hard for me to find the data that I need. Is there any explanation for how to work with the add-on data, or is there a better/simpler way to get the data from vSphere into Splunk? To start simple, I want a search to show me the system's overall CPU consumption from all hosts in the last 4 hours.

StefanW · ‎06-21-2020

Hello, does somebody know any ready app or something to parse dell idrac syslog messages?

Posts	18
Solutions	1
Karma Given	0
Karma Received	2
Member Since	‎06-21-2020

Online Status	Offline
Date Last Visited	‎04-13-2021 04:12 AM

Timezone configuration with daylight savings time

Round _value by mstats metrics

Monitor twice the same directory

Splunk Addon Unix and Linux high iostat values

Integrate telegraf not working as expected

Start ITSI create entitiy is missing

Best way to extend Apps

Splunk App for Infrastructure deploy through deplo...

Windows NPS Logfile not monitored

Alternative to vmware app

Timezone configuration with daylight savings time

Round _value by mstats metrics

Re: Monitor twice the same directory

Re: Splunk App for Infrastructure deploy through d...

Splunk Addon Unix and Linux high iostat values

Integrate telegraf not working as expected

Re: Start ITSI create entitiy is missing

Re: Best way to extend Apps

Re: Windows NPS Logfile not monitored

Re: Alternative to vmware app

VMware Add-on: How to create search that shows the...

Analyzing Dell iDrac syslog messages