About burwell

burwell · ‎01-21-2021

Another idea: test out the PDF delivery with a super simple dashboard just to make sure all is working. I suggest one panel.

burwell · ‎12-23-2020

How about something like this index=security sourcetype="Computers" "Computer Status"=Enabled earliest=-12mon@mon | timechart span=1day count | timechart span=1month earliest(count) AS count | eventstats avg(count)as avg max(count) as max min(count) as min | where count = max .. change stats -> eventstats and then do some kind of comparison

burwell · ‎12-17-2020

So build on what @sjodle suggests here's what works best for me when I want to list a set of hosts to list in a slack window with carriage returns after each name. add space to the end of host name create new field name affected_hosts using values(host) so we have one field with all host names modify the new field to replace spaces with carriage return finally: use the field $result.affected_hosts$ in your slack message <your search here> | eval host=host+" " | stats values(host) as affected_hosts | rex mode=sed field=affected_hosts "s/ /\n/g"

burwell · ‎12-09-2020

| tstats count WHERE index=myindexname earliest=-60m latest=now

burwell · ‎12-05-2020

Hi. So the http event collect (HEC) can send to a set of indexers. I guess you are limited by how fast the HEC can send to the indexers and how fast the indexers in turn can ingest. I have seen recommendations from Splunk to indexer ingestion to be in the 300 to 400GB/day ingestion per indexer from all inputs but we push our indexers much harder.

burwell · ‎12-05-2020

Could install from the tar file in the interim? https://docs.splunk.com/Documentation/Forwarder/8.1.0/Forwarder/Installanixuniversalforwarder#Install_the_universal_forwarder_on_Linux Also file a bug with Splunk support.

burwell · ‎11-25-2020

Hi. What simplified things for me was to ask the people creating logs not to have the trailing comma. So the logs look like this {"timestamp": "2020-11-25 08:59:24 UTC", "hostname": "foo.com", "status": "failed"} {"timestamp": "2020-11-25 08:59:29 UTC", "hostname": "bar.com", "status": "passed"} Props are [my_sourcetype] INDEXED_EXTRACTIONS = JSON KV_MODE = none SHOULD_LINEMERGE=false TIME_PREFIX = timestamp\":\s*\" TIME_FORMAT=%Y-%m-%d %H:%M:%S %Z MAX_TIMESTAMP_LOOKAHEAD=23 TRUNCATE=99999

burwell · ‎11-12-2020

Can you provide exact props and exact sample event?

burwell · ‎11-06-2020

The map command was exactly what I needed. And thanks for the correction of the services command (works with/without first /) Also, if you have other fields you want to refer to in map use $field$

burwell · ‎11-05-2020

You could add a final | search status=clear

burwell · ‎11-05-2020

I have a search that will return a number of search ids. index=_audit | <various modifications>| table search_id Example.. I end up with search_id 1604617764.7885_5E002618-3E1F-491E-88C9-516508A9DB66 1604617764.7886_5E002618-3E1F-491E-88C9-516508A9DB66 1604617764.7887_5E002618-3E1F-491E-88C9-516508A9DB66 Now for each of search_id I want to do a rest call to find out information about the search For example, for the first search_id I want to call: | rest services/search/jobs/1604617764.7885_5E002618-3E1F-491E-88C9-516508A9DB66 splunk_server=local Since | rest is a generating command, I can't figure out how to do this.

burwell · ‎10-12-2020

I think search head clustering is the solution they you want. It allows you to share knowledge objects amongst the heads. And if you have four or more you can lose a head without undue pain. After you have it setup, the day to day administering should take less time than monitoring each individual head. You could use a monitoring console to monitor the SHC. You could use it to setup all kinds of alerts.

burwell · ‎10-02-2020

Hi. So how many panels are in each dashboard? Each one counts as a search.

burwell · ‎08-14-2020

I had those happen today. Restarting the forwarder fixed the problem.

burwell · ‎07-30-2020

So what's your architecture? How much data are your indexers indexing each day? How many indexers do you have? Are there other indexers that are not having cpu issues? I think a lot more info is needed here.

burwell · ‎07-30-2020

Hmm. Have you looked at the Splunkd.log on these indexers? What else is running? Could it be there is so much searching there are no free cycles? Does /bar/log/messages offer any hints?

burwell · ‎07-25-2020

Hello. I have seen 500 errors with bad cookies. To test can you run in incognito mode? If you don't get the 500 then use your browser settings to delete the cookies for your Splunk server. This has worked for me a number is times.

burwell · ‎07-14-2020

@tonyclifford Did you ever solve this problem? I am actually seeing the opposite problem: sometimes stats by _time on a summary data displays the x-axis in local time. I want UTC time.

burwell · ‎07-07-2020

Thanks on the even/oddness of members. In terms of what Splunk advises, I don't see any mention of having an odd number of members: https://docs.splunk.com/Documentation/Splunk/8.0.4/DistSearch/SHCarchitecture We have two main SHC, one with 5 heads and one with 6 heads. We haven't seen many of the election issues you mention that I am aware of. I have also administered a SHC with 4, again without issues. We have occasionally seen times where a head will decide that another member is more worthy to be captain, presumably based on some kind of timing issues. Never quite sure. We live with 6 heads and have hundreds of users.

burwell · ‎07-07-2020

You might find this post useful. https://community.splunk.com/t5/Alerting/Splunk-Alerting-rate/m-p/447186 Basically it is the role of the captain to distribute the searches. The link above will help you identify the reason for the skipped searches. It might be that you need to add one or more heads to your cluster to run jobs. You might need to require that the captain itself run jobs. I'd like to encourage you to add a fourth Splunk head. It makes your life a lot easier when you need to take one head down for whatever reason.

burwell · ‎07-01-2020

Hi. Can you show us what you have tried so far?

burwell · ‎06-26-2020

Hello. I work at the same company and am the technical contact with Splunk. Could you contact me please? Just look up burwell internally. Thanks!

burwell · ‎06-24-2020

Hello @mah I suggest you submit a new question as this is a different question. Thanks!

burwell · ‎06-19-2020

This response seems like a very good explanation of the watchdog messages https://community.splunk.com/t5/Archive2/watchdog-No-response-received-from-IMonitoredThread/td-p/479701

burwell · ‎06-16-2020

Hi. I was a beta tester of Splunk 7.3.1 and we "rolled our own" as the DFS manager was not available. Starting with 8.0 we used the DFS manager. The documentation says they don't support rolling your own. You have to install Spark 2.3.3 and Java yourself on your Spark nodes and manage the Spark configurations.

Posts	466
Solutions	50
Karma Given	145
Karma Received	165
Member Since	‎05-01-2014

Online Status	Offline
Date Last Visited	a month ago

Splunk bug in Enterprise 9.1 and 9.2: Indexers are...

Upgrading Enterprise from 9.1.7 -> 9.2.4: hit cloc...

Changing permission of a private knowledge object ...

Run a generating command over a set of values

Non-admin user needs savedsearch job date in dashb...

Best practices for deployment server: make sure ap...

MLTK: How best to conditionally update model in an...

Replacing third party server certificate while Spl...

How does Splunk log keyword pair extraction overwr...

How can I combine stats count by host into a singl...

Re: Scheduled Reporting but is Delivering Empty Re...

Re: How to create splunk rule query to determine ...

Re: Slack Webhook Alert

Re: Indexes Alert If No Incoming Data

Re: Capacity of HEC token

Re: splunkforwarder-8.1.0 New installation on Ubun...

Re: how to forward JSON data to splunk Properly

Re: Splunk timestamp offset GMT

Re: Run a generating command over a set of values

Re: how can i perform a filter after i do a search

Run a generating command over a set of values

Re: Best Practices For Syncing Knowledge Objects o...

Re: Big Data Brought Down The Hammer On Me... Thou...

Re: Splunk Forwarder issue

Re: 'Memory Usage on Splunk server

Re: 'Memory Usage on Splunk server

Re: Cannot login

Re: How do you get a timechart to display local ti...

Re: How to transfer reports between clustered sear...

Re: How to transfer reports between clustered sear...

Re: create an alert from two diffrent events

Re: Verizon media

Re: How do you extract a timestamp from a filename...

Re: Splunk WEB GUI Hang up On Loading

Re: DFS manager: Spark node failed to initialize

Join the Conversation