Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Join the Conversation
Without signing in, you're just watching from the sidelines. Sign in or Register to connect, share, and be part of the Splunk Community.
- Find Answers
- :
- About landen99
landen99
Motivator
Member since:
02-26-2014
07-02-2024
Community Statistics
| Posts | 475 |
| Solutions | 19 |
| Karma Given | 191 |
| Karma Received | 167 |
| Member Since | 02-26-2014 |
Activity Feed
- Got Karma for Re: renaming fields in search. 01-02-2025 04:35 AM
- Got Karma for Re: Merge two fields into one field. 10-23-2024 09:28 AM
- Got Karma for Re: Merge two fields into one field. 10-23-2024 09:24 AM
- Got Karma for Re: DNS debug log dns.log Format Review. 07-08-2024 10:21 AM
- Got Karma for Re: Merge two fields into one field. 04-25-2024 04:04 AM
- Got Karma for Re: Merge two fields into one field. 03-07-2024 12:35 PM
- Posted Re: Splunk time parse on Splunk Enterprise Security. 01-04-2024 09:38 AM
- Got Karma for Re: Dark theme. 06-12-2023 10:27 AM
- Got Karma for How do I get the dark theme to all of Splunk, regardless of version?. 06-12-2023 10:26 AM
- Got Karma for Re: Show all lines for Windows event-log events. 05-03-2023 09:47 AM
- Posted Re: Dealing with OMG huge events on Getting Data In. 03-08-2023 12:04 PM
- Posted Re: Browser sluggish when browsing through large raw data events. on Security. 03-08-2023 11:10 AM
- Got Karma for Re: inputlookup followed by transaction - No results found. 11-23-2022 07:30 AM
- Got Karma for Re: How can I generate a list of users and assigned roles?. 11-04-2022 06:20 AM
- Karma Re: Splunk web stuck at Loading page.. for att35. 10-12-2022 11:47 AM
- Karma Re: What are your Splunk t-shirt ideas? for nickstone. 10-04-2022 03:13 PM
- Karma Re: What are your Splunk t-shirt ideas? for ccl0utier. 10-04-2022 03:08 PM
- Posted Download Splunk Add-On Builder version 2.2.0 on All Apps and Add-ons. 10-04-2022 02:45 PM
- Posted How to resolve: ServiceNow TA Error "Failed to load current state for selected entity in form!" on Getting Data In. 08-24-2022 07:08 AM
- Got Karma for Re: how can i see all of the searches that are looking for a specific field?. 08-10-2022 08:02 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 |
07-18-2017
01:51 PM
Also, it seems that there are blocked queues:
index=_internal blocked=true | table host _raw
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:46:24.696 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:45:53.696 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
What could this mean?
... View more
07-18-2017
01:45 PM
I clarified the question to indicate that there is only one indexer configured and referenced in these logs. And that this happens to other forwarders as well.
... View more
07-18-2017
01:02 PM
The following internal metrics logs from statusmgr logs show the same host (X) fail, try, and connect to the same indexer (Y) in quick succession:
Host _raw
DESKTOP-X 07-18-2017 04:25:27.193 -0500 INFO StatusMgr - destHost=192.168.0.Y, destIp=192.168.0.Y, destPort=9997, eventType=connect_done, publisher=tcpout, sourcePort=8089, statusee=TcpOutputProcessor
DESKTOP-X 07-18-2017 04:25:27.149 -0500 INFO StatusMgr - destHost=192.168.0.Y, destIp=192.168.0.Y, destPort=9997, eventType=connect_try, publisher=tcpout, sourcePort=8089, statusee=TcpOutputProcessor
DESKTOP-X 07-18-2017 04:25:26.749 -0500 INFO StatusMgr - destHost=192.168.0.Y, destIp=192.168.0.Y, destPort=9997, eventType=connect_fail, publisher=tcpout, sourcePort=8089, statusee=TcpOutputProcessor
This same pattern appears for all the forwarders in the environment. What could this mean, is it a problem and how do I troubleshoot it further?
Also, it seems that there are blocked queues:
index=_internal blocked=true | table host _raw
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:45:20.067 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:46:24.696 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Z 07-18-2017 15:44:40.758 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=tcpin_cooked_pqueue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-X 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:44:49.066 -0500 INFO Metrics - group=queue, name=udp_queue, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
DESKTOP-Y 07-18-2017 15:45:53.696 -0500 INFO Metrics - group=queue, name=httpinputq, blocked=true, max_size_kb=0, current_size_kb=0, current_size=0, largest_size=0, smallest_size=0
... View more
07-12-2017
11:35 AM
3 Karma
I downvoted this post because the solution does not work. it just leaves you with output containing the values of field_2
... View more
06-29-2017
04:29 PM
If you disable the management port, what port does the UF fetch updates on? Is it not still on 8089 until the management port is changed?
... View more
06-29-2017
11:49 AM
The management port can be changed from 8089 (https://answers.splunk.com/answers/27394/running-splunk-forwarder-on-port-other-than-8089.html) in the web.conf mgmtHostPort setting but you would want to change that for all Splunk instances communicating with each other in the entire Splunk deployment. Send out the app with the web.conf setting from the DS to the UFs with the setting to restart and then add the web.conf setting to the DS and restart it. They will all be talking on the port that you specified for management communication including DS checkins and app pushes.
... View more
06-21-2017
03:15 PM
test connectivity with:
openssl s_client -connect company.splunkcloud.com:443
https://www.bearfruit.org/2008/04/17/telnet-for-testing-ssl-https-websites/
... View more
06-16-2017
07:05 AM
(Thank you, Gregg Woodcock, for the vote of confidence.)
It is possible that either you are still getting new events within your time range or that the bundles are out of synch between the search head and the indexers.
On the possibility that the events being indexed between the two searches fall within the time range being searched, the _indextime of the new events may fall outside your window while _time may continue to fall within your time range for awhile. In that case, even the same search will yield different results until the events being indexed stop falling within your time period.
Try the same search until successive attempts yield the same number of events. Then try the two searches without the dedup and compare the number of events. Look for words which assure that a field will be present and search on those instead to improve both accuracy and speed of the search (and to set the real number of events for comparison with the search time extraction performance). It is also worthwhile to verify that the search results are complete and that all indexers are reporting their results fully.
Added: Following DalJeanis' comment in the other answer, the appearance of the field userPhoneNumber in one of the normalized searches shows that a difference in knowledge object bundles yields different results. You may want to check autolookup or alias for that field.
... View more
04-17-2017
04:46 PM
Firewall is the most common problem. Check the firewall with the command on the client:
telnet ip port
If the DS is on a Windows box, make sure that the Windows firewall is either turned off or not blocking.
... View more
04-17-2017
04:44 PM
Check the firewall with the command on the client:
telnet ip port
If the DS is on a Windows box, make sure that the Windows firewall is either turned off or not blocking.
... View more
03-28-2017
04:15 PM
Thank you for the feedback and for the update. The question has been updated to note the AIX connection.
... View more
03-28-2017
02:16 PM
How do I resolve the following errors seen on an AIX system running an Universal Forwarder with TA-NMON installed?:
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load program /usr/opt/freeware/bin/rpm:
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module /usr/lib/librpm.so.0.0.0.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Dependent module /opt/splunkforwarder/lib/libz.a(libz.so.1) could not be loaded.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" The module has an invalid magic number.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module rpm.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Dependent module /usr/lib/librpm.so.0.0.0 could not be loaded.
03-26-2017 06:40:40.439 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module .
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load program /usr/opt/freeware/bin/rpm:
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module /usr/lib/librpm.so.0.0.0.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Dependent module /opt/splunkforwarder/lib/libz.a(libz.so.1) could not be loaded.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" The module has an invalid magic number.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module rpm.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Dependent module /usr/lib/librpm.so.0.0.0 could not be loaded.
03-26-2017 06:40:40.857 -0400 ERROR ExecProcessor - message from "/opt/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" Could not load module .
... View more
02-16-2017
01:30 PM
This solution assumes that you are starting with field1 and field2 not multivalue.
... View more
02-02-2017
01:54 PM
Splunk certifications (themselves) through Admin is free. Architect certification costs some.
... View more
02-02-2017
06:18 AM
I want to rename any number of fields/columns based on simple patterns.
From:
randomfields, a1.name1.stuff, a2.name2.foo, a3.othername.bar, a4.pickme.null
To:
randomfields, name1, name2, othername, pickme
How do I do this when the number of fields to be renamed can be 20, 30, or more? One considered solution involves either: transpose, rex, transpose, or: untable, xyseries, but both have brought their own issues with column names in the first event instead of in the field names, and with the key field respectively.
Also, they come from a SQL table in this case and look a little different than the examples above, so if you know the general form returned by dbxquery following the pattern above and know how to tell SQL or Splunk to return the fields in the right form matching above, please feel free to share that approach as well.
... View more
02-01-2017
11:59 AM
6 Karma
Perhaps more elegant (and practical for many fields) to do:
| rename field* AS *
... View more
11-07-2016
11:32 AM
400 cleared my kvStoreStatus error. Thank you.
curl -k -s https://localhost:8089/services/server/info | grep kvStore
... View more
11-01-2016
09:24 AM
Ideally, these regex strings should be matched against the user agent field:
\((?:KHTML, [\w ]+\)|compatible;|Android [\d\.]+; [\w \.;:]+\)) +(?<browser>[\w ]+)[\/ ]+(?<browser_ver>[\w\.+]+)
\((?:compatible;[^;]+; |Linux; )?(?<os>[a-zA-Z]+(?: NT)?) ?(?<os_ver>[\d\.]+)
... View more
10-14-2016
03:23 PM
The most efficient approach is a subsearch filter using the case-insensitive search command before the first pipe (preferred) if the list of values is under 50k:
index=myindex OR index=myindex2 [| inputlookup mycsv | search (cpu=1 hostname=stuff) OR (cpu=2 hostname=stuff2) | table hostname | rename hostname AS host ]
Creating a multi-value field to match on the lookup table, the mvappend command does the job:
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/CommonEvalFunctions
... | eval field_mvappend=mvappend(field_mv, "middle value", field_sv) | lookup mycsv field_mvappend OUTPUT mynewfield | search mynewfield=*
Or if the issue is only with case, you can either format the case in the lookup and in the source data using lower(). I prefer all field values to be lowercase.
... View more
10-14-2016
03:20 PM
Searching for events which match any of multiple values for the same field times several fields in a lookup using the subsearch filter or the mv_append eval function.
... View more
- Tags:
- splunk-enterprise
10-14-2016
03:17 PM
On the forwarder, you can see the search peers using tstats quite effectively.
| tstats count where index=_* by splunk_server
... View more
10-11-2016
11:40 AM
values allows the list to be much longer but it also removes duplicate field values and sorts the field values.
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
07-02-2024
08:32 PM
|
Karma from
Karma given to
