About vgrote

vgrote

And another not so happy user here. The documentation clearly states "When a search head cluster member is in manual detention, it stops accepting all new searches from the search scheduler or from users. Existing ad-hoc and scheduled search jobs run to completion. New scheduled searches are distributed by the captain to search head cluster members that are up and not in detention." As expected, an interactive search is refused. Yet when I monitor the active_historical_search_count of a member in detention, I observe the count going up and down. When I look at the Job Manager screen, I see lots of newly created jobs. Either I misunderstood the detention feature, or the documentation is off the mark, or there is a bug. What is it?

vgrote

Hi, the documentation I found details the update of a two-site cluster in "site-by-site" fashion, which is solid as a rock. We normally go that way, yet w/o taking down one site's the peers at once but by updating them one by none. And there is a description of a rolling update, where I did not find any mention of multi-site clusters. I tried a combination of both by rollingly updating one site and then the other, which at the end of the day did not speed up things very much, I still had to wait in the middle for the cluster to recover and become green again. Did I miss a description of the rolling update of a multi-site indexer cluster? What would be the benefit? And what's the difference anyway between going into maintenance mode and a rolling update? Thanks in advance Volkmar

vgrote · ‎05-23-2023

Correction: not on SLES12, one of the last remaining pockets with SLES11. And the root cause looks rather simple: $ splunk cmd mongod -version mongod: /lib64/libpthread.so.0: version `GLIBC_2.12' not found (required by mongod) $ ldd --version ldd (GNU libc) 2.11.3 On SLES12 is comes back with $ splunk cmd mongod -version db version v4.2.17-linux-splunk-v4 ... $ ldd --version ldd (GNU libc) 2.22 On the non-technical side this leads into the compatibility matrix and the various ways Linux distros are bundled and updated. Perhaps that library can be updated separately. Hope this helps Volkmar

vgrote · ‎05-22-2023

Hi, same problem on SLES 12. We upgraded to wiredTiger when we moved to 9.0.4 a while ago but only noticed the down rev version these days. I opened a case with Splunk since the error message is not exactly helpful for me when the migration log is about a month old and refers to the migration log. I will post the solution here. VGVG

vgrote · ‎03-07-2023

Hi, we ran into the same question, and currently there is no support for Universal Forwarders on AIX 7.3 (ref. case 3173471), but an Enhancement Request has been filed: "SPL-236862 AIX 7.3 for the Universal Forwarder (Support requested)". I just wonder what people do in the mean time, install regardless and keep your fingers crossed? HTH VGVG

vgrote · ‎01-19-2023

Running the Customer Success Toolkit's error report I noticed a warning on lots of Universal Forwarders that doesn't make sense to me: 19.01.23 10:49:53,614 01-19-2023 10:49:53.614 +0100 WARN UserManagerPro - Can't find [distributedSearch] stanza in distsearch.conf, using default authtoken HTTP timeouts host = Unix Box source = /opt/forwarder/data/var/log/splunk/splunkd.log sourcetype = splunkd 19.01.23 10:33:28,659 01-19-2023 10:33:28.659 +0100 WARN UserManagerPro - Can't find [distributedSearch] stanza in distsearch.conf, using default authtoken HTTP timeouts host = Windows Box source = C:\Program Files\Splunk\UniversalForwarder\var\log\splunk\splunkd.log sourcetype = splunkd Our Universal Forwarders have no distsearch.conf. Any idea why this is reported? And how to turn it off? We already have enough noise in our data. Thanks in advance Volkmar

vgrote · ‎11-13-2022

Hi jaspal95, sorry, I have no idea other than bringing a Unix shell to Windows. VGVG

vgrote · ‎11-03-2022

Thanks, Rich, for the swift reply. I realised that my problem statement was not correct, "convert ... into MET" is misleading. "At input time, honour a given time zone and do not assume local time" would have been slightly better. My problem is that there are different data formats in that input file, some UTC, some local time. Splunk converts all from local time into UTC for storage. Therefore I want Splunk to honour a time zone if present. Otherwise processing them later together and with data from other sources becomes a bit complicated. However, either my tests yesterday were wrong or some plug has been pulled, yet it seems to work as expected now: 2022-11-02T16:32:01+0300 UTC Test -> 02/11/2022 14:32:01.000 2022-11-02T16:32:02-0400 UTC Test -> 02/11/2022 21:32:02.000 2022-11-02T16:32:03-0000 UTC Test -> 02/11/2022 17:32:03.000 2022-11-02T16:32:040100 UTC Test -> 03/11/2022 08:31:41.000 = time of data entry (CET/+0100) 2022-11-02T16:32:05CET Test -> 02/11/2022 16:32:05.000 2022-11-02T16:32:06DUMMY Test -> 03/11/2022 08:51:38.000 = time of data entry (CET/+0100) 2022-11-02T16:32:07GMT Test -> 02/11/2022 17:32:07.000 2022-11-02T16:32:08UTC Test -> 02/11/2022 17:32:08.000 All's well that ends well! Volkmar

vgrote · ‎11-02-2022

Hi, I searched a lot and found no answer. I have data with the above timestamp and I want to convert it into local time. extract="year, month, day, hour, minute, second, zone" with (\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(\S+)\s+ works OK when time zone is given in the form of "+0000", yet not with "UTC". Is there something like "litzone" available? Thanks in advance Volkmar

vgrote · ‎06-22-2022

Plus ca change ... https://docs.splunk.com/Documentation/Splunk/9.0.0/ReleaseNotes/Knownissues 2021-09-22 SPL-212495, SPL-196040, SPL-219811 Excessive logging 'WARN SearchResultsFiles Unable to parse site_label, label=invalid due to err="Invalid site id: invalid"' for SearchResultsFiles Workaround: none

vgrote · ‎06-02-2022

Off topic: I wonder who or what changed the subject of this thread from "One input, two TZ" to "Why when we have One input, we are receiving two TZ?", which has nothing to with the Splunk side problem at all. I think this is not exactly helpful.

vgrote · ‎06-02-2022

@VatsalJagani: thanks for your suggestion. Sadly there is no chance to discriminate by host, source or sourcetype since the data arrives from a single syslog file input source. The colleagues sending the data already signalled that splitting the data into two streams may be an option, so all will be well soon.

vgrote · ‎06-01-2022

Hi, we have one syslog input where we receive log data from two different sources. One runs on local time, i.e. CEST, and carries a distinct string "abc", while the other runs on UTC and carries "def". For some unknown reason the UTC one doesn't carry "UTC" or "+00:00" with it, that information got stripped in transfer. Therefore it is currently off by two hours. To fix that, I want to pass the "abc" through unchanged, and set "UTC" on the "def", so that it will be correctly displayed at search time. My experiments with props.conf and transforms.conf (and datetime.xml) were not successful, since once the timezone is set at input time, it seems impossible to change it selectively for "def". Transforming the sourcetype is easy, but then it is too late, and the same applies to the host, so setting the TZ depending on a transformed parameter is not an option. Any ideas, apart from a conversation with the people who send the broken data? Thanks in advance Volkmar

vgrote · ‎05-06-2022

Did you check search.log? Any result? Did you run find ./ -type f -name '*.csv' | xargs grep ",$" What was the result?

vgrote · ‎05-05-2022

As garias_splunk wrote: you can try something like this command to find a .csv file that contains a comma followed by an end of line sign find ./ -type f -name '*.csv' | xargs grep ",$"

vgrote · ‎03-25-2022

Sorry to say, but I just installed 8.2.5 and ran straight into this issue 😞 VGVG

vgrote · ‎03-09-2022

Hello SanjayReddy, " [monitor:///apache/*/logs] " is mentioned as an example in https://docs.splunk.com/Documentation/Splunk/8.1.5/Data/Specifyinputpathswithwildcards#Input_examples. "/.../" would recurse into all subdirectories, which could be quite a long journey leading potentially across unknown directory structures, plus I got version B working anyhow which is probably taking less time. I just wonder if it works as documented. Kind Regards, Volkmar

vgrote · ‎03-09-2022

Hi, we have a directory with daily log files I want to read into Splunk 8.1.5: /dir1/dir2/dir3/dir4/file-20220309.log, file-20220308.log, ... Version A, working: "[monitor:///dir1/dir2/dir3/dir4]" Version B, working: "[monitor:///dir1/*/d*/dir4/*]" Version C, failing: "[monitor:///dir1/*/d*/dir4]" Version C would in theory match the example of "[monitor:///apache/*/logs]" in the documentation, wouldn't it? That is, as long as "logs" is a directory. Do I miss something here? Do I see a bug? Is there a limit on the number of wildcards in a path? Puzzled in Hamburg Volkmar

vgrote · ‎09-28-2021

So it looks like the author may have wanted to express something along "any repetition of a character that is not a colon", or in regex terms "(?:[^:])+", an asterisk would also match the space between two colons. YMMV

vgrote · ‎08-30-2021

Thanks again, Rich. That's the way I did it, lots of steps, but better be safe than sorry. Volkmar

Posts	24
Solutions	1
Karma Given	3
Karma Received	1
Member Since	‎02-18-2021

Online Status	Offline
Date Last Visited	a week ago

Update a two-site indexer cluster: to rock or to r...

Why does Universal Forwarders complain about missi...

How do I convert "2022-11-01T23:56:07UTC" into MET...

Why when we have One input, we are receiving two T...

Wildcard magic in monitor stanzas- Need help with ...

How to add / remove SHC members with static captai...

Why are we seeing this message: WARN SearchResults...

Can .rbsentinel files w/o a corresponding director...

Re: Search Head in a Search Head Cluster is in man...

Update a two-site indexer cluster: to rock or to r...

Re: KV Store Version Upgrade failed?

Re: KV Store Version Upgrade failed?

Re: Splunk Universal Forwarder can install in AIX ...

Why does Universal Forwarders complain about missi...

Re: WARN SearchResultsCSVSerializer - CSV file con...

Re: How do I convert "2022-11-01T23:56:07UTC" into...

How do I convert "2022-11-01T23:56:07UTC" into MET...

Re: Warning in internal log: unable to parse site_...

Re: Why when we have One input, we are receiving t...

Re: Why when we have One input, we are receiving t...

Why when we have One input, we are receiving two T...

Re: Why are we seeing this message: WARN SearchRes...

Re: WARN SearchResultsCSVSerializer - CSV file con...

Re: tstats where index=_internal no results

Re: Wildcard magic in monitor stanzas

Wildcard magic in monitor stanzas- Need help with ...

Re: Splunk App for Unix and Linux Bad Regex

Re: How to add / remove SHC members with static ca...