Clear. So, an event with _time field with "+", in practice, represents a complete _time extraction with all "date_*" underfields inside 👍 Thanks 👏👏 ... View more

I think it's a right analysis 👍 Maybe, just for some tests 😊i'll try playing with "max_fd" in limits to see how System works. Just to stress the System 😂 as said in first post, it's only a test to understand better UF. I just saw how dangerous, sometimes, is to introduce the "..." or "*" or any other wildcard in path inputs, since UF could get crazy 🤷‍♂️like crcSalt, which could ingest x2/x3/x4/... data if not right blacklisted (think about log rotation with maybe gz/zip/bz extensions 🤷‍♂️ ). Anyway, there's something else than only fd. In a stable environment, UF should "leave free" the file (and drop its fd) after "time_before_close" (5 by default), so can process other files in queue. Another strange situation, i can't see any WARN about fd in splunkd.log, as sometimes i saw in other situations 🤔🤔🤔log explicitly said the max_fd was raised 🤔now not! Strange! Maybe this behaviour occurs different on different distros, as should be a System problem, not directly related to UF work 🤷‍♂️ ... View more

WARN .... Reason: binary. Should be right, since no binary is grant in props and by default is set to not get access to binaries. Descriptors to 100 is default, and it's ok, but should progress anyway. And in splunkd.log i can't see any WARN about descriptors. Now, why the "/etc" with all its ascii system files are not ingested since it's before the "/usr"? ... View more

Already done, System Paths are monitored, but no file is ingested 🙄 I think this is a security feature to exclude direct access to "/" 👍   Monitored Directories: /... /.autorelabel /afs /bin /boot /boot/.vmlinuz-5.14.0-284.11.1.el9_2.x86_64.hmac /boot/.vmlinuz-5.14.0-284.30.1.el9_2.x86_64.hmac /boot/config-5.14.0-284.11.1.el9_2.x86_64 /boot/config-5.14.0-284.30.1.el9_2.x86_64 /boot/efi /boot/grub2 /boot/initramfs-0-rescue-d264ca908f764f5191a3c479f3e6f4bc.img /boot/initramfs-5.14.0-284.11.1.el9_2.x86_64.img /boot/initramfs-5.14.0-284.11.1.el9_2.x86_64kdump.img /boot/initramfs-5.14.0-284.30.1.el9_2.x86_64.img /boot/initramfs-5.14.0-284.30.1.el9_2.x86_64kdump.img /boot/loader /boot/symvers-5.14.0-284.11.1.el9_2.x86_64.gz /boot/symvers-5.14.0-284.30.1.el9_2.x86_64.gz /boot/System.map-5.14.0-284.11.1.el9_2.x86_64 /boot/System.map-5.14.0-284.30.1.el9_2.x86_64 /boot/vmlinuz-0-rescue-d264ca908f764f5191a3c479f3e6f4bc /boot/vmlinuz-5.14.0-284.11.1.el9_2.x86_64 /boot/vmlinuz-5.14.0-284.30.1.el9_2.x86_64 /dev /dev/almalinux /dev/block /dev/bsg /dev/cdrom /dev/char /dev/core /dev/cpu /dev/disk /dev/dma_heap /dev/dri /dev/fd /dev/hugepages /dev/initctl /dev/input /dev/log /dev/mapper /dev/mqueue /dev/net /dev/pts /dev/rtc /dev/shm /dev/snd /dev/stderr /dev/stdin /dev/stdout /dev/vfio /etc /home /lib /lib64 /media /mnt /proc /proc/acpi /proc/bus /proc/dma /proc/fb /proc/fs /proc/irq /proc/keys /proc/kmsg /proc/net /proc/sys /proc/tty /root /run /sbin /srv /sys /This_is_Just_A_Test /usr     This can be also guessed by "/This_is_Just_A_Test" path, which contains many .txt files. With "/..." they are skipped, with explicit,     [monitor:///This_is_Just_A_Test]     They are ingested 👍 I really think it's a security feature to prevent "/" to be fully accessed. ... View more

Hi. I also have a large size files on some servers, about 10Gb per day in 3 files each server, and those files during the day are very delayed to be ingested, with ACK to true. While those files delay from 1 to also 4 hours to be indexed, other files on same servers are ingested fine in realtime. So, also with UF 8.2.12, i think it's a thruput of Network Infrastructure, or maybe too many datas from those inputs 🤷‍♂️ I also have   [thruput] maxKBps = 0 [general] parallelIngestionPipelines = 2 [queue] maxSize = 100MB [queue=parsingQueue] maxSize = 10MB I don't think there are other methods, since it's a phisiological problem 🤷‍♂️ The only way, maybe, is to add more Indexers in SPLUNK Infra or ask the Applicative Teams to split those file in more servers 🤷‍♂️ ... View more

OK, let's take back the argument... upgrading from 7.x to 8.x, new servers, new infrastructure... same annoying message... 🙄 We saw how to hide warning message icons in dashboards. Well, this is good! Now, how to set a threashold for the ms or remove the message at all from UI?   It becomes very very annoying!!! 😑 ... View more

Maybe could be tcp_Kprocessed == Kb received by the receiver as a packet of events kb == the real Kb (compressed) written on Indexer storage So, for my purposes, i keep on using the sum of "kb" as volume of data from UF to Indexers. Yes, it's not documented at all 🙄🙄🙄🤷‍♂️ Thanks! ... View more

I'm counting the "kb" as volume of data received and ingected into Indexers. Is this wrong, so? So, what's the relation beetwen "kb" and "tcp_Kprocessed" ? I'm still in doubt 🤔🤔🤔 ... View more

They did a real great mess, after 7.0, and some 8.x release Also with false in optimize_ui_prefs_performance, i'm now on 8.2.12 version, 1) optimize_ui_prefs_performance to true destroyes all old users customization on search tab ... also with optimize_ui_prefs_performance to false, 2) new ui-prefs.conf are not created anymore, only old ui-prefs are managed 3) also etc/users/launcher/local/ui-prefs.conf to remove "Explore Splunk Enterprise" banner has gone away! 4) users can't change Alerts/Reports/Dashboards object view modality (general/owner/app), since it's defaulted and reverted back to "All" next time you load the page!!!     5) seems ui-prefs is right managed only in "app/search/search|alerts|reports|dashboards" (default search App) This is really a great mess!!! We have had many users complain about this poor UI management!!!   ... View more

FACT: ui-prefs is broken! ☹️ ... View more

They did a real great mess, after 7.0, and some 8.x release 😤 Also with false in optimize_ui_prefs_performance, i'm now on 8.2.12 version, 1) optimize_ui_prefs_performance to true destroyes all old users customization on search tab ... also with optimize_ui_prefs_performance to false, 2) new ui-prefs.conf are not created anymore, only old ui-prefs are managed 3) also etc/users/launcher/local/ui-prefs.conf to remove "Explore Splunk Enterprise" banner has gone away! 4) users can't change Alerts/Reports/Dashboards object view modality (general/owner/app), since it's defaulted and reverted back to "All" next time you load the page!!! 5) seems ui-prefs is right managed only in "app/search/search|alerts|reports|dashboards" (default search App) This is really a great mess!!! 😩😰😱 Check here, https://community.splunk.com/t5/Splunk-Enterprise/quot-ui-prefs-conf-quot-no-more-working-from-Version-7-to/m-p/669361 We have had many users complain about this poor UI management!!! 🤢🙈 ... View more

Whis is as you have 2 UF on same machine. Maybe you should only increase the limits.conf,     [thruput] maxKBps = <integer> * The maximum speed, in kilobytes per second, that incoming data is processed through the thruput processor in the ingestion pipeline. * To control the CPU load while indexing, use this setting to throttle the number of events this indexer processes to the rate (in kilobytes per second) that you specify. * NOTE: * There is no guarantee that the thruput processor will always process less than the number of kilobytes per second that you specify with this setting. The status of earlier processing queues in the pipeline can cause temporary bursts of network activity that exceed what is configured in the setting. * The setting does not limit the amount of data that is written to the network from the tcpoutput processor, such as what happens when a universal forwarder sends data to an indexer. * The thruput processor applies the 'maxKBps' setting for each ingestion pipeline. If you configure multiple ingestion pipelines, the processor multiplies the 'maxKBps' value by the number of ingestion pipelines that you have configured. * For more information about multiple ingestion pipelines, see the 'parallelIngestionPipelines' setting in the server.conf.spec file. * Default (Splunk Enterprise): 0 (unlimited) * Default (Splunk Universal Forwarder): 256     Since by deault it send at 256Kb/s. I set it to 2048 for many UFs which send much data. You could also try a 0 to disable thruput control. ... View more

Turning OFF ACK may cause data-loss. ... View more

For some strange reason, in version 8.2.12 "ui-prefs" is not managed anymore by default. This is not much documented, as it should!!! 😠 To get rid of it, and get back with original ui management, we could try to edit a "local/web-features.conf"     [feature:ui_prefs_optimizations] optimize_ui_prefs_performance = false     But it's like playing the lottery, sometimes works, others not, with new apps not at all 🤦‍♂️   I don't think this is a good idea changing the bahaviour of UI so drastically. Above all, it's not documented anywhere, and we had to go around the web to understand it!!! 🤔🤔🤔🤔🤔 ... View more

The only way to reset the situation, is to manually edit the "etc/users/user/app/local/eventtypes.conf & tags.conf" "etc/apps/app/local/eventtypes.conf & tags.conf" "etc/apps/app/metadata/local.meta" and delete the objects there. And restart the Splunkd. But if you are inside a cluster, it's not much comfortable 😐 ... View more

No info at all? 🙄 ... View more