We have simple csv lookup like: network,descr 192.168.0.0/24,network_name Lookup description in transforms.conf: [networklist_allocs_all] filename = networklist_allocs_all.csv max_matches = 1 min_matches = 1 default_match = OK match_type = CIDR(network) Any search command like: ... | lookup networklist_allocs_all network AS src_ip ... too often crashing splunk on the indexers. It is about 5-10 crashlogs on the each indexers per day. Part of crashlog at the end of question. How can we resolve this situation? Seems that splunk began to crash after update from 7 to 8 version. We did't any changes in lookup format or definition. Unfortunately we can't open support case for some reason, so ask for community help. crash-xx.log: [build 6db836e2fb9e] 2020-02-13 17:00:56 Received fatal signal 11 (Segmentation fault). Cause: No memory mapped at address [0x0000000000000058]. Crashing thread: BatchSearch Registers: RIP: [0x0000564E2F44470F] _ZN14LookupMatchMap16mergeDestructiveERS_ + 31 (splunkd + 0x223670F) RDI: [0x00007FC9C01FCA80] RSI: [0x0000000000000000] RBP: [0x0000000000000000] RSP: [0x00007FC9C01FC9A0] RAX: [0x00007FC9CDB3AE08] RBX: [0x0000000000000000] RCX: [0x0000000000000001] RDX: [0x00007FC9BFB7F608] R8: [0x00007FC9C01FC9C0] R9: [0x00007FC9C01FC9BF] R10: [0x0000000000000010] R11: [0x0000000000000080] R12: [0x00007FC9928E33C0] R13: [0x00007FC9C01FCA80] R14: [0xAAAAAAAAAAAAAAAB] R15: [0x00007FC9BFB7F600] EFL: [0x0000000000010246] TRAPNO: [0x000000000000000E] ERR: [0x0000000000000004] CSGSFS: [0x0000000000000033] OLDMASK: [0x0000000000000000] OS: Linux Arch: x86-64 Backtrace (PIC build): [0x0000564E2F44470F] _ZN14LookupMatchMap16mergeDestructiveERS_ + 31 (splunkd + 0x223670F) [0x0000564E2F444EE8] _ZN14UnpackedResult8finalizeEv + 168 (splunkd + 0x2236EE8) [0x0000564E2F445E26] _ZN18LookupDataProvider6lookupERSt6vectorIP15SearchResultMemSaIS2_EERK17SearchResultsInfoR16LookupDefinitionPK22LookupProcessorOptions + 2118 (splunkd + 0x2237E26) [0x0000564E2F451B7F] _ZN12LookupDriver13executeLookupEP29IFieldAwareLookupDataProviderP15SearchResultMemR17SearchResultsInfoPK22LookupProcessorOptions + 367 (splunkd + 0x2243B7F) [0x0000564E2F451C22] _ZN18SingleLookupDriver7executeER18SearchResultsFilesR17SearchResultsInfoPK22LookupProcessorOptions + 98 (splunkd + 0x2243C22) [0x0000564E2F43BDFF] _ZN15LookupProcessor7executeER18SearchResultsFilesR17SearchResultsInfo + 79 (splunkd + 0x222DDFF) [0x0000564E2F08FC7D] _ZN15SearchProcessor16execute_dispatchER18SearchResultsFilesR17SearchResultsInfoRK3Str + 749 (splunkd + 0x1E81C7D) [0x0000564E2F07F528] _ZN14SearchPipeline7executeER18SearchResultsFilesR17SearchResultsInfo + 344 (splunkd + 0x1E71528) [0x0000564E2F1931D9] _ZN16MapPhaseExecutor15executePipelineER18SearchResultsFilesb + 153 (splunkd + 0x1F851D9) [0x0000564E2F193901] _ZN25BatchSearchExecutorThread13executeSearchEv + 385 (splunkd + 0x1F85901) [0x0000564E2DEDCB8F] _ZN20SearchExecutorThread4mainEv + 47 (splunkd + 0xCCEB8F) [0x0000564E2EC41AC8] _ZN6Thread8callMainEPv + 120 (splunkd + 0x1A33AC8) [0x00007FC9CC7B76BA] ? (libpthread.so.0 + 0x76BA) [0x00007FC9CC4EC41D] clone + 109 (libc.so.6 + 0x10741D) Linux / c-6.index.splunk / 4.4.0-21-generic / #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 / x86_64 /etc/debian_version: stretch/sid ... Last errno: 0 Threads running: 13 Runtime: 436566.363455s argv: [splunkd -p 8089 start] Process renamed: [splunkd pid=8371] splunkd -p 8089 start [process-runner] Process renamed: [splunkd pid=8371] search --id=remote_d-0.search.splunk_scheduler__gots_eWFuZGV4LWFsZXJ0cw__RMD5d361bb2fcae608a1_at_1581602460_31361_374F9CE8-43DB-48C4-9F22-7982EC4B6AD5 --maxbuckets=0 --ttl=60 --maxout=0 --maxtime=0 --lookups=1 --streaming --s idtype=normal --outCsv=true --acceptSrsLevel=1 --user=gots --pro --roles=admin:power:user Regex JIT enabled RE2 regex engine enabled using CLOCK_MONOTONIC Preforked process=0/59436: process_runtime_msec=54987, search=0/188869, search_runtime_msec=1240, new_user=Y, export_search=Y, args_size=356, completed_searches=3, user_changes=2, cache_rotations=3 Thread: "BatchSearch", did_join=0, ready_to_run=Y, main_thread=N First 8 bytes of Thread token @0x7fc9c70845e8: 00000000 00 e7 1f c0 c9 7f 00 00 |........| 00000008 SearchExecutor Thread ID: 1 Search Result Work Unit Queue: 0x7fc9b77ff000 Search Result Work Unit Queue Crash Reporting Type: NON_REDISTRIBUTE Number of Active Pipelines: 2 Max Count of Results: 0 Current Results Count: 352 Queue Current Size: 0 Queue Max Size: 200, Queue Drain Size: 120 FoundLast=N Terminate=N Total Bucket Finished: 1.0012863152522, Total Bucket Count: 16 ===============Search Processor Information=============== Search Processor: "lookup" type="SP_STREAM" search_string=" networklist_allocs_all network AS dest_ip " normalized_search_string="networklist_allocs_all network AS dest_ip" litsearch="networklist_allocs_all network AS dest_ip " raw_search_string_set=1 raw_search_string=" networklist_allocs_all network AS dest_ip " args={StringArg: {string="networklist_allocs_all" raw="networklist_allocs_all" quoted=0 parsed=1 isopt=0 optname="" optval=""},StringArg: {string="network" raw="network" quoted=0 parsed=1 isopt=0 optname="" optval=""},StringArg: {string="AS" raw="AS" quoted=0 parsed=1 isopt=0 optname="" optval=""},StringArg: {string="dest_ip" raw="dest_ip" quoted=0 parsed=1 isopt=0 optname="" optval=""}} input_count=885 output_count=0 directive_args= ========================================================== ... View more

Is it possible to force Splunk to set up specific fields (sourcetype, source, host) from HEC local stanza but not from event parameters? For example if i have inputs.conf like: [http://http_test] description = test hec input disabled = 0 connection_host = dns index = main source = test_hec_source sourcetype = test_hec_sourcetype Client can rewrite index, host, source and sourcetype if post in data json like: { "host": fake_host, "sourcetype": fake_sourcetype, "index": fake_index, "source": fake_source } But in certain situations i need to deny modifications of this parameters. Of course i can do it with transforms.conf, but it is not convinient. ... View more

I have reason to change *.conf in local directory of app on the searchhead in shcluster, and i need to tell to this searchhead to send changes to captain for spreading via whole cluster. How i can do it via cli or REST? Thank you ... View more

We have big file with events in each line of file. Every minute file transfers via rsync to forwarder with new events. Events format: timestamp="XXXX" login="YYY" action="ZZZ" Size of file about 10Mb, with 15000 lines. Sometimes (1-2 times per minute) forwarder compile one event from multiple (usually 2) lines. Configs from forwarder: inputs.conf: [monitor:///file.log] disabled = false followTail = 1 index = f_logs sourcetype = f_logs multiline_event_extra_waittime = true time_before_close = 10 props.conf: [f_logs] TIME_PREFIX = timestamp=\" TIME_FORMAT = %s SHOULD_LINEMERGE = true MAX_EVENTS = 1 What i have to do to solve this problem? ... View more

I trying to create leases table from log. For this task i have write python script which prepare data for lookup. Is it possible to write this data back to splunk for using them in lookup command. So i need something like outputlookup, but from python script. Thank you. ... View more

We have an index with access logs from multiple hosts and systems with different sourcetypes. When I trying to add information from a dynamic lookup to events and save them in a summary index with the collect command, I can't save original information about source, sourcetype, and host because collect command arguments take values as text, but not field values. For example, search: index=access sourcetype=*_type_access | lookup xxx AS yyy | collect index=enriched_access sourcetype=sourcetype saves results with sourcetype equal "sourcetype", but not the original sourcetype. When I try to rename sourcetype, result is the same. Where a, I going wrong? ... View more