About las

las · ‎07-02-2020

Hi Guiseppe. I have not been sucessfull yet, but I'm still working on the principal and have asked another question on how the reference the _meta variable in a transform. Kind regards Lars

las · ‎07-02-2020

Hi. I have a business requirement where I need to index data from multiple of our vendors that also use Splunk. The vendors have added a _TCP_ROUTING to send data to both our Heavy Forwarders and their own infrastructure. I have a dedicated port for each vendor in my inputs.conf on the Heavy Forwarder: [splunktcp-ssl:9997] disabled = 0 _meta userindex::splunk_test My idea was to have a different userindex for each input stanza Next step is a generic props.conf: [host::*] TRANSFORMS-force_index = force_index Finally I was hoping it would be possible to do the magic in my transforms.conf: [force_index] DEST_KEY = MetaData:Sourcetype REGEX = (.+) FORMAT = $1 SOURCE_KEY = _meta:userindex WRITE_META = true I know I'm not rewriting the index, but it is easier to look at the sourcetype, as the events get indexed and it should be a small change to rewrite the index instead of the sourcetype. Long story... so to the question. Is it possible to reference the _meta variable I have set in the input stanza in the regex of the transform on the same Heavy Forwarder? Kind regards Lars P.S. I agree it is a bad idea to rewrite the index, it should be set at the source, but I think it is necessary, as our indexes do not match those of our vendors and I want each vendors data to be indexed in the same index.

las · ‎06-24-2020

Hi. Yes, that is the easy way, my problem is that I'm getting data from a Universal Forwarder, that I do not have any control over, it is located at a vendor, but I'm also required to store the data. The Universal Forwarders have used the index stanza, so there is not an easy way to overwrite that.

las · ‎06-24-2020

Hi Guiseppe. I still haven't put this one to rest. If I use the _meta in inputs.conf [splunktcp-ssl:9990] disabled = 0 _meta userindex::index1 [splunktcp-ssl:9991] disabled = 0 _meta userindex::index2 then I might be able to route everything thru to my transform where I could do something like this [force_index] DEST_KEY = _MetaData:Index REGEX = userindex::(\w*)\s FORMAT = $1 Do you think this might be a possible solution? Kind regards. Lars

las · ‎03-30-2020

Hi Guiseppe. I had the idea, that it would be from Universal Forwarders. Generally I aggree with you, that it is a bad idea to have the same sourcetypes in multiple indexes, but here it makes sense, as only a select few will have access to data from a specific supplier. Kind regards Lars

las · ‎03-30-2020

Hi. We are about to ingest logs from multiple suppliers, where the individual supplier has full control over their infrastructure. My take was to to create a couple of heavy forwarders and dedicate a port to each supplier: supplier1 sends data to port 9991 supplier2 sends data to port 9992 ... This part I think I have working. The next problem is that I have a need to separate the data from supplier1 from supplier2, My thought was to create a index per supplier. The problem is then how do I route data received from port 9991 to index1 regardless of what is configured on the Universal Forwarder, except for Splunk stuff (internal ...) the different suppliers might use the same source or sourcetype, so it is only the receiving port on the heavy forwarder I might use to separate the data. Any help is much appreciated Kind regards

las · ‎02-20-2020

Thank you for your answer, I was just curious if anybody had done it, and if it was done, if they found an add-on or created either a scripted- or modular-input

las · ‎02-10-2020

Hi. It seems Microsoft has exposed the audit log for Azure DevOps, https://docs.microsoft.com/en-us/rest/api/azure/devops/audit/audit%20log/query?view=azure-devops-rest-5.1 Has anyone tried to index this log and how did you do it? Kind regards las

las · ‎01-07-2020

We are using windows Server 2016, that might have an impact

las · ‎01-03-2020

We see the exact same problem also on 8.0.1 @chrisitanmolecki have you opened a case with support?

las · ‎06-20-2019

There was no problems with the searching of the data. The problem was with the metric-name, where the props somehow didn't set the first part of the name (process....) so SAI didn't pick up, that there was any data in the index. This is probably some inconsistency with Splunk Add-on for Windows infrastructure.

las · ‎06-06-2019

Hi. I have a setup on Splunk 7.2.4 with a search head, that searches both an index-cluster and a standalone indexer. I have deployed splunk-add-on-for-infrastructure131 on both the index-cluster and the standalone indexer. I have deployed splunk-app-for-infrastructure131 on the search-head. When I try to use SAI on the search head I only get results from the standalone indexer not from the Cluster. This is my first experience with metric indexes so I'm not sure if there has to be some special considerations when using a index-cluster. Other data is searchable in this setup, so the connection is in order between the search-head and the indexcluster. Can anyone please help getting the clustered data available in the search head? The index is on the indexcluster, and on the Clustermaster I can see it has some data in it (5 bucket 0.03 GB, 2.8M events on one of the indexers). Kind regards las

las · ‎04-17-2019

I had a few issues, but now it seems to work [snow:mid] NO_BINARY_CHECK = true CHARSET = AUTO TIME_FORMAT = %m/%d/%y %H:%M:%S TIME_PREFIX=^ TZ = Europe/Copenhagen MAX_TIMESTAMP_LOOKAHEAD=30 SHOULD_LINEMERGE=true LINE_BREAKER=([\r\n]+) category=Custom Kind regards

las · ‎04-17-2019

Thank you.

las · ‎04-17-2019

Hi. I can't see a download for windows on the openJDK page referenced in the doc. Do you know if OpenJDK is just supported on Linux? Kind regards las

las · ‎04-17-2019

I forgot to mention, the Splunk installation is on Windows, and I can't find an OpenJDK for Windows Binary download. I did find AdoptOpenJDK, but that is not supportet.

las · ‎04-17-2019

Hi. With Oracles new licensing policy for java, does anybody know if the Splunk DB Connect can use another Java? Kind regards las

las · ‎04-11-2019

This props.conf seems to work [snow:mid] NO_BINARY_CHECK = true CHARSET = AUTO TIME_FORMAT = %m/%d/%y %H:%M:%S TIME_PREFIX=^ MAX_TIMESTAMP_LOOKAHEAD=30 category=Custom Thank you very much

las · ‎04-11-2019

Sorry, I should have been more clear on the second text box. That is a screen copy of Splunk. So lines 1 and 2 are Splunk Timestamp as extracted from the log Lines 3 thru 15 included are the Splunk event Line 16 (Collapse) event Line 17 Splunk info on the event (host, source and sourcetype) Lines 18 and 19 are Splunk Timestamp as extracted from the log Line 20 is the event Line 21 Splunk info on the event (host, source and sourcetype) I will try your config and get back with the result. Kind regards

las · ‎04-10-2019

Hi. I'm indexing the log from the Service-now MID-Server, that should be nice and easy, most lines start with a timestamp, and a few are multiline events. 04/10/19 11:50:24 (365) LogStatusMonitor.60 stats threads: 67, memory max: 910.0mb, allocated: 402.0mb, used: 29.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes 04/10/19 11:51:24 (373) LogStatusMonitor.60 stats threads: 67, memory max: 910.0mb, allocated: 402.0mb, used: 29.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes 04/10/19 11:52:24 (406) LogStatusMonitor.60 stats threads: 67, memory max: 910.0mb, allocated: 402.0mb, used: 30.0mb, standard.queued: 0 probes, standard.processing: 0 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes 04/10/19 11:52:45 (197) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Worker starting: JDBCProbe source: 7f753a46db16a740c53f6c16ca9619a9 04/10/19 11:52:45 (275) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Waiting for connection semaphore 04/10/19 11:52:45 (275) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Obtained connection semaphore 04/10/19 11:52:45 (275) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af SELECT DISTINCT pco.PCO_Ref, pco.Act_Plan, pco.Description, pco.Impact, pco.Prioritet, pco.risk, pco.Txt_type, convert(varchar(MAX), pco.Txt) as Txt, pco.Urgency, status.CloseType, status.PCOClose, status.PCOPlan2, status.PCOrelease3, act.SenesteStart, act.SenesteEnd FROM "PobPco"."dbo"."PCO" pco join "PobPco"."dbo"."PCO_status" status on pco.PCO_Ref = status.PCO_Ref FULL outer join "PobPco"."dbo"."Act" act on pco.PCO_Ref = act.PCO_Ref and (act.Sequence='430' OR act.Sequence='491') WHERE (pco.Act_Plan='LEV_UDK_LEVERANCESITE' OR pco.Act_Plan='APPL_TWS_INITIERET' OR pco.Act_Plan='TEK_GENEREL_WEB') 04/10/19 11:53:24 (392) LogStatusMonitor.60 stats threads: 67, memory max: 910.0mb, allocated: 402.0mb, used: 30.0mb, standard.queued: 0 probes, standard.processing: 1 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes On the universal forwarder, there's just an input.conf, that sets sourcetype and index. On the indexer I have a props.conf with this content: [snow:mid] SHOULD_LINEMERGE = true NO_BINARY_CHECK = true CHARSET = AUTO disabled = false TIME_FORMAT = %m/%d/%y %H:%M:%S category = Custom My problem is that for some reason Splunk doesn't do event break correctly 10/04/2019 11:53:24.000 04/10/19 11:53:38 (565) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf050000001.xml 04/10/19 11:53:38 (596) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf140000001.xml 04/10/19 11:53:38 (611) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf240000001.xml 04/10/19 11:53:38 (627) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf430000001.xml 04/10/19 11:53:38 (643) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf530000001.xml 04/10/19 11:53:38 (643) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Enqueuing: D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\work\monitors\ECCSender\output_s\ecc_queue.16a06aacf530000002.xml 04/10/19 11:53:38 (643) Worker-Standard:JDBCProbe-69923714db3cf300e6067742399619af Worker completed: JDBCProbe source: 7f753a46db16a740c53f6c16ca9619a9 time: 0:00:53.368 04/10/19 11:53:38 (880) ECCSender.1 Sending ecc_queue.16a06aacf050000001.xml 04/10/19 11:53:38 (989) ECCSender.1 Sending ecc_queue.16a06aacf140000001.xml 04/10/19 11:53:39 (098) ECCSender.1 Sending ecc_queue.16a06aacf240000001.xml 04/10/19 11:53:39 (192) ECCSender.1 Sending ecc_queue.16a06aacf430000001.xml 04/10/19 11:53:39 (270) ECCSender.1 Sending ecc_queue.16a06aacf530000001.xml 04/10/19 11:53:39 (364) ECCSender.1 Sending ecc_queue.16a06aacf530000002.xml Collapse host = ATP-34UMID02 source = D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\logs\agent0.log.0 sourcetype = snow:mid 10/04/2019 11:53:24.000 04/10/19 11:53:24 (392) LogStatusMonitor.60 stats threads: 67, memory max: 910.0mb, allocated: 402.0mb, used: 30.0mb, standard.queued: 0 probes, standard.processing: 1 probes, expedited.queued: 0 probes, expedited.processing: 0 probes, interactive.queued: 0 probes, interactive.processing: 0 probes host = ATP-34UMID02 source = D:\ServiceNow\MID_Server\atp-34umid02_dev01\agent\logs\agent0.log.0 sourcetype = snow:mid

Posts	114
Solutions	9
Karma Given	42
Karma Received	19
Member Since	‎12-05-2011

Online Status	Offline
Date Last Visited	‎07-03-2020 05:32 AM

Re: overwrite index on heavy forwarder based on po...

Can I use a _meta variable from inputs in a transf...

Re: overwrite index on heavy forwarder based on po...

Re: overwrite index on heavy forwarder based on po...

Re: overwrite index on heavy forwarder based on po...

overwrite index on heavy forwarder based on port

Re: Has anyone indexed Azure Devops audit log?

Has anyone indexed Azure Devops audit log?

Re: Event: Show source loading ....

Re: Event: Show source loading ....

Re: Metrics index in index cluster not searchable

Metrics index in index cluster not searchable

Re: Index Service-Now MID server logs

Re: Dependency on Oracle java?

Re: Dependency on Oracle java?

Re: Dependency on Oracle java?

Dependency on Oracle java?

Re: Index Service-Now MID server logs

Re: Index Service-Now MID server logs

Index Service-Now MID server logs