About darrenfuller

supreet · ‎02-06-2024

For me, it is going to be ongoing thing and not a one time effort. So wondering if there is a way to achieve this

RedStapler · ‎05-10-2022

This method worked for me on an Windows Event log where "Account Name:" appears twice on every event and I only wanted the 2nd name, not the first. The only oddity unresolved was when one of the two names were null in the event. There might be a way to fix that. Other than that, it worked. index=* host=*servername* password EventCode=4625 | rex field=Message "Account Name:\s+(?<ACCOUNT_NAME>[^\r\n]+)" max_match=0 | eval Account_Name_1=mvindex(Account_Name, 0) | eval Account_Name_2=mvindex(Account_Name, 1) | table DomainController, Account_Name_2

darrenfuller · ‎03-23-2021

I have a csv file that is written to once a day. The input points to a custom sourcetype [csvtest] which has appropriate settings for the data within. inputs.conf: [monitor://c:\opt\splunk\etc\apps\csvtest\data\csvtest.csv] index = main sourcetype = csvtest props.conf: [csvtest] LINE_BREAKER = ([\r\n]+) SHOULD_LINEMERGE = false TIME_PREFIX = ^ TIME_FORMAT = %Y-%m-%d MAX_TIMESTAMP_LOOKAHEAD = 15 TRUNCATE = 10000 and data looks as so: 2020-01-26,,,1.0,,,,,,,,9,,,,,,,,,, 2020-01-27,,,2.0,,,,,,,,19,,,,,,,,,, 2020-01-28,,,1.0,1.0,,,1.0,,,,11,,,,,,,,,, 2020-01-30,,,0.0,2.0,,,2.0,,,,27,,,,,,,,,, 2020-01-31,,,0.0,2.0,,,2.0,,,,17,,,,,,,,,, 2020-02-03,,,0.0,3.0,,,3.0,,,,29,,,,,,,,,, 2020-02-04,90.0,12.0,0.0,3.0,,,3.0,139.0,,,34,,,,,,,,,, 2020-02-05,96.0,8.0,0.0,3.0,,,3.0,150.0,,,43,,,,,,,,,, 2020-02-06,104.0,0.0,0.0,3.0,,,3.0,169.0,,,62,,,,,,,,,, 2020-02-08,130.0,25.0,0.0,3.0,,,3.0,197.0,,,39,,,,,,,,,, 2020-02-10,167.0,81.0,0.0,3.0,,,3.0,259.0,,,8,,,,,,,,,, 2020-02-11,184.0,79.0,0.0,3.0,,,3.0,285.0,,,19,,,,,,,,,, 2020-02-12,257.0,44.0,0.0,2.0,1.0,,3.0,313.0,,,9,,,,,,,,,, 2020-02-13,306.0,16.0,0.0,2.0,1.0,,3.0,340.0,,,15,,,,,,,,,, 2020-02-14,353.0,0.0,0.0,2.0,1.0,,3.0,364.0,,,8,,,,,,,,,, 2020-02-17,399.0,0.0,0.0,2.0,1.0,,3.0,402.0,,,0,,,,,,,,,, 2020-02-18,418.0,0.0,0.0,2.0,1.0,,3.0,421.0,,,0,,,,,,,,,, 2020-02-19,436.0,0.0,0.0,2.0,1.0,,3.0,456.0,,,17,,,,,,,,,, 2020-02-20,462.0,,0.0,1.0,2.0,,3.0,479.0,,,14,,,,,,,,,, 2020-02-21,483.0,,0.0,0.0,3.0,,3.0,498.0,,,12,,,,,,,,,, 2020-02-22,540.0,,0.0,1.0,3.0,,4.0,553.0,,,9,,,,,,,,,, Every time this file adds a new line to the bottom , the whole file is ingested... causing duplicate data in the index. I have created the same scenario on Linux and the forwarder appropriately uses the default initCrcLength and identifies it has seen the file before and only ingests the new event. in the _internal index, i am seeing events like so> 03-23-2021 20:00:02.851 -0400 INFO WatchedFile - Will begin reading at offset=0 for file='C:\opt\splunk\etc\apps\csvtest\data\csvtest.csv'. Is this just a windows forwarder symptom? Is there an inputs.conf setting I can utilize to fix this? Thanks in advance!

lim2 · ‎01-12-2021

Splunk UF agent to 'reprocess' a monitor file. Noticed that there are events in the SplunkUF monitored file are indexed. Wondering if ./splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/messages --reset would do an incremental reindex of the missed events of the file?

fk319 · ‎05-29-2020

All this I understand. In my case app1, app2, and app3 are the same app, but they are different users. They share the same dashboards and searches but have different data. I am looking for a way for an admin, to review mylookup for all three apps at the same time. in my case, mylookup will seldom change, but if user1 makes a change in app1, I would like for the admin to look at their dashboard and review the change without checking the apps individually.

darrenfuller · ‎05-29-2020

Hi bnichols024, I think your REGEX is incorrect....you made the capture group a named group called event_type, rather than looking for the string. Try this: [setnull] REGEX = (event_type = SETXATTR) DEST_KEY = queue FORMAT = nullQueue

BongoTheWhippet · ‎05-06-2020

You're right, but ewwww, that's expected behaviour?

DalJeanis · ‎04-30-2020

@caiomozer - It's always easiest to see someone ELSE's issues. Can't tell you the number of hours we've spent chasing things like that over the decades.

jonzatlmi · ‎04-08-2020

thank you!

darrenfuller · ‎04-06-2020

If you have sufficient access on Splunk, if you create all the files, then compress them into a tar.gz file, then use: Apps -> Manage Apps -> Install app from file, to load the app onto your splunk installation, you may not need command line access. Then restart the Splunk instance and it should work fine. Splunk will run on your laptop/desktop so you can create and test the configs before you install them on the production instance..

ashwinipatil007 · ‎04-06-2020

yes I have forwader which monitors the SMTP system

merrelr · ‎04-01-2020

I had to update the props to use 127.0.0.1 instead of it's actual IP. I'm not sure what changed since yesterday with my testing. I left the endpoint as /services/collector/event and my props/transforms are working. props.conf [host::127.0.0.1] TRANSFORMS-$hostname_rename = host_rename_$hostname transforms.conf [host_rename_$hostname] REGEX = (.*) DEST_KEY = MetaData:Host FORMAT = host::$hostname

snix · ‎03-30-2020

After looking closer at it I did find most of the events contained a combination of multiple events into one event. Not sure why because I would think what you have would work. I don't pretend to understand much about return carriages and new lines in the little amount of programing I have to deal with but it looked good. I took some of the output from the log file and pasted it into Notpad++ and did a show of all characters and it showed CR LF at the end of each line so that looks good to me. That said I commented out the LINE_BREAKER line and replaced it with "BREAK_ONLY_BEFORE = \d\d?:\d\d:\d\d" which I found under the log4j stanza and it worked. Since I don't grasp 100% what I am doing I am sure this is not the best way to do it but it did get the results I was looking for. If someone understands what is going on and would like to explain it I am all ears. I think this will end up being a good post in general for others trying to do something similar and just needs a useful example of what it would look like.

DavidHourani · ‎03-31-2020

You're most welcome ! For SQS on AWS there will be no duplication whatsoever it works like a charm. As for dbconnect, yes you have many options for de-duplication you can chose what you like ^^ Feel free to up-vote and accept if this was helpful ! Cheers, David

cdemir · ‎05-04-2020

Using custom alert actions in ITSI is a little different than in ES. If you wish to use a custom alert action for an event generated by a correlation search, you have to set it up as an action rule on an aggregation policy. You also have to configure the custom alert action in a conf file to be selectable as an action against an event. Parameters to invoke the action are configured as part of the policy. There’s not a whole lot of documentation on this, but there is some to help you get started on docs. Edit*** I would also like to add that you could technically just use your custom alert action from ES on ITSI events and episodes. Episode information is indexed in itsi_grouped_alerts and notable events are indexed in itsi_tracked_alerts. If the SHC with itsi installed is running from the same indexer cluster as your deployment of ES, you can search the itsi internal indexes from ES as well.

teodor1746 · ‎03-29-2020

Thank you for the advice, I probably will as this more for me to learn and kind of be a proof of concept if it works. I have one working RPi now, but I plan to add 2 more and would like to control them from the splunk server if possible.

vrmandadi · ‎03-25-2020

This worked. [ alt ] CHARSET=UTF-8 INDEXED_EXTRACTIONS=JSON KV_MODE=JSON LINE_BREAKER=([\r\n]+) MAX_TIMESTAMP_LOOKAHEAD=60 NO_BINARY_CHECK=true SHOULD_LINEMERGE=true TRUNCATE=10000 category=Structured description=JavaScript Object Notation format. For more information, visit http://json.org/ disabled=false pulldown_type=true TIME_PREFIX=(Date()\d+)\\/\"}]

manjunathmeti · ‎03-05-2020

hi @maria_n, Try this: index=kohls_prod_infrastructure_openshift_raw kubernetes.container_name=sign-template-services | rex field=message "-\s(?<Response_Time>[\d\.]+)" Sample query: | makeresults | eval message = "::ffff:100.65.19.1 - - [05-Mar-2020 09:58:48 CST] \"GET /health HTTP/1.1\" 200 30 - 7.807 ms\n" | rex field=message "-\s(?<Response_Time>[\d\.]+)"

manjunathmeti · ‎03-05-2020

Hi @harishalipaka, Try this query: | makemv delim="," Drive | makemv delim="," Free_Space | eval fields = mvzip(Drive,Free_Space) | rex field=fields "(?<Drive1>[C]+):,(?<Free_Space1>[\d]+\sGB)" | rex field=fields "(?<Drive2>[D]+):,(?<Free_Space2>[\d]+\sGB)" | table Drive1, Drive2, Free_Space1, Free_Space2 Sample query: | makeresults | eval _raw="Drive Free_Space C:,D: 500 GB,450 GB E:,D: 250 GB,150 GB C:,E: 250 GB,1 TB S:,D:,C:,G 120 GB,450 GB,250 GB,800 GB" | multikv forceheader=1 | makemv delim="," Drive | makemv delim="," Free_Space | eval fields = mvzip(Drive,Free_Space) | rex field=fields "(?[C]+):,(?[\d]+\sGB)" | rex field=fields "(?[D]+):,(?[\d]+\sGB)" | table Drive1, Drive2, Free_Space1, Free_Space2

darrenfuller · ‎02-26-2020

Hi Saurabh0912, No, Changing your license pool will not cause any data to be deleted. What you may find, if the new pools are not set up correctly and your groups of indexers start to exceed the volume assigned to them, is that you could see new license exceeded warnings. Even that message is saying that all of the Indexers in the pool you are deleting, will lose the shared license volumes...not their data volumes. As long as, at the end of your exercise, every one of your indexers are in license pools that cover the intake volume of them and their license pool brethren, you should be good to go.

darrenfuller · ‎02-26-2020

Hi newbie2tech, You need to create a mongo formatted query (since kvstore is a mongodb at its core, albeit a modified mongodb, so you can't just point a mongodb tool at it to manage it.) Like so, if you have a query to look at your kvstore like so: |inputlookup my_kv_store where LastUpdateTime<1551139200 in Mongodb query format that would look like so: {"LastUpdateTime": {"$lt": 1551139200}} Next, you need to take that and url encode it, using a tool like: http://meyerweb.com/eric/tools/dencoder/ which turns this: {"LastUpdateTime": {"$lt": 1551139200}} Into this: %7B%22LastUpdateTime%22%3A%20%7B%22%24lt%22%3A%201551139200%7D%7D Lastly, run the following from a command line that supports curl, and can access your Splunk search head: curl -k -u admin:changeme -X DELETE https://splunkhost:8089/servicesNS/nobody/<appname_where_kvstore_is_defined>/storage/collections/data/<kv_store_name>?query=%7B%22LastUpdateTime%22%3A%20%7B%22%24lt%22%3A%201551139200%7D%7D Making sure you replace the and placeholders with the actual values relevant to your environment and pasting your mongo query after the query= point in the command. Last point, measure twice, cut once.. Maybe create a test kvstore, put some dummy data in it and test this before you run it against your 3 years kvstore data and accidentally blow it all away. A backup of the kvstore might also be in order before this. (Call me paranoid). Good luck. Darren

sbarreig · ‎01-17-2020

Hello, In have Splunk Enterprise as the Indexer and the Search Header, I´m triyng to collect performance data from a linux server so I installed in the server the Universal Forwarder and the Splunk_TA_nix Add On. I did the basics configuration to inputs.conf but at the end I didt collect the performance data like lsoft,top,nstats,etc. How did you achive to collect netstat,top,ps? Thanks

niketn · ‎12-22-2019

@darrenfuller you would need additional escaping in Splunk for " and \ while defining the regular expression. Following is a test you can run in Splunk SPL to validate the REGEX: | makeresults | eval _raw=" 2020-02-20 20:22:02.202020 test:>\"value\" test1:>\"value1\" test2:>\"this is a \\\"test\\\"\" test3:>\"this is \\\"a test\\\" ok\" test4:>\"this is \\\"another\\\" test ok?\"" | rex "(?<field>[\w\_]+)\:\>\"(?<value>(?:[^\"\\\]|\\\.)*)\"" max_match=0 If the above works as per your expectation, you would need to use following REGEX in transforms.conf ([\w\_]+)\:\>\"((?:[^\"\\\]|\\\.)*)\" PS: If you pass the regex101 regular expression to Splunk SPL I have provided above, you can notice that it is unable to parse the regular expression as quote and backslash need additional escaping in Splunk.

woodcock · ‎11-29-2019

You should be able modify the time at the very end of the processing with INGEST_EVAL like this: [appropriate stanza header here] INGEST_EVAL _time = _indextime

darrenfuller · ‎11-28-2019

The minimum required capabilities to run the debug | refresh are: edit_scripted refresh_application_licenses web_debug (If you have just web_debug, it will allow you to see the refresh button, but when you press it, it throws a gnarly 5XX error, with these two others, everything runs fine.)

Posts	111
Solutions	18
Karma Given	139
Karma Received	28
Member Since	‎07-20-2014

Online Status	Offline
Date Last Visited	‎03-23-2021 05:42 PM

Windows monitored file duplicate data

Regex / Transforms issue.

Non-Padded Milliseconds on Timestamp

Re: Ingest only rows containing certain text from ...

Re: How do I select first and second match as sepa...

Windows monitored file duplicate data

Re: inputs.conf `monitor` (re)ingest historic data

Re: How to search custom_data CSV files across sev...

Re: Issue sending events to nullQueue.

Re: Raspberry Pi Universal Forwarder Bug Report fo...

Re: How to check if a time field is between two ho...

Re: How can you rewrite log data to overwrite sens...

Re: size and timestamp of lookup files

Re: Setting up an alert in splunk to notify user ...

Re: HEC (HTTP Event Collector) host rename using p...

Re: Creating Custom Field Extractions

Re: Heavy Forwarder Redundancy (with DB Connect, A...

Re: Custom Alert Action in ITSI

Re: Can a Raspberry Pi that acts a universal forwa...

Re: Help with props.conf for timestamp

Re: How to extract a field with regex

Re: split specific values from list

Re: Does deleting or modifying license pool cause ...

Re: Deleting Data from KV Store

Re: *Nix add-on with official universal forwarder ...

Re: Regex / Transforms issue.

Re: how to set timestamp format for each event in ...

Re: refresh for sub set of admin role