About blaise

blaise · ‎01-24-2021

some of my data in index "main" shows as 2420 days old, yet my "Frozen Age" is set to 365 days, shouldn't the old data get deleted ? I am looking at this from the monitoring console, Index Detail: Instance, and the status line has this information: Data Age vs Frozen Age (days) 2420 / 360 I am using Splunk Entreprise 8.0.5 and my index "main" is defined as so: [main] repFactor = auto coldToFrozenDir = /opt/splunk_buckets/archive/main/frozendb thawedPath = /opt/splunk_buckets/archive/main/thaweddb # frozen time is 12 months frozenTimePeriodInSecs = 31104000

blaise · ‎11-25-2020

While getting Netflow data using streams, I aggregate a variable "bytes_in" as a sum of the bytes_in received in a flow, this works well, and I get a new variable called "sum(bytes_in)" The problem I am experiencing is this: Streams in the previous version, used to index the data for the aggregate "sum(bytes_in)" as "bytes_in". Now since I upgraded to Streams 7.2, Streams indexes the data as "sum(bytes_in)" And this is causing me a lot of issues when trying to get this data into a data model, every time I try to use this data, the SPL fails This works in a normal search: I use stats to get the max value of "sum(bytes_in)" and distinguish events by src_ip,dest_ip, etc index=streams | stats max(sum(bytes_in)) AS bytes_in by src_ip,dest_ip,dest_port,src_port,timestamp | where isnum(bytes_in) But When I try and import the streams data into a data model, the name of the variable "sum(bytes)" seems to be causing a lot of issues: error message: "Error in 'eval' command: The 'sum' function is unsupported or undefined." So I need to either: - from streams, rename the variable name "sum(bytes_in)" to "bytes_in for example, but I cannot find how to do this in streams OR - in the data model, rename the variable name "sum(bytes_in)" to "bytes_in for example, but I cannot seem to find how to do that as well

blaise · ‎05-20-2020

Sorry I realise this was asked 5 years ago 🙂 The answers were really useful for me, but for the purpose described in the question and to add a tool to your knowledge, I like using this command: | tstats count where host= by index,sourcetype,source Hope this helps

blaise · ‎04-25-2019

Hi willymac650, try this regular expression: (("[a-zA-Z0-9_-.]+@[a-zA-Z0-9_-.]+")+), the double brackets tells it to repeat the pattern matching, I am no expert, I just googled :"find repeat patterns in regex" and one of the pages explained this, I have tried it on regex101.com with your data and I am able to match multiple times. The only problem now is that I don't know how to name each match using $ Hope this helps Blaise

blaise · ‎12-04-2018

Hi Prakash, thank you , you are correct and that was my mistake, the props.conf definition needs to be on the indexers. Once I did that , it started working Thank you heaps for your help, it is appreciated 🙂 Blaise

blaise · ‎12-04-2018

Hello I have finally resolved the issue, the problem was I have a distributed environment ... so like Prakash suggested, the props.conf needs to be on the indexers, where the timestamp extraction is done. I have completely removed the props.conf from the universal forwarder server, where I only left the inputs.conf to define the inputs. Thank you for all your help Blaise

blaise · ‎12-04-2018

Hi Rich and Prakash I have tried both suggestions and it still is not working thank you both for your replies, you both suggested to use : TIME_FORMAT to %m/%d/%Y %H:%M:%S but my raw events timestamps shows as : [05/12/2018 10:32:03] text text ... where 05 is the day of the month %d and 12 is the month %m so the correct TIME FORMAT should be : %d/%m/%Y %H:%M:%S please explain why you suggested otherwise, I am getting really confused ... I am also wondering why all my attempts are failing, is it possible that another definition or config somewhere could take precedence over the app's props.conf ? Thank you again Blaise

blaise · ‎12-03-2018

Hi Rich I have tried your suggestion and it still is showing events for the 12th April, instead of the 4th Dec I tried those 2: TIME_PREFIX = '^[' TIME_PREFIX = '[' to confirm my props setttings on the universal forwarder, I found this great command: [splunk@bautoprod01 local]$ splunk cmd btool --app=autosys props list [autosys_events_prod] MAX_TIMESTAMP_LOOKAHEAD = 19 SHOULD_LINEMERGE = false TIME_FORMAT = %d/%m/%Y %H:%M:%S TIME_PREFIX = '[' [splunk@bautoprod01 local]$ pwd /opt/splunkforwarder/etc/apps/autosys/local [splunk@bautoprod01 local]$ So the above confirms that the settings are applied ("distributed"), but yet it still is not working Thank you for your help anyway Blaise

blaise · ‎12-03-2018

Hello I am trying to extract a timestamp from this type of events. Here, 04 is the day of month and 12 is the month, Dec on the search head, these events currently appear as 12th April [04/12/2018 10:16:04] CAUAJM_I_40245 EVENT: CHANGE_STATUS STATUS: SUCCESS JOB: esysprod_NOA_5_min_box [04/12/2018 10:26:03] CAUAJM_I_40245 EVENT: CHANGE_STATUS STATUS: SUCCESS JOB: esysprod_EX900 MACHINE: It looks pretty straightforward, but I cannot figure out what I am doing wrong. The source type for these events is called : "autosys_events_prod" So, I created a props.conf as below, and located it in the app that gets distributed from my deployment server: I also verify on the server where the log is created that the props.conf file is updated, and I also restart Splunk on the Universal Forwarder. [splunk@msplunkutil01 local]$ cat props.conf [autosys_events_prod] TIME_PREFIX = ^[ TIME_FORMAT = %d/%m/%Y %H:%M:%S SHOULD_LINEMERGE = false MAX_TIMESTAMP_LOOKAHEAD = 19 I have tried different time prefix(es) without success. How do I know if my props.conf is actually being used? Everything I have tried seems to have no effect so far. What is the best way to troubleshoot this ? Thank you for your help in advance. it is the first time I am trying to extract a timestamp from an event, so I might be doing something wrong. Blaise

blaise · ‎10-31-2018

I have tried it on regex101.com and I think this will help you: \s+[.]{5}"",".+?",".+?",(?".+?"),(?".+?"), it extracts both emails and creates two fields called "email1" and "email2" to contain the result of the match. \s+ one or more space [.]{5} 5 dots "", 2 double quotes characters, followed by a coma ".+?" 2 double quotes with anything inside, the ? is to make the match small (greedy?) , a coma ".+?", same as above again (?".+?") same as above but this time it has parentheses around, so that says that it needs to be saved, by default it would be saved into $1, but the ? part is actually naming the variable into which the matching part will be saved , a coma (?".+?") same as above but this time the variable is called email2 , a coma Hope this helps Blaise

Posts	10
Solutions	1
Karma Given	1
Karma Received	2
Member Since	‎06-14-2011

Online Status	Offline
Date Last Visited	3 weeks ago

index size , "Data Age vs Frozen Age" explanation

In streams, the aggregation of a variable sets the...

How do I extract a timestamp from an event with br...

index size , "Data Age vs Frozen Age" explanation

In streams, the aggregation of a variable sets the...

Re: How can I get a list of all hosts plus their s...

Re: How do I extract an email address from raw dat...

Re: How do I extract a timestamp from an event wit...

Re: How do I extract a timestamp from an event wit...

Re: How do I extract a timestamp from an event wit...

Re: How do I extract a timestamp from an event wit...

How do I extract a timestamp from an event with br...

Re: How do I extract an email address from raw dat...

Join the Conversation