About timrich66

timrich66 · ‎06-22-2021

hi @kamlesh_vaghela afraid I've tried that and it doesn't return anything

timrich66 · ‎06-22-2021

Hi, I'm sure I'm not the first to ask this question, but I can't seem to find an answer that covers what I am trying to achieve. I have an index which collects job stats - start, end, fail, success etc What I would like to do is create a table to display all the jobs I am interested in in one column, then the start, end and run times and a status column. Like this - Column A Column B Column C Column D Column E Jobname Start Time End Time Run Time Status abc 08:00 08:01 1 Success The search below gives me everything EXCEPT I cannot calculate 'Run Time' because the events are separate. I've tried with 'streamstats' and 'transaction' without any success. index=foo sourcetype=bar_prd "p-foo*" earliest=-6h | rex "JOB: (?<j>p-foo-[a-z\-]+)" | rex "STATUS: (?<s>\w+)\s" | eval ST=if(s="RUNNING",_time,"") | eval ET=if(s="SUCCESS",_time,"") | eval Status=if(s="SUCCESS","Success","") | eval ST=strftime(ST,"%Y-%m-%d %H:%M:%S.%Q") | eval ET=strftime(ET,"%Y-%m-%d %H:%M:%S.%Q") | stats values(ST) as "Start Time", values(ET) as "End Time", values(Status) by j As ever, I'd be very grateful for assistance.

timrich66 · ‎05-20-2021

Yes, you're right, thanks. It was the 'chart' rather than 'timechart' that made the real difference. All working as planned.

timrich66 · ‎05-19-2021

Your suggestion nearly worked, but not quite. I have taken your search and modified it as follows - index=foo ... | rex "Started=\"\d{4}-\d\d-\d\d (?<x>\d{2})" | eval s=strptime(TimeStarted, "%Y-%m-%d %H:%M:%S.%Q") | eval e=strptime(TimeCompleted, "%Y-%m-%d %H:%M:%S.%Q") | eval r=(e - s) | streamstats count as index by _time JobName | eventstats count as count by _time JobName | eval JobName=case(JobName!="BIDOFF",JobName,JobName="BIDOFF" AND x==12,JobName." - midday",JobName="BIDOFF" AND x==13,JobName." - 1pm") | chart values(r) as "Run Time" by _time JobName This works out 'r' (Run Time) and 'x' (the hour the job runs). Then, using a case statement, determines the JobName field based on the hour and the jobname. For job "BIDOFF' the supplemental time based comments are added. For all other jobnames, the name remains the same. Thank you very much for this. Every day is a learning day 🙂

timrich66 · ‎05-19-2021

Thank you 🙂 I now have a line. What I need to do now is find out how to make it separate lines for each run. To explain - each day the BIDOFF job runs at 12:00 and 13:00. Ideally, I would like a separate result for each run which then gives a separate line on the timechart.

timrich66 · ‎05-19-2021

timrich66 · ‎05-19-2021

@ITWhisperer - thanks - I should have made it clear that I wish to chart both values. The options sum(), max() and avg() don't work for me. I've also tried "mvexpand JobName" but this doesn't help. If I can only chart one, the first run at midday is the most important.

timrich66 · ‎05-19-2021

I've been searching and trying options for a couple of days now with this search and cannot find a solution. I am using DB Connect to interrogate a database to get events that show me the start and end times for a suite of jobs. This works fine. Each day I have a single event detailing start and end time for each job. However, one of the jobs runs twice. I am trying to create a timechart showing the run time for each job. The job that runs twice gives two results for one day so when the timechart runs, the stats are there, but not the visualisation. I'm sure there must be a simple solution to this, but I can't work it out. Is there a way to get the results to show in a timechart? Thanks in advance My search - index=foo sourcetype=bar JobName="BIDOFF" earliest=-7d@d latest=-0d@d+7h | eval s=strptime(TimeStarted, "%Y-%m-%d %H:%M:%S.%Q") | eval e=strptime(TimeCompleted, "%Y-%m-%d %H:%M:%S.%Q") | eval r=(e - s) | timechart values(r) by JobName Results - _time BIDOFF 2021-05-12 32.940000 33.000000 2021-05-13 33.013000 33.034000 2021-05-14 32.907000 33.110000 2021-05-15 2021-05-16 2021-05-17 32.936000 33.030000 2021-05-18 33.077000 34.547000

timrich66 · ‎02-22-2021

Hello All, I have an alert that has an Action to create a SNow Incident when a file has not been received by a certain date. The first time the alert ran and the action was triggered, a new Incident was created. Subsequently, the original Incident has been reopened. This is not what I want. I need to have a new Incident for each time the action is triggered. I have tried setting 'state' to 1, which implies New in the documentation here - https://docs.splunk.com/Documentation/AddOns/released/ServiceNow/Commandsandscripts state No Number 4 The state of the incident. For example: 1 - New 2 - Active 3 - Awaiting Problem 4 - Awaiting User Info 5 - Awaiting Evidence 6 - Resolved 7 - Closed I have also left the Correlation ID blank so that a new (unique) ID is created. Correlation ID A unique ID to support third-party application integration. It should only consist of alphanumeric characters, underscore (_), and hyphen (-) in its value. Leave blank for the Splunk Add-on for ServiceNow to generate a unique ID for you. Thanks in advance for support

timrich66 · ‎02-11-2021

Hi, @isoutamo Can you explain your thinking? It is unlikely, but may be possible. I don't understand why the nullqueue option isn't working though.

timrich66 · ‎02-11-2021

Hi @scelikok , yes, the code is deployed from our Cluster Master (/opt/splunk/etc/master-apps/_cluster/local) to indexers (/opt/splunk/etc/slave-apps/_cluster/local). It is also present on HF in /opt/splunk/etc/system/local).

timrich66 · ‎02-03-2021

Hello All, Grateful for assistance on this one. We have several areas where servers are HA pairs and write to a server specific log. However, because they are an HA pair, their own log and the equivalent log on the paired server is visible via a shared drive. Thus, server 'A' produces 'serverlogA' but can also see 'serverlogB'. Server 'B' produces 'serverlogB' and can also see 'serverlogA'. Because both servers are in the same Server Class, we end up with duplicated events from both server logs. We cannot only ingest from one server because they also have unique log files and a failure on the ingesting server would require manual intervention to move to the paired server. I have tried to nullqueue the events as shown below, but not had any success. Please let me know your thoughts on how to work around this issue. Thanks props.conf [source::/apps/lvservices/mnt/logs/abc/whyluaap182_ContractEnquiry.log] TRANSFORMS-nullq_cms_uaap181 = nullq_uaap181 [source::/apps/lvservices/mnt/logs/abc/whyluaap181_ContractEnquiry.log] TRANSFORMS-nullq_cms_uaap182 = nullq_uaap182 transforms.conf [nullq_uaap181] SOURCE_KEY = MetaData:Host REGEX = whyluaap181 DEST_KEY = queue FORMAT = nullQueue [nullq_uaap182] SOURCE_KEY = MetaData:Host REGEX = whyluaap182 DEST_KEY = queue FORMAT = nullQueue

timrich66 · ‎01-15-2021

Hello All, FYI - I have found the issue. Deliberate mistake? Maybe not, but it should have been obvious.. Original props.conf - [source::///xxxx/Log/xxxxServer.log] SHOULD_LINEMERGE=true MAX_EVENTS=10000 TIME_PREFIX=\+\+\+\+ \w+ Working props.conf [source::///xxxx/Log/xxxxServer.log] SHOULD_LINEMERGE = true MAX_EVENTS = 10000 TIME_PREFIX = \+\+\+\+ \w+ Yes, it was pesky spaces 🙂 Watch out for them! All now working as planned.

timrich66 · ‎01-07-2021

@richgalloway yes, props.conf is pushed from CM to indexers The reason that the sourcetype is being set based on host name is because the sourcetype includes the environment - e.g. dev1, dev2, prod1, prod2 etc The source file has the same path and name on all servers. The consumers of the logs do not necessarily know which hosts make up which environment. Therefore, by including the environment in the sourcetype, the users can find their data more easily.

timrich66 · ‎01-07-2021

Hello helpful people, I'm afraid I have an issue that is related to many questions already asked, but I have not been able to come up with a solution. I have a log file that creates large events - more than 257 lines at a time. To test the file, I took an extract and uploaded it manually. Using this file, I was able to create props.conf entry as shown below and the events ingested correctly, without breaking. When I applied this to our clustered environment, the breaking has returned. Events - ++++ information 2021-01-06 16:38:53 host = xxxx.xxxx.net process = 00002fa8 thread = 73ffe380 context = Server::calculate(), module Request failed with error(s): <?xml version='1.0'?> Show all 257 lines [031004] Variable has no value. [035006] Cannot have child <xxxxx[E.3] (B6I2)> (xxx) on link xxxxxxxxxxxxxxxxxxx (B6I1)> </clc:Error> </xxxx__xxxxx_xxxx_xxx_f123_2> Show all 257 lines props.conf [source::///xxxx/Log/xxxxServer.log] SHOULD_LINEMERGE=true MAX_EVENTS=10000 TIME_PREFIX=\+\+\+\+ \w+ The reason I am using source and not sourcetype is because this source file is common to a number of environments and I am already changing sourcetype using props and transforms to determine the sourcetype per servername. Thanks in advance for help - much appreciated.

timrich66 · ‎09-24-2020

This issue was caused by the setting 'indexAndForward' in outputs.conf causing events to be dropped when the queues were too busy. The 'indexAndForward' stanza was set to false and the events are now all available via the SH.

timrich66 · ‎09-16-2020

Hi @isoutamo Yes, $splunkhome/system/local/outputs.conf points to indexers. indexers and SHs are clustered

timrich66 · ‎09-16-2020

This issue is primarily related to events ingested via the IMAP Mailbox App We are running a distributed environment with a HF, 3x indexer and 3x search head (accessed via a VIP). The install has been carried out as per the README.txt instructions for a distributed environment. Some events are only appearing when searched for on the HF. They do not appear when searched for on the SH's. The results are mixed in that some email events do not appear at all on the SH's and some events may or may not appear. That is a search on HF returns 11 events. The same search on SH returns 8 events. As always, thanks very much for assistance.

timrich66 · ‎09-14-2020

Hi, I am getting the same error on a unix box. Did you ever come to a conclusion? The input works fine for exactly the same path on a different server.

timrich66 · ‎09-11-2020

Thanks for the link. It makes interesting reading. I have asked our unix SA to add debugging to the UF so I can get more details. Events from different files on the server in question are being ingested and the same file name on a different server is also providing events.

timrich66 · ‎09-11-2020

In our non-prod environment, some files are not written to on a regular basis. In these cases the UF often needs to be restarted to start the ingestion of events. The monitor stanza in use is very basic and does not have an 'ignoreOlderThan' value set. Here is an example - [monitor:///my_non_prod_path/Log/app.log] index=my_non_prod_index sourcetype = nonprd_sourcetype disabled=false Are there global variables that would override this and cause events to not be ingested? Thanks

timrich66 · ‎09-02-2020

The file is tiny. I will look at what other options are available. Thanks

timrich66 · ‎09-02-2020

Thanks for the reply. I am, however, still confused. There are a number of Questions about how to ingest with the correct format - eg https://community.splunk.com/t5/Archive/How-to-ingest-doc-format-file-into-splunk-with-correct-format/td-p/488474 As I have stated, I am not concerned with the format within the doc, only the filename is of importance.

timrich66 · ‎08-26-2020

Hello all, My latest challenge is to ingest a Word doc into our environment. According to everything I have read so far, this should be straight forward as Splunk can ingest 'any' file. At this point I should point out that I am not concerned about the contents of the file (as this all needs to be obfuscated). I only need to ingest the file to get its name. I am not concerned about whether or not Splunk can read the 'Word' type formatting. The file is created daily with the format - "My Word Doc ddmmyyyy hh mm.doc" I am only interested in the "ddmmyyyy hh mm" part to ensure that it has been created today. I cannot get the doc file to ingest at all. Not even in an unformatted state. If I save the file as a ".txt" file, then it is ingested. Unfortunately, the 'save as' option is not an option in production. I have tried using 'whitelist=' option without any success. Can anyone suggest a solution? Is there something in my installation that is stopping Word docs from being ingested? Has anyone else had a similar experience? Thanks

timrich66 · ‎08-18-2020

I appear to have fixed this. I have changed the monitor path to read "*.*.finish" and the source name has remained unchanged since.

Posts	59
Solutions	3
Karma Given	6
Karma Received	2
Member Since	‎11-27-2017

Online Status	Offline
Date Last Visited	‎05-30-2022 04:43 AM

How to create dashboard with trend line to show pe...

Why does this search not sort

Run two saved searches from one dropdown

Accessing restricted Windows share

"Inner Join" works in query, not DB Connect

Replace letters with numbers

Calculate time server CPU above alert percentage

The curious case of missing events

Calculate elapsed time from two events

Timechart woes

Re: Calculate elapsed time from two events

Calculate elapsed time from two events

Re: Timechart woes

Re: Timechart woes

Re: Timechart woes

Re: Timechart woes

Re: Timechart woes

Timechart woes

Alert to create NEW ServiceNow Incident

Re: Nullqueue filtering

Re: Nullqueue filtering

Nullqueue filtering

Line Merge difficulties

Re: Line Merge difficulties

Line Merge difficulties

Re: Events on Heavy Forwarder not available on Sea...

Re: Events on Heavy Forwarder not available on Sea...

Events on Heavy Forwarder not available on Search ...

Re: Skipping itemPath, does not match path....

Re: Why does Splunk 'lose interest' in files that ...

Why does Splunk 'lose interest' in files that are ...

Re: Ingesting Word document

Re: Ingesting Word document

Ingesting Word document

Re: Sourcefile name changes at index time - interm...

Join the Conversation