About tyronetv

tyronetv · ‎01-27-2014

Within my event data I have a file name for a data set that we move around between services. Input files are sent in a zip file named "< env >.< app >.< client >< site >.< date >.zip". Where: < env > is the environment as "test", "qa", or "prod" < app > is the appcode in \w\w\w\d\d\d format < client > is the 3-digit client number < site > is the 2-digit site code < date > is the 3-digit julian date plus "01" for AM, or "02" for PM Example: test.abc123.51720.02701.zip I use the entirety of the file name as my 'source file' (SFIL) to track it through the three systems that touch/move it. What I would like to do is ALSO track by or which are parts of the previously define source file (SFIL). Suggestions?

tyronetv · ‎01-23-2014

According to http://wiki.splunk.com/Deploy:DeploymentServer it is $SPLUNK_HOME/bin/splunk reload deploy-server There is other information there but I think that answers you question.

tyronetv · ‎11-18-2013

sourcetype=vshell PNUM=27640 |transaction PNUM | reverse |table _time PNUM action loginid clientip SFIL |search action=write OR action=read [vshell] EXTRACT-vshell_sar = (?i) (?P [^ ]+) opened (?P [^ ]+) for (?P \w+)\s(?P \d+) bytes transferred EXTRACT-vshell_pnum = ^\w+\s+\d+\s\d+:\d+:\d+\s+\w+-\w+\s\w+\[(?P \d+)\] EXTRACT-vshell_SNOD = Connection accepted from (?P \d+\.\d+\.\d+\.\d+):\d+

tyronetv · ‎11-18-2013

To get the remote host (clientip) I need to gather from the first line (based upon vshelld[27640]). Nov 17 00:02:25 HOST vshelld[27640]: [ID 681337 local3.notice] conn,2011830: Connection accepted from 167.117.16.147:3098. That's why I think I need to use a transaction.

tyronetv · ‎11-18-2013

I have an FTP log (using VSHELL from http://www.vandyke.com/) wherein each user session gets a unique ID but the user can (and often does) send multiple files. I am trying to write a report that shows time, ID, login, IP, file, size and for the most part it works, unless the "trigger" file is the same byte count and then the report is a bit wonkey.... Essentially, the transfer line looks like: Nov 17 00:02:27 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p created directory /DIR/. Nov 17 00:02:39 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000010692381.PDF for write 940646 bytes transferred. Nov 17 00:02:40 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000010692381.tkt for write 910 bytes transferred. Nov 17 00:02:52 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000020888169.PDF for write 1032185 bytes transferred. Nov 17 00:02:52 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000020888169.tkt for write 910 bytes transferred. Nov 17 00:03:04 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000020891502.PDF for write 1045536 bytes transferred. Nov 17 00:03:04 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/000020891502.tkt for write 910 bytes transferred. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/110005569971.PDF for write 933715 bytes transferred. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: kai101p opened /DIR/110005569971.tkt for write 910 bytes transferred. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] dbg ,2011830: [LOCAL DEBUG] RECV: CHANNEL_OPEN[session] Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] conn,2011830: Session channel open request accepted. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: Sftp subsystem initialized; remote version is 3. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: Sending VERSION packet to remote (3) Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: Sftp subsystem terminated. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] conn,2011830: Session channel has been closed (pid: none). Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: Sftp subsystem terminated. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] conn,2011830: Session channel has been closed (pid: none). Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] sftp,2011830: Sftp subsystem terminated. Nov 17 00:03:13 HOST vshelld[27640]: [ID 681337 local3.notice] conn,2011830: Session channel has been closed (pid: none). I wrote a rex to pull out the file and bytes (910 above) but when there are, multiple transfers and each file includes a 'trigger' file of (in the case of above) 910 bytes the report is inaccurate. I do a transaction based upon the vshelld[27946], which includes all the steps from connection to disconnect and multiple transfer lines (as above) so I can pull the credentialed user's name as well as remote host, etc., and when I attempt to report on time, user, file, size it generates one line per the vshelld[27640] and then lists all the files names but only lists 910 one time regardless of how many trigger files. I would think this is some sort of sub-transaction but I'm not sure how to do it and get a proper grouping...i.e. _time, file & trigger (they have the same timestamp in the log), login, clientip, etc. ,etc. Help? I hope this is enough information to get the conversation started and hopefully get somewhere closer to the answer I need.

tyronetv · ‎11-18-2013

1) Splunk is showing me the 'offset' time based upon my TZ setting. 2) For some reason, for this source type ONLY, splunk is showing the AM/PM declaration. Not sure why.

tyronetv · ‎11-18-2013

Sample log line date part: Nov 16 22:48:36 props.conf on indexer TIME_PREFIX = ^ TIME_FORMAT = %b %e %H:%M:%S MAX_TIMESTAMP_LOOKAHEAD = 15 Time reported in Splunk - 11/17/13 12:48:36.000 AM I use "%e" because the day field is populated without leading zero's. What am I doing wrong?

tyronetv · ‎11-12-2013

The awk statement is fine and almost a mirror of what I've already done. I am looking for something that essentially mimics the expansion of the entire monitor stanza to include file names identified by the white/black lists as well as the monitor line.

tyronetv · ‎11-12-2013

Does anyone know of a tool that will 'expand' the monitor stanza from inputs.conf on a universalforwarder to show an example of logs to be watched? I.e., I have a monitor stanza: [monitor:///path/to/some/*/dir] whitelist = /file_name(s).log$ And before I restart splunk and do the 'hope it works' I was wondering if there was a tool that would, using Splunk's logic, show me all the files the above would 'see' for monitoring. I have multiple 'client' directories (being replaced above by the *) where some have specific logs and some do not. I would rather write one monitor for each type of log verses writing a new monitor stanza per client dir/log type. And I need to test it before pulling the trigger and not impact other, already configured, data-gathering.

tyronetv · ‎11-06-2013

My original search uses automatic lookups. Hence SAccount and Account vs PNOD and SNOD

tyronetv · ‎11-06-2013

Here's my current search generating a report using my two lookup tables. Luckily C:D logs are all key/value pairs. sourcetype=connectdirect host = HOST_NAME ( RECI=CTRC ) | transaction keepevicted=true PNUM RECI STAR STOP | eval VOLUME=(SBYX/1048576)| search VOLUME>0 | table STAR PNUM SAccount Account SFIL DFIL Are you saying I can do : table STAR PNUM PNOD SNOD SFIL DFIL |lookup nodes.csv Node AS PNOD OUTPUT Account | lookup nodes.csv Node AS SNOD OUTPUT Account AS SAccount ?

tyronetv · ‎11-06-2013

I have five machines providing the C:D service. Some provide service over private circuits and one provides 'public' access. So the PNOD can be one of my five machines with the SNOD being one of 145 clients OR PNOD can be someone in a subset of my clients with SNOD being one of my five machines. I am generating reports per-machine currently but my long term goal is one report that shows the file movement within the organization.

tyronetv · ‎11-06-2013

My company uses ConnectDirect (C:D) as a tool for file transfer. Within the connect direct logs the hosts are referred to by the variables PNOD and SNOD where PNOD is the primary node and SNOD is the secondary node. Because C:D can either receive or send a host can be either PNOD or SNOD. The hosts are configured in a file such that: nodename Verbose_name ACCT1 "My Client's LA Site" ACCT2 "Internal PHX Site" etc... Therefore PNOD can be one of the local machine ID or the remote machine and the same for SNOD. The assumption is that when the SNOD is the local machine it is inbound traffic. When the PNOD is the local machine it is outbound traffic. Now, my question: Is it possible to have one lookup file that works for both SNOD and PNOD providing the Verbose name in reports. I.e -- File_1 Received from My Clients LA Site File_2 Sent to My Clients LA Site The only difference is in the first one it is PNOD and in the second one it is SNOD. Is this easy and I am just over looking it? Currently I have two lookup files, exactly the same, except for one is headered "SNOD,Account" and the second is headered "PDOD,SAccount".

tyronetv · ‎11-06-2013

When initially set up my splunk install is set to capture only the most recent version of a log: /path/to/log/dir/logfile. Well, sometimes, due to maintenance, etc., Splunk is shut off and when restarted I have to go through the process of reloading (via oneshot) the data from logfile.1 logfile.2 logfile.3 . . . to get caught up. On systems I've set up, I just have inputs.conf configured for /path/to/log/dir with a whitelist on logfile*$ What will happen if I change the previous configuration? Will it re-index all the previous logfile.\d+$ files or is it self-aware enough to not do that?

tyronetv · ‎10-22-2013

Okay. I'm a klutz. The lookup file and lookup definition were set properly but when I added the automatic lookup definition I missed resetting the permissions to all. My bad.

tyronetv · ‎10-22-2013

I have a search that works against a key value pair in Connect Direct logs. Initially I had the report set up to do post-process re-naming of the SNOD value to the human readable name. Yesterday I added a lookup table and ran the search multiple times with the expected results, i.e., the lookup worked. I replaced the search in the Dashboard panel with the new search and was told "zero results." I ran the inspector and it stated: This search has completed and found 2,764 matching events. However, the transforming commands in the highlighted portion of the following search: search sourcetype=connectdirect host = napa-v120 ( RECI=CTRC ) | transaction keepevicted=true PNUM RECI STAR STOP | eval VOLUME=(SBYX/1048576) | search VOLUME>0 | timechart span=30m sum(VOLUME) by Account useother=f usenull=f | rename NAPA_CD_V120 as "Inbound" (bold to show what was highlighted). If I just run the search, everything works. I can change date/time ranges, etc., with no problems. Why is the display as a dashboard not working? I have removed the "rename ..." to no effect. If I go back to "timechart span=30m sum(VOLUME) by SNOD ..." and then do post process renames it works though.

tyronetv · ‎10-02-2013

I saw this here at .conf and the issue for me was that another app (specifically the mobile app) was subsuming the login.css. We renamed it and did a refresh and it my issue was resolved. I subsequently removed the mobile app. Make sure that some other app you have installed is not taking over for the new core services/html/css.

tyronetv · ‎08-21-2013

transaction RID startswith="DAO CALL START" endswith=" Connection" Yeah... I should of figured that out sooner. Thanks for the assist though.

tyronetv · ‎08-19-2013

Doesn't work. Been there done that. 😞 As you can see from my example, there are multiple connections to the DB under one RID. I need each one encapsulated and that search didn't do it for me.

tyronetv · ‎08-19-2013

AFAICT from the logs, and because I am doing a transaction based upon the RID, the connect is just the next sequential step after the DAO START.

tyronetv · ‎08-19-2013

I have a set of two logs that share a common field (RID). One log contains the "user" actions while the other log contains the java, odbc, etc., actions (ibatis log). What I need to do, is figure out if there is a delay between when a user makes a request and the actual connection to the database. 2013-08-19 06:40:32,904 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning - <DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.getTransactionDetails {INPUT PARAMS} {transactionid=2404611298, callinguserid=-5710326788253269740} 2013-08-19 06:40:32,908 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-473946} Connection ... some other actions ..... 2013-08-19 06:40:33,332 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning - <DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.lockTransaction {INPUT PARAMS} {transactionid=2404611298, latestchangeid=449327573, callinguserid=-5710326788253269740} 2013-08-19 06:40:33,333 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-473948} Connection ...some other actions.... 2013-08-19 06:45:06,130 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning - <DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.getCouponFieldMappings {INPUT PARAMS} {documentTypeId=2642998, callinguserid=-5710326788253269740} 2013-08-19 06:45:06,130 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-486974} Connection ... and other various versions of this.... 2013-08-19 06:40:32,904 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning - <DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.getTransactionDetails {INPUT PARAMS} {transactionid=2404611298, callinguserid=-5710326788253269740} 2013-08-19 06:40:32,908 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-473946} Connection ... some other actions ..... 2013-08-19 06:40:33,332 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning - <DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.lockTransaction {INPUT PARAMS} {transactionid=2404611298, latestchangeid=449327573, callinguserid=-5710326788253269740} 2013-08-19 06:40:33,333 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-473948} Connection ...some other actions.... 2013-08-19 06:45:06,130 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 c.r.t.r.o.service.OnlineDecisioning -<DAO CALL START> {USER ID} null {DAO PROCEDURE} OnlineDecisioning.getCouponFieldMappings {INPUT PARAMS} {documentTypeId=2642998, callinguserid=-5710326788253269740} 2013-08-19 06:45:06,130 DEBUG [andler-RMICallHandler-192] RID=1376919632894-63948 java.sql.Connection - {conn-486974} Connection ... and other various versions of this.... As you can see, the connection is VERY fast. Unfortunately, sometimes, it is not. When it is NOT we have a problem that is about ready to increase exponentially and cause some pain for our customers and, well, me. What I want to do is put in place a 'real time' monitor that says the time between the line with <DAO CALL START> in a sourcetype of t2_app and the java.sql.Connection in the sourcetype of ibatis cant exceed X seconds or milliseconds. (And, also to be able to graph normal response times.) I can do a <DAO START> to a <DAO COMPLETE> but that is too much. Just need to know from start to DB connect. Suggestions? Thanks!

tyronetv · ‎03-04-2013

This was the fix. I had used case before (another question) but was suggested I use rangemap. For this search using case is the clear winner. Thanks.

tyronetv · ‎03-01-2013

2013-03-01 12:35:28,878 INFO [ler-HTTPThreadGroup-17042] RID=1362170128682-2299470 c.r.t.i.s.e.applicationImageArchiveConnection - application response time was 138 milliseconds. 2013-03-01 14:35:22,040[ndler-HTTPThreadGroup-681] INFO dis.service.application.ImageArchiveConnection - [1] application response time was 128 milliseconds. 2013-03-01 12:35:21,950 INFO [ler-HTTPThreadGroup-17053] RID=1362170121771-2299465 c.r.t.i.s.e.applicationImageArchiveConnection - application response time was 124 milliseconds.

tyronetv · ‎03-01-2013

Maybe I wasn't clear. The default bucket has 20,000+ count that do not exist in the search. Read my initial post. Search pipe to count gives 50K+ results. Search pipe to rangemap gives 70K+ results.

tyronetv · ‎03-01-2013

They have "no value" that I can find. There should be no counts in the default bucket. But, to more specifically answer, if you look at the query, I believe the default should include values > 1000

Posts	66
Solutions	4
Karma Given	5
Karma Received	23
Member Since	‎09-07-2011

Online Status	Offline
Date Last Visited	‎08-11-2020 12:50 PM

Splunk 8.0 upgrade has no web server running

How to edit my search to reflect stats related the...

Why is the Splunk Add-on for Unix and Linux consum...

Formatting a multi column report.

email not sending from windows search head

File system changes (not log monitoring)

Repeating Field Extraction

source type identification in props.conf

regex difficulty with host / path discrimination

Finding an Error in the middle of a Transaction

Fields within fields - search time extract

Re: Deployment Server - reload configs without res...

Re: Transaction sub process stats

Re: Transaction sub process stats

Transaction sub process stats

Re: Time Format being ignored, why?

Time Format being ignored, why?

Re: Expand inputs.conf with wildcards

Expand inputs.conf with wildcards

Re: Lookup table with two primary keys

Re: Lookup table with two primary keys

Re: Lookup table with two primary keys

Lookup table with two primary keys

Changing inputs.conf to include previously scanned...

Re: Dashboard shows no Results but Search works . ...

Dashboard shows no Results but Search works . .

Re: Upgraded to 6 and the login page for splunk-we...

Re: Transactions within transactions

Re: Transactions within transactions

Re: Transactions within transactions

Transactions within transactions

Re: rangemap with too many values

Re: rangemap with too many values

Re: rangemap with too many values

Re: rangemap with too many values