Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About justinfranks
justinfranks
Path Finder
Member since:
09-25-2013
06-05-2020
Community Statistics
| Posts | 29 |
| Solutions | 1 |
| Karma Given | 4 |
| Karma Received | 6 |
| Member Since | 09-25-2013 |
Activity Feed
- Karma Re: Sub-Search in a join is truncating returned events at 50000 for starcher. 06-05-2020 12:48 AM
- Karma Re: How to use a date field that isn't _time to search HelpDesk tickets closed in the last 30 days? for kristian_kolb. 06-05-2020 12:47 AM
- Karma Re: How to find all results in an inputlookup and display zero if no events are matched? for HiroshiSatoh. 06-05-2020 12:47 AM
- Got Karma for How to use a date field that isn't _time to search HelpDesk tickets closed in the last 30 days?. 06-05-2020 12:47 AM
- Got Karma for Re: Splunk DB Connect: How to use $rising_column$ as an aliased column?. 06-05-2020 12:47 AM
- Got Karma for Re: Splunk DB Connect: How to use $rising_column$ as an aliased column?. 06-05-2020 12:47 AM
- Karma Re: Charting the 2 fields on the same chart. for lguinn2. 06-05-2020 12:46 AM
- Got Karma for Indexing Epoch times in DB Connect. 06-05-2020 12:46 AM
- Got Karma for Indexing Epoch times in DB Connect. 06-05-2020 12:46 AM
- Got Karma for Re: Indexing Epoch times in DB Connect. 06-05-2020 12:46 AM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-15-2017 04:34 PM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-15-2017 04:28 PM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-15-2017 03:08 PM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-14-2017 09:43 PM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-14-2017 09:42 PM
- Posted Re: Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-14-2017 09:41 PM
- Posted Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-13-2017 08:04 PM
- Tagged Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-13-2017 08:04 PM
- Tagged Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-13-2017 08:04 PM
- Tagged Sub-Search in a join is truncating returned events at 50000 on Splunk Search. 02-13-2017 08:04 PM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 2 | |||
| 0 | |||
| 0 | |||
| 0 |
02-15-2017
04:34 PM
Thanks for this! You are a life saver.. this appears to have worked perfectly.
... View more
02-15-2017
04:28 PM
Ahh you are correct. That fixed the operator issue (ie me) 😛
... View more
02-15-2017
03:08 PM
Thanks for following up!
I am having troubles getting this to work. Splunk does not seem to be adding in the results from the OR section.
Running just the initial search yields 13,748 events over the last 30 days;
source="*playbackinit.log" s=139 OR (source="*playbackevent.log" s=139 e="start")
if I run them separately I get 13,748 and 8156 respectively.
Any idea where I am going wrong?
Cheers,
Justin
... View more
02-14-2017
09:43 PM
Hi cmerriman,
Thanks for the reply. I don't think that will give me what I want.
The logs are coming from streaming servers. The playbackinit log registers when a customer clicks on a video and a player ID (pid) is generated. If they actually click play, it is logged in the playbackevent log.
The trick to it, is that the customer info is only logged in the playbackinit log, so in order to see who has clicked play on a video, i have done an inner join.
If there is a better way to do it, i'd love to hear it.
Cheers,
Justin
... View more
02-14-2017
09:42 PM
Hi Iguinn,
I don't think it will, let me explain.
The logs are coming from streaming servers. The playbackinit log registers when a customer clicks on a video and a player ID (pid) is generated. If they actually click play, it is logged in the playbackevent log.
The trick to it, is that the customer info is only logged in the playbackinit log, so in order to see who has clicked play on a video, i have done an inner join.
If there is a better way to do it, i'd love to hear it.
Cheers,
Justin
... View more
02-14-2017
09:41 PM
Hi gvmorley,
I'd love to be able to simplify the search but I don't think I can.
The logs are coming from streaming servers. The playbackinit log registers when a customer clicks on a video and a player ID (pid) is generated. If they actually click play, it is logged in the playbackevent log.
The trick to it, is that the customer info is only logged in the playbackinit log, so in order to see who has clicked play on a video, i have done an inner join.
If there is a better way to do it, i'd love to hear it.
Cheers,
Justin
... View more
02-13-2017
08:04 PM
Hello Peoples..
I have this issue with a search, here is the search that I am performing.
source="*playbackinit.log" s=139 | join type=inner pid [search source="*playbackevent.log" s=139 e="start"]
I get 49,614 events returned and I would expect to see more. The job shows this message:
[subsearch]: Subsearch produced 50000 results, truncating to maxout 50000
I have tried to change this behaviour in limits.conf, here is an excerpt from my limits.conf file:
[join]
subsearch_maxout = 500000
subsearch_maxtime = 120
subsearch_timeout = 180
Nothing I change seems to change this limit. Are there any suggestions here I could try?
Regards,
Justin.
... View more
08-17-2015
06:25 PM
Hi all,
I have a confusing issue here. I have a proprietary SQL system that I am connecting to, to get data from. It is AttacheBI which runs C-tree Ace SQL in the back end and I only have read-only access to the data.
I have established my connection through the use of an JDBC-ODBC bridge and I can browse the tables and fields using DB Query. So the connection is good there.
I am trying to setup a dbmon-tail, however, there is no single unique field in the data. I have spoken with Attache support and I would need to concat 3 fields in order to get my unique field to use as a Rising Column. Here is the SQL command I am using:
select TO_NUMBER(concat(concat(TO_CHAR(so.financialyearstart, 'YYYYMMDD'), LPAD(TO_CHAR(glt.batchnum), 10, '0')), LPAD(TO_CHAR(glt.seqnum), 10, '0'))) AS unique_concat, glt.*, glte.*, gl.description
FROM admin.systemgloptions2 so, (admin.generalledger_transaction glt LEFT JOIN admin.generalledger_transactionextension glte ON glt.code = glte.code AND glt.seqnum = glte.seqnum AND glt.batchnum=glte.batchnum)
LEFT JOIN admin.generalledger gl ON gl.code = glt.code
WHERE char_length(glt.code) > 0 {{AND $rising_column$ > ?}}
ORDER BY unique_concat
Here is a screen shot of the Input:
This query works fine in the DB Query part of DBX, but when I use it to create the dbmon-tail, I get this error:
Encountered the following error while trying to save: In handler 'dbx-monitors': Unable to xml-parse the following data: Failed to validate: com.splunk.config.SplunkConfigurationException: Error validating dbmonTail for monitor=dbmon-tail://AttacheBi... See splunkd.log for full data.
Looking at the logs, I can see that DBX has issues that the field unique_concat is not in the data.
From the splunkd.log:
08-18-2015 11:19:01.472 +1000 ERROR AdminManagerExternal - Received malformed XML from external handler:\nFailed to validate: com.splunk.config.SplunkConfigurationException: Error validating dbmonTail for monitor=dbmon-tail://AttacheBi/attachebi_general_ledger: [FairCom][ODBC FairCom Driver 10.3.0.19319(Build-140121)][ctreeSQL] -20006 Column unique_concat not found/specified with query
Any ideas on how I might be able to get around this? As stated before, I cannot modify the source in any way, as it is a locked down accounting package.
... View more
08-05-2015
06:41 PM
@nadid See my answer.
... View more
08-05-2015
06:39 PM
2 Karma
I know this is old but someone reading this might find it helpful.
You can use alias's with the rising column.
{{WHERE f.$rising_column$ > ?}}
and then in the tail input settings you place the field name.
This works for me in the same situation.
... View more
09-07-2014
08:24 PM
1 Karma
Hello,
I am indexing HelpDesk tickets and I am trying to derive some stats about these tickets.
During the indexing of these events (indexed from a MySQL DB using DB Connect), I have set my timestamp field to be the date the ticket was opened. This generate events like this:
2014-09-07T23:05:44.000 tech_name="Justin Franks" client_name="Wilfred Server" job_ticket_id=10840 problem_type3=Backup problem_type2="Backup and Restore" problem_type1="INFORMATION TECHNOLOGY" department_name= first_response_date= close_date= status_type_name=Open last_updated=1410095145.000 group_name="IT | Backups"
Is there a way to search on which tickets have been closed in say... the last 30 days? How would I change _time to a different field and re-search on it?
... View more
08-17-2014
11:02 PM
I added a lookup to find the name of the school but this worked great. Thanks.
... View more
08-17-2014
10:07 PM
Sorry. cname is a lookup based on the cid field.
... View more
08-17-2014
04:43 PM
I have a semi-static CSV that I am using for an input lookup to generate usage data on trial application we have in our environment. The lookup is working well except if no events are matched to the lookup. In these cases, I would like to display a zero count.
Here is my lookup code:
sourcetype=etv e=logon s=139 ur=TEACHER [| inputlookup etvd_trial_ids_week4.csv | rename "Customer ID" as cid, CurrentTrialStartDate as earliest | format] | stats dc(u) as dis_count by cname, cid
This gives me a table with two entries:
Caringbah High School CS04370 1
Hurlstone Agricultural High School CS04252 17
However, I know that there are 4 entries in my CSV:
| inputlookup etvd_trial_ids_week4.csv | rename "Customer ID" as cid, CurrentTrialStartDate as earliest
gives the following output:
CS04252 07/23/2014:00:00:00
CS04370 07/18/2014:00:00:00
CS05096 07/17/2014:00:00:00
CS09634 07/21/2014:00:00:00
Is it possible to display a zero in the dis_count field if no matching events are found?
... View more
- Tags:
- inputlookup
- search
05-12-2014
04:52 PM
I thought so too but here is what I get:
http://i.imgur.com/UR0dLKB.png
However, I'd like to have them on different bars of the count on the bottom.
... View more
05-11-2014
06:44 PM
I have a log of login timestamps. I would like to display the total count and total unique value count on the same bar chart.
Here is my search:
sourcetype="etv" cname="*" ur="*" | stats dc(u) as "unique", count by u | chart sum(count) as "Total Logins", sum(unique) as "Total Unique Logins"
Where 'u' is the UserID.
However, this search charts the "Total Logins" by "Total Unique Logins", which is not what I want.
Is there a way chart this information, split by their field names?
... View more
04-15-2014
09:57 PM
1 Karma
Ok, I was able to resolve this one myself.... eventually, by using:
http://docs.splunk.com/Documentation/DBX/1.1.3/DeployDBX/Configuredatabasemonitoring
and
http://docs.splunk.com/Documentation/DBX/1.1.3/DeployDBX/Troubleshoot#If_your_timestamp_is_not_of_type_datetime.2Ftimestamp
Additionally, by following MaverickT's response from http://answers.splunk.com/answers/98601/db-connect-timestamp-issues
Here is my resolved setup:
props.conf ( $SPLUNK_HOME$/etc/apps/dbx/local/props.conf 😞
[xcart_shop]
TIME_FORMAT=%Y-%m-%d %H:%M:%S
MAX_DAYS_AGO = 3650
inputs.conf ( $SPLUNK_HOME$/etc/apps/dbx/local/inputs.conf 😞
[dbmon-tail://Xcart_Shop_DB/etv_xcart_shop_orders]
host = webmysql.screenrights.org
interval = 15m
output.format = kv
output.timestamp = 1
output.timestamp.column = the_date
output.timestamp.format = yyyy-MM-dd HH:mm:ss
output.timestamp.parse.format = yyyy-MM-dd HH:mm:ss
query = select FROM_UNIXTIME(o.`date`) as the_date, o.orderid, d.productid, d.productcode, d.amount\r\nfrom xcart_orders o, xcart_order_details d \r\nwhere o.orderid = d.orderid {{and o.$rising_column$ > ?}}
sourcetype = xcart_shop
tail.rising.column = orderid
disabled = 0
table = etv_xcart_shop_orders
... View more
04-14-2014
09:52 PM
UPDATE:
I was able to change the MYSQL command to change the format of the date.
select FROM_UNIXTIME(o.`date`, '%Y-%m-%d %H:%i:%s') the_date, o.orderid, d.productid, d.amount
from xcart_orders o, xcart_order_details d
where o.orderid = d.orderid {{and o.$rising_column$ > ?}}
My rising column is: orderid
My timestamp column is: the_date
My timestamp format is: "yyyy-MM-dd HH:mm:ss"
However, when I try to index I get a bunch of errors the same.
2014-04-15 14:30:12.096 dbx1309:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '2008-10-07 11:26:07'
... View more
04-14-2014
04:24 PM
2 Karma
I have a MySQL DB with event data stored that I would like to index. I have no problems indexing the data but nothing I do will make Splunk recognise the date field.
The date is stored in an INT field as Epoch Time. e.g. "1397249090" or "1397433109". Note: It does not store it with the millisecond suffix. Just a straight 10-digit INT.
I have tried to use MYSQL to convert it but nothing seems to work;
FROM_UNIXTIME( date ) just adds '.000' to the end of the data. e.g. "1397249090.000" or "1397433109.000" and is still not recognised by Splunk when indexed.
cast(datefield as date) OR cast(datefield as datetime) OR cast(datefield as timestamp) all return nothing from a DB Query.
I would try a MYSQL convert but I am not sure on the syntax to get it work.
I think the issue is that it is an INT and not stored as a DATE. However, given this lack of success I have tried to setup a props.conf input.
My props.conf stanza is:
[xcart_shop]
TIME_FORMAT = %s
However, when I tried my DB Connect tail, I get this from the log file:
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397249090'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397249090'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397249090'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397266447'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397281370'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397301307'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397340247'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397380188'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397380299'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397433109'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397435663'
2014-04-15 08:34:00.399 dbx2849:WARN:ResultSetOutputPayload - Unrecognized timestamp format: '1397438935'
Anyone have any ideas or point me in the right direction?
... View more
- Tags:
- Splunk DB Connect 1
04-13-2014
04:41 PM
Hi,
When running that exact command without the rising column bit, i get:
productid amount orderid cast(o.`date` as date time)
with no data in the cast column.
Unfortunately, Splunk is not recognising the date when it gets indexed, so the "_time" field is when the data was indexed. This is why I am trying to convert it before it is indexed.
... View more
04-10-2014
04:11 PM
Hello,
I have an MySQL database and I am trying to index some data from it. I can connect with no problems and I can index the data if I use:
select * from <table>
However, the date field is stored as an INT.
I have tried the following code but it doesn't work.
select d.productid, d.amount, o.orderid, cast(o.`date` as datetime)
from xcart_orders o, xcart_order_details d
where o.orderid = d.orderid {{ and o.$rising_column$ > ? }}
How would I convert from Unix Epoch time store as an INT to something that Splunk will recognise as DATETIME? Normally, in SQL I would use DATEADD(); but Splunk doesn't seem to recognise that command.
... View more
10-01-2013
08:25 PM
I know the math points to total_count being equal to 0 but Splunk reports it as being equal to 1. Hence my confusion.
... View more
10-01-2013
06:17 PM
Hi Javier,
You question need clarifying. What data are you trying to keep?
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
06-05-2020
02:04 AM
|
