Deployment Architecture

Bin limits?

mistydennis
Communicator

I'm running a search over two sourcetypes - an inputlookup (contains metadata about the pdf) and the log file (contains download counts of the pdf) in order to show what pdfs are being downloaded over a given period of time. I've applied a bin function to the download counts so that any distinct IP is counted once during a 30 minute time period (to eliminate possible bots). I'm finding that the bin only works if the time picker is set to less than 4 months. As soon as I run a search for download counts OVER a 4 month time period, only about 70% of the hits are counted. Here's what I'm using:

| inputlookup 'foo' | join type=left 'pdfName' [search response_code=200 | bin _time span=30m | stats count dc(src_ip) as distinctip, dc(pdfName) as DownloadCounts by pdfName, agentType, IPLocation, _time, src_ip | stats count AS Downloads by pdfName] |table pdfName, pdfTitle, pdfAuthor, Downloads 

What am I doing wrong? Is there a bin limit? Or is this a complication of using join? Any help would be greatly appreciated.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this. For best performance, it's often recommended that you add one or more metadata field filters in the base search, such as index, sourcetype, source or host. So, if possible add those to your log file query.

index=foo sourcetype=bar response_code=200 | bin _time span=30m | stats count by pdfName, agentType, IPLocation, _time, src_ip | stats count AS Downloads by pdfName | append [| inputlookup 'foo' ] | stats values(*) as * by PdfName

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Try this. For best performance, it's often recommended that you add one or more metadata field filters in the base search, such as index, sourcetype, source or host. So, if possible add those to your log file query.

index=foo sourcetype=bar response_code=200 | bin _time span=30m | stats count by pdfName, agentType, IPLocation, _time, src_ip | stats count AS Downloads by pdfName | append [| inputlookup 'foo' ] | stats values(*) as * by PdfName

mistydennis
Communicator

This didn't give me exactly what I wanted but it pointed me in the right direction. I ended up using:

index=foo response_code=200 | bin _time span=30m | stats count by pdfName, agentType, IPLocation, _time, src_ip | stats count AS Downloads by pdfName | join [|inputlookup 'foo' |]

The append command gave me two rows (one was empty) but the join command gave me a single line.

I really appreciate the help, thank you so much!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...