Getting Data In

How to monitor pastes from pastebin by keywords?

monteirolopes
Communicator

Hi,

I would like to monitor pastes from pastebin by keywords. For example, every time that my keyword is mentioned,
the entire publication would be indexed in Splunk.

Does anyone have any suggestions on how I can monitor these events?

Best regards,
Lopes.

Tags (2)
0 Karma

marcuspauli
New Member

Hi Lopes,

I had the sae issue and decided to use the "pastehunter" and import the outcoming JSON file into Splunk.
Works like nice. 🙂

Check my git for the original I've forked it from.
https://github.com/icepaule/PasteHunter

Cheers
Marcus

0 Karma

marcuspauli
New Member

For little HowTo I did this, check out my page at http://www.mpauli.de/pastebin-gatherings.html

Cheers
Marcus

0 Karma

LukeMurphey
Champion

There are a few options, depending on how much work you want to do.

1. Use the Bing search app (easy but won't index the entire document)

You could use the Bing Search app to do this it won't return the entire document.

The following search would look for pastes that contain the work Splunk:

| bingsearch query="splunk site:pastebin.com" | table name displayUrl snippet

Here is what the results would look like:

alt text

2. Write a custom web-site scraper (harder, but can be made to do whatever you want)

You could write your own scripted input in Python using something like scrapy (https://scrapy.org/). This is more involved since you will need Python code but you can make this script do whatever you want.

0 Karma

simon_lavigne
Path Finder

You can also create a Google Custom Search which specifies which URL's you want to search and use this custom input.

https://github.com/xg5-simon/Splunk_Google_CSE

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!