While forwarder-to-indexer traffic can be wrapped in SSL, it's not technically an HTTP connection, and therefore won't properly traverse a web proxy. The 2 ways I know how to accomplish this are as follows: Use an intermediate forwarder (generally within a DMZ). Internal hosts have access to this host, and send their logs to the IMF. That host has outbound access to the indexer layer. Use a SOCKS v5 Proxy If you wish to secure your forwarder-to-indexer traffic behind a proxy, note that as of 6.3, Splunk supports the use of SOCKS v5 proxies for forwarder-to-indexer traffic. Details are available on-line at: http://docs.splunk.com/Documentation/Splunk/6.6.3/Forwarding/ConfigureaforwardertouseaSOCKSproxy ... View more

While forwarder-to-indexer traffic can be wrapped in SSL, it's not technically an HTTP connection, and therefore won't properly traverse a web proxy. The 2 ways I know how to accomplish this are as follows: Use an intermediate forwarder (generally within a DMZ). Internal hosts have access to this host, and send their logs to the IMF. That host has outbound access to the Cloud stack. Use a SOCKS v5 Proxy If you wish to secure your forwarder-to-indexer traffic behind a proxy, note that as of 6.3, Splunk supports the use of SOCKS v5 proxies for forwarder-to-indexer traffic. Details are available on-line at: http://docs.splunk.com/Documentation/Splunk/6.6.3/Forwarding/ConfigureaforwardertouseaSOCKSproxy ... View more

Hello. Cisco's IOS products support syslog as the network protocol over which logs are sent. There's 2 parts to the answer: 1) Configuring the IOS devices to send their logs. Refer to the Cisco documentation relevant to your devices for details. Generally, it's a matter of defining the syslog destination and the log level. An example may look something like this: service timestamps log datetime service timestamps debug datetime service sequence-numbers logging 169.254.123.234 logging trap 5 where 169.254.123.234 is your syslog server, and you're capturing NOTICE or higher log messages. 2) Configure a syslog server. Here, you have 2 choices: .......(1) Install and configure a stand-alone syslog server (such as rsyslog, syslog-ng, or Kiwi). Your Cisco devices will sed their logs to this syslog server, which writes the files down to disk. A Splunk Universal Forwarder can grab the files from there. http://docs.splunk.com/Documentation/Splunk/6.6.1/Data/Monitorfilesanddirectories provides some decent insight on general file and directory monitoring. There are a few other Answers posts you may reference on this topic. .......(2) Configure Splunk to listen for syslog. You can configure a standard UDP input on port 514 for this (Splunk natively can listen for syslog). There's some great documentation on configuring this at http://docs.splunk.com/Documentation/Splunk/6.6.1/Data/Monitornetworkports and http://docs.splunk.com/Documentation/Splunk/6.6.1/Data/HowSplunkEnterprisehandlessyslogdata We generally recommend that customers use a syslog server, as it provides a framework for handling heterogenous sourcetypes without a lot of effort. For relatively simple environments (limited sourcetypes), the syslog input is Ok, too. Best of luck in your endeavours! ... View more

The way I handled this recently was to forego the time picker and use a multi-select in concert with the automagic "date" field. Note: I've limited the date picker here to the last 15 days, so as not to overtax the SH. Adjust the parameter to meet your needs. <input type="multiselect" token="DATE_tok" searchWhenChanged="true"> <label>Date</label> <choice value="*">Any</choice> <prefix>(</prefix> <suffix>)</suffix> <valuePrefix>date="</valuePrefix> <valueSuffix>"</valueSuffix> <delimiter> OR </delimiter> <fieldForLabel>date</fieldForLabel> <fieldForValue>date</fieldForValue> <search> <query>sourcetype="SplunkNinjaDataSource" | stats count by date</query> <earliest>-15d@h</earliest> <latest>now</latest> </search> </input> Just add the token $DATE_tok$ into your search, and you're off to the races. HTH ... View more

Part of the beauty of managing the GAiA OS is that the vast majority is directly derived from RedHat EL (depending on the version of GAiA, predominantly RHEL3 or RHEL5). The output you're looking to ingest is from "mpstat", a standard *Nix application. The *NIX TA (available at https://splunkbase.splunk.com/app/833/ ) does the necessary work to make sense of this input type without reinventing the wheel. The sourcetype will be "cpu". ... View more

LEA natively will include accounting information if the rule is configured to support it - this is part of the LEA "standard." From the 2011 LEA Update document: When a user specifies Account as the Track option, in a Security Policy rule for instance, byte information is included in the log record. Note: In NG log records are composed of fragments and fragments of the same chain might be spread over the log file and sometimes even across file boundaries. When the file read mode is Unified updates in the form of record fragments like accounting bytes are not available in LEA_ONLINE mode. When the file read mode is not Unified then record fragment updates should not be counted as separate connections. The thing that can be very confusing about accounting is that not every log entry for a rule with accounting enabled with include accounting data. The reason for this is that Check Point will often generate two log entries for any rule hit with accounting enabled The following are the fields included in accounting logs: bytes (LEA_VT_INT) - The number of bytes transmitted in the connection. elapsed (LEA_VT_DURATION_TIME) - the duration of the connection has_accounting (LEA_VT_INT) - When 1, this indicates that the log record has a matching record in the accounting log packets (LEA_VT_INT) The number of packets in the session segment_time (LEA_VT_TIME) - Presence of this field indicates the connection has closed. start_time (LEA_VT_TIME) - The date and time the connection started. Use the elapsed field to calculate the time the connection ended. From personal experience, I can tell you that I have never seen the "has_accounting" field populated with anything but "0". Here, however, is an example of a log event from my Splunk environment at home being fed by Check Point: time=1478265919|loc=105729|fileid=1478231941|action=accept|orig=192.168.1.1|i/f_dir=inbound|i/f_name=eth0|has_accounting=0|uuid=<581c8c3f,00000001,0101a8c0,c0000000>|product=VPN-1 & FireWall-1|inzone=Internal|outzone=DMZ|rule=33|rule_uid={0AD23AF3-7991-405D-A22B-BFB88BBE0B1A}|rule_name=Misc Out|src=192.168.1.153|s_port=50134|dst=198.51.100.7|service=8000|proto=tcp|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={1C2E2384-57BA-5B4C-9F2A-2C0A4EDABE04};mgmt={obfuscated};date=1473349961;policy_name=Main_FW]|origin_sic_name=CN={obfuscated}|start_time= 4Nov2016 9:25:19|segment_time= 4Nov2016 9:25:19|elapsed=0:00:00|packets=2|bytes=104|client_inbound_packets=1|client_outbound_packets=1|server_inbound_packets=1|server_outbound_packets=1|client_inbound_bytes=64|client_outbound_bytes=40|server_inbound_bytes=40|server_outbound_bytes=64|client_inbound_interface=eth0|server_outbound_interface=eth3|__pos=7|__nsons=0|__p_dport=0 and here is the corresponding initial hit in the log: (note "has_accounting=0"): time=1478265919|loc=235290|fileid=1478032508|action=accept|orig=192.168.1.1|i/f_dir=inbound|i/f_name=eth0|has_accounting=0|uuid=<581c8c3f,00000001,0101a8c0,c0000000>|product=VPN-1 & FireWall-1|inzone=Internal|outzone=DMZ|rule=33|rule_uid={0AD23AF3-7991-405D-A22B-BFB88BBE0B1A}|rule_name=Misc Out|src=192.168.1.153|s_port=50134|dst=198.51.100.7|service=8000|proto=tcp|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={1C2E2384-57BA-5B4C-9F2A-2C0A4EDABE04};mgmt={obfuscated};date=1473349961;policy_name=Main_FW]|origin_sic_name={obfuscated} In short, it's in there, you just need to know what to look for. HTH ... View more

I've taken the liberty of updating the diagram slightly to reflect both changes in code since 6.2 and recommendations between this and one other post. JPG format: https://drive.google.com/open?id=0B3PXaVtuNWbnMzJ6bTlkcXRmMFE Visio format: https://drive.google.com/open?id=0B3PXaVtuNWbnRGtyM2phX2tWQ3M Thanks @rob_jordan for the great effort and for sharing! ... View more

You have a few different options. Two of the ones I can think of right off the top of my head include the following possibilities: 1) If the device supports netflow, forward the netflow data to a collector and forward from there. There are a multitude of possible configuration scenarios for this 2) Run a span off the interesting traffic ports and use either Stream to capture or run a forwarder on the listening device A third (not suggested) method would be to run the device in debug mode and forward the debug data accordingly. I strongly suggest against this, though. ... View more

It occurred to me that not everyone will be up on how to get going with API access for Enphase... For those of you who are interested in this integration, be advised that there are a few steps required in order to enable API access: 0) Make sure API access is allowed and you've documented both your API username and your system ID. - Navigate to https://enlighten.enphaseenergy.com, log in, then access the "Settings" by clicking on the down arrow next to your name - You'll see your API ID under "API Settings" - Your system ID is most easily distinguised in the URL that is presented to you when you log in... https://enlighten.enphaseenergy.com/pv/systems/{SYSTEM_ID}/overview 1) Navigate to https://developer.enphase.com/docs/quickstart.html for a brief overview of the API integration and operation 2) Open a Developer Account by registering at https://developer.enphase.com/signup 3) Define the "Application" on the Enphase site (in order to generate an App Key): https://developer.enphase.com/admin/applications - You'll need to define the app tier you wish to use -- "Watt" is the free tier, allowing for up to 10k hits per month 4) Activate the app by navigating to the link provided when you define the app 5) Document the app key along with your API user ID 6) Test from the CLI before defining the REST API input - From the CLI of your Splunk instance, run a simple curl, such as: curl -v "https://api.enphaseenergy.com/api/v2/systems/{SYSTEM_ID}/summary?key={MY_APP_KEY}&user_id={MY_USER_ID}" This should return an HTTP 1.1/200 response with JSON-formatted payload ... View more

Hi there! Funny you should ask, as I just started playing on this use case -- Splunking data from my Enphase system. The easiest way I know of to do what we want is to use the REST API Modular Input App -- see https://splunkbase.splunk.com/app/1546/#/details for details. This will add a new data input method (you can browse from Settings -> Data -> Data Inputs) that provides a nice, simple UI for building the query. No need for curl scripts and cron jobs 😉 In my testing, I'm using the Enphase "stats" API call. Because Enphase wants time in epoch, you'll want to tweak the app's token.py ($SPLUNK_HOME/etc/apps/rest_ta/bin/token.py) to add a token for the current time (or some variant thereof, such as current time minus 5 minutes (the interval that Enphase supports for the stats command)) . In my case, I added the following: import calendar def epochdate(): # UTC format: timenow = datetime.datetime.utcnow() timenow = datetime.datetime.now() return calendar.timegm(timenow.timetuple()) Using "calendar" is the cleanest way I've found so far to do the conversion. I can then use the %epochdate% token in my config. I am eager to hear how you fare with this project! Happy Splunking! ... View more

< disclaimer > I haven't yet had the chance to test this on a live system, so YMMV < /disclaimer > The way I know of to pull the rules is via the PAN-OS XML API. You'll want to use the Xpath parameter to target just the rule base itself, so Xpath=/config/devices/entry/vsys/entry/rulebase/security should do the trick. Your API call will look something like this: https://{PANORAMA_IP_or_HOST}/api/?type=config&action=show&key=apikey&xpath=/config/devices/entry/vsys/entry/rulebase/security You're most interested in the "Rule Name" field, as my understanding is that this is the unique identifier in the traffic log that you can correlate against. For more information, reference the PAN API usage guide (available here): https://www.paloaltonetworks.com/documentation/71/pan-os/xml-api Good luck, and Happy Splunking! ... View more