With Syncsort Ironstream, you can collect log data from SMF, RMF, Syslog and other z/OS sources, and forward that data in real time to the Splunk® Enterprise analytics platform. That gives you visibility into your z/OS environment. Ironstream also integrates with Splunk’s Enterprise Security and IT Service Intelligence applications. This goes beyond IT operational analytics to give you a firmer grasp of potential security threats in your z/OS environment. It ensures that your critical business services are being delivered on time.
For more information on Ironstream: http://www.syncsort.com/en/Products/Mainframe/Ironstream
IBM Common Data Provider for z Systems can forward mainframe data to Splunk in near real-time.
It supports a wide variety of data including 140 data sources and 100+ SMF record types, and it can stream structured and unstructured data or use batch mode to collect data. IBM Common Data Provider for z Systems also has advanced filtering capabilities including RegEx and time filtering.
You can also learn more about IBM Common Data Provider directly on Splunkbase.
Splunk and IBM recently hosted a joint webinar on how they are partnering to help customers gain access to mainframe machine data more quickly for real-time investigation, analytics, and pattern analysis. You can watch the replay here: https://onlinexperiences.com/scripts/Server.nxp?LASCmd=AI:4;F:QS!10100&ShowKey=43016&AffiliateData=F...
IBM Transaction Analysis Workbench for z/OS ("Workbench") can forward a wide range of logs from various z/OS subsystems to Splunk.
Recent enhancements to Workbench include features specifically for streaming logs in JSON Lines format off z/OS to a Splunk TCP data input.
Some key points about log forwarding with Workbench:
Example JCL to forward CICS monitoring facility (CMF) performance class (SMF type 110) records from a dumped SMF data set:
//S1 EXEC PGM=FUWBATCH //STEPLIB DD DISP=SHR,DSN=FUW.SFUWLINK //LOGIN DD DISP=SHR,DSN=SMF.MVS1(-1) //SYSPRINT DD SYSOUT=* //SYSIN DD * STREAM NAME(SPLUNK) TRANSPORT(TCP) HOST(mysplunk) PORT(6789) + LINES FLAT OMITNULL NOTITLE FIELDCASE(LOWER) ASCII LF ZONE JSON STREAM(SPLUNK) CODE(CMF) FIELDS( * Insert list of CMF fields you want to forward )
Related Splunk app on GitHub (currently just one dashboard, for CICS performance).
Some relevant topics in the Workbench (version 1.3) product documentation:
Getting started ► Overview ► Features ► Log forwarding
Forwarding logs to analytics platforms ► Splunk
Extracting logs to CSV or JSON ► Streaming JSON Lines over TCP
Disclosure: I am the author of the Workbench product documentation.
Ironstream from Syncsort can do all of this work for you. It will handle all of the issues related to SYSLOG, z/OS SMF records, log4j and flat files. It deals with the compression, the triplets, the binary data and converts the data from EBCDIC to ASCII. It does this very efficiently, even offloading a lot of the work to a zIIP engine in order to keep the MSU cost of this work to an absolute minimum. This is all done in real time to give you the best data latency possible while not impacting the existing workload on your system.
Hi - if you need to get mainframe data (security, database, CICS, FTP, TCPIP, master console messages and much more), please see meas-info.com. Our Mainframe Event Acquisition System (MEAS)product will allow you to monitor, filter and forward - in real time - any/all events from the mainframe that you would like to see in Splunk. It take roughly 1/2 day to install and no IPL necessary. Please give us a call if you need any more information.
Hi - check out the Mainframe Event Acquisition System (MEAS) which will send mainframe data to Splunk. Events such as security activity, database accesses, CICS transaction activity, dataset access, FTP, TCPIP, RMF, SMP/E and more. You have the ability to filter so that you can send only what you really want to Splunk for further alerting and reporting. www.meas-info.com.
I did a little exploration with a third party about getting performance metrics off of Nonstop/Tandem hardware. We wrote a program to collect the metrics and write them out using a vaguely sensible format to a socket. Then it was just a simple TCP input in the Splunk server and some extractions. It was quite a successful prototype and proof-of-concept although we didn't end up releasing the product.
There is no forwarder code for mainframe systems today. You could always submit a feature request asking for it. If you don't, then chances are it will never happen. (Splunk product management does not look at Splunkbase questions around a particular topic as a proxy for actual feature requests)
Also, you need to be explicit about what you are looking for. There are at least 3 common "mainframe" operating systems, and programs compiled for one WILL NOT work on the other. You have:
All of these are "mainframe" operating systems, each with their own APIs and idiosyncracies. When 99% of all people think "mainframe" they are thinking of z/OS, but the alternatives exist. (Functionally speaking, Linux/s390 would be the least difficult for Splunk to port a forwarder to - the other two could be much worse)
I downvoted this post because not really true, there are a variety of forwarding options on the market like ibm's common data provider... although appreciate this post might have been correct when initially written!
I have looked into this in the past. You will more than likely need to use some 3rd party software to create metrics that Splunk will collect. There are currently no Splunk Apps/built in functionality (a side from using syslog) for doing this. This company seemed to have a solution that would plug into Splunk http://www.infosecinc.com/meas.php