I have a MSSQL Datasource and having trouble parsing the timestamps (ProcessWorkStart field) correctly. For some reason it is parsing all events with the exact same timestamp (1969-12-31 18:59:59.999) no matter how I am configuring the inputs.conf. Please help! See below a sample event and my inputs.conf SAMPLE EVENT 1969-12-31 18:59:59.999 CoreWorkerProcessResultId=37114016 CoreProcessId=2 CoreProcess="xxxxxxx" CoreWorkerId=1 CoreWorker=xxxxxxxx GetWorkStart="2014-09-14 03:15:48.7170000" GetWorkMs=54 TotalWorkCount=0 GetTotalWorkCountMs=1 HasWork=false MessageCount=0 ProcessWorkMs=0 Inputs.conf [dbmon-tail://SAMMY/SAMMY_Oper_Logs] index = emme interval = 60 output.format = kv output.timestamp = 1 output.timestamp.column = ProcessWorkStart output.timestamp.format = yyyy-MM-dd HH:mm:ss.SSS query = select [CoreWorkerProcessResultId]\r\n ,cwpr.[CoreProcessId]\r\n ,cp.Description as [CoreProcess]\r\n ,cwpr.[CoreWorkerId]\r\n ,cw.Description as [CoreWorker]\r\n ,[GetWorkStart]\r\n ,[GetWorkMs]\r\n ,[TotalWorkCount]\r\n ,[GetTotalWorkCountMs]\r\n ,[HasWork]\r\n ,[MessageCount]\r\n ,[ProcessWorkStart]\r\n ,[ProcessWorkMs]\r\n from CoreWorkerProcessResult cwpr \r\n inner join CoreProcess cp on cp.CoreProcessId = cwpr.CoreProcessId\r\n inner join CoreWorker cw on cw.CoreWorkerId = cwpr.CoreWorkerId\r\n where cwpr.CoreWorkerProcessResultId > 37000000 {{AND $rising_column$}} sourcetype = emme_core_stat tail.rising.column = CoreWorkerProcessResultId table = EMME_Oper_Logs disabled = 1 ... View more

Can Hunk read files in .HAR archives? ... View more

I need to create a report that shows max indexed volume per day by month per index. The following search gives me the indexing volume per day per index. How can modify this search to get what I need? index=_internal source=license_usage.log type=Usage | timechart span=1d sum(eval(b/1024/1024/1024)) AS volume_b by idx ... View more

I am using the JMS Input with Websphere MQ and have successfully been able to monitor queues and pull the data into splunk. However, my Websphere admin has told me that the JMS app is not pulling messages out of the queues fast enough and it's causing the queues to back up. The Websphere admin suggested I add multiple inputs for each of the queues and I ended up with a total of 36 JMS inputs. After adding the inputs and restarting Splunk, the jms input is no longer working. I can also see that the corresponding java process fails. Splunkd shows the following errors. Please help! 8/19/14 12:00:14.568 PM 08-19-2014 12:00:14.568 -0400 INFO ExecProcessor - New scheduled exec process: python F:\Splunk\etc\apps\jms_ta\bin\jms.py host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" Traceback (most recent call last): host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" File "F:\Splunk\etc\apps\jms_ta\bin\jms.py", line 129, in <module> host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" do_run() host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" File "F:\Splunk\etc\apps\jms_ta\bin\jms.py", line 48, in do_run host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" run_java() host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" File "F:\Splunk\etc\apps\jms_ta\bin\jms.py", line 99, in run_java host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" process = Popen(java_args) host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" File "F:\Splunk\Python-2.7\Lib\subprocess.py", line 711, in __init__ host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" errread, errwrite) host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" File "F:\Splunk\Python-2.7\Lib\subprocess.py", line 948, in _execute_child host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" startupinfo) host = LOUWEBWPL20S01 source = F:\Splunk\var\log\splunk\splunkd.log sourcetype = splunkd 8/19/14 12:00:15.582 PM 08-19-2014 12:00:15.582 -0400 ERROR ExecProcessor - message from "python F:\Splunk\etc\apps\jms_ta\bin\jms.py" WindowsError: [Error 87] The parameter is incorrect ... View more

I'm trying to extract a string in a field that spans multiple lines. See example below. 03/09/2014 07:10:38 AM - Process(****.****) User(***) Program(*****) Host(*****) AMQ****: The data received from host '******* (*******)' is not valid. EXPLANATION: An error has been detected, and the WebSphere MQ error recording routine has been called. The failing process is process *****. ACTION: Use the standard facilities supplied with your system to record the problem identifier and to save any generated output files. Use either the WMQ Support site: http://www.ibm.com/software/integration/wmq/support/, or IBM Support Assistant (ISA): http://www.ibm.com/software/support/isa/, to see whether a solution is already available. If you are unable to find a match, contact your IBM support center. Do not discard these files until the problem has been resolved. ----- amqxfdcx.c : 829 -------------------------------------------------------- The part i'm trying to extract is below: The data received from host '******* (*******)' is not valid. ... View more

I have a requirement to route events to separate indexes based on two conditions. 1) must contain the string PI_EVENT 2) Get the value contained in the value The index it needs to be routed to will be the value of businessdomainid + -sec (ex. businessdomainid1-sec) How do I write my regex and format statement to have this work? Here's my transforms.conf so far [Security] SOURCE_KEY = _raw DEST_KEY = _MetaData:Index REGEX=(?m)\<BusinessDomainId\>(BusinessDomainId1|BusinessDomainId2|BusinessDomainId3)\</BusinessDomainId\>|<LogEventTypeCode>PI_EVENT</LogEventTypeCode> FORMAT=$1 Sample event <ELLogInputMessage> <Header> <LogEventTypeCode>PI_EVENT</LogEventTypeCode> <LogSeverityCode>CRITICAL</LogSeverityCode> <LogEventDateTime>2014-05-06T23:59:59.9999999-05:00</LogEventDateTime> </Header> <SourceInformation> <EAPMId>1</EAPMId> <HostMachineName>HostMachineName3</HostMachineName> <HostEnvironmentName>HostEnvironmentName3</HostEnvironmentName> <ComponentId>ComponentId3</ComponentId> <ComponentName>ComponentName3</ComponentName> <ApplicationEventCorrelationId>ApplicationEventCorrelationId3</ApplicationEventCorrelationId> <UserId>UserId1</UserId> <UserSrc>UserSrc1</UserSrc> <BusinessDomainId>BusinessDomainId1</BusinessDomainId> <BusinessDomainName>BusinessDomainName1</BusinessDomainName> </SourceInformation> <DataAccessInformation> <DataCompId>DataCompId2</DataCompId> <TypeOfAccess>VIEW</TypeOfAccess> <SubjectOfInterest> <SubjectId>SubjectId13</SubjectId> <SubjectName>SubjectName13</SubjectName> <SubjectDomainName>SubjectDomainName3</SubjectDomainName> </SubjectOfInterest> <AccessDateTime>2014-05-06T23:59:59.9999999-05:00</AccessDateTime> </DataAccessInformation> <DetailedLogInformation>anyType</DetailedLogInformation> </ELLogInputMessage> ... View more