Thanks., I have created props.conf on Universal Forwarder : [esd:trace] CHARSET = AUTO but when I search the data , it is shown in binary format. ... View more

Thanks., I have created props.conf on Universal Forwarder : [esd:trace] CHARSET = AUTO but when I search the data , it is shown in binary format. ... View more

Those log files are written by an .NET application running on Windows onto a CIFS share. I have mounted the CIFS share on a linux server, I had issues of extreme slowness and lag monitoring logs directly from CIFS shares directly using a universal forwarder running on Windows server. Now there 100's of such logs files that need to be monitored and written continuously. ... View more

hi @harsmarvania57 file NMDox_PRD.EP6XWBSDE26931\ Started\ 2017-09-14.txt NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt: Little-endian UTF-16 Unicode English text, with very long lines, with CRLF line terminators ... View more

NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt is the example of log file which I want to monitor. All the files are of same type in the directory .. only the number in the log file name after BSDE changes. Also the date stamp in the log file of course. ... View more

Thanks @deepashri_123 file NMDox_PRD.EP6XWBSDE26931\ Started\ 2017-09-14.txt NMDox_PRD.EP6XWBSDE26931 Started 2017-09-14.txt: Little-endian UTF-16 Unicode English text, with very long lines, with CRLF line terminators and the path is "esc_logs" just confirmed from inputs.conf ... View more

hi Scott, Jkat54 we are planning a similar activity. Our environment is huge with multiple indexers. We plan to separate the monitoring tools and production applications. I plan to spin up new machines, install splunk add them to index cluster and before that update outputs.conf file with new machines. We've 6 indexers on RHEL 6.7 running Splunk 6.5.4. few questions I have are - once the new peers are in cluster with old indexers and receive data , doesn't this data also get replicated to old servers and continue to use up storage on old peers? How can we verify that the replication is complete and all new servers have the old data copies replicated ? so that I can shutdown the old servers. Current replication factor - 2 and search Factor - 1. What should I update these to when new peers are added ? Thanks ... View more

hi Guilmxm, how did your data migration go about ? we're planning similar activity wherein we want to migrate existing Splunk with its data to new hosts. Most of our data is critical and again huge snapshot (TB) with long retention policies. We obviously will have new cluster setup and then have the outputs.conf updated to send data to new cluster , but for old data which still remains critical, how can we make it searchable if we decide to decommission old servers soon ? any ideas/thoughts from community would be appreciated. ... View more

Thanks, I actually tried using the following settings in props.conf and got most of it to work. But there's still single JSONevent getting merged after every ERROR event. LINE_BREAKER = ((\d{4}-\d{2}-\d{2})|({) MUST_BREAK_AFTER = } After doing above all the JSON events starting '{' are separated and also the ERROR events starting timestamp 2017-03-27 are also separated. But at the end of the ERROR event , the first JSON event following it is merged and the again rest are working as expected. Log file - {"timestamp":"01:03:54.630", "action":"Begin Processing", "object":"KnowledgeArticles"} {"timestamp":"01:03:55.583", "action":"Found", "object":"Articles", "total":"4173"} {"timestamp":"01:03:55.583", "action":"Processing", "object":"Article", "ID":"000012887", "Ver":"kA1800000008cLaCAI", "Type":"Information__kav"} 2017-03-27 01:03:55.599: ERROR - Message: A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - An existing connection was forcibly closed by the remote host.); InnerException: An existing connection was forcibly closed by the remote host; StackTrace: at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action 1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParserStateObject.SNIWritePacket(SNIHandle handle, SNIPacket packet, UInt32& sniError, Boolean canAccumulate, Boolean callerHasConnectionLock) at System.Data.SqlClient.TdsParserStateObject.WriteSni(Boolean canAccumulate) at System.Data.SqlClient.TdsParserStateObject.WritePacket(Byte flushMode, Boolean canAccumulate) at System.Data.SqlClient.TdsParser.TdsLogin(SqlLogin rec, FeatureExtension requestedFeatures, SessionData recoverySessionData) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource 1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource 1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource 1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource 1 retry) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource 1 retry) at System.Data.SqlClient.SqlConnection.Open() at MyService.ServiceWork.GetDocumentHistory(String docId) at MyService.ServiceWork.ProcessKnowledgeArticle(KnowledgeArticleVersion kav) at MyService.ServiceWork.ProcessSalesforceObjects() at MyService.ServiceWork.ProcessSalesforce() at MyService.ServiceWork.TimerElapsed(Object sender, ElapsedEventArgs e) {"timestamp":"01:03:55.599", "action":"Waiting", "object":"Timer"} {"timestamp":"01:05:55.616", "action":"Elapsed", "object":"Timer"} {"timestamp":"01:05:55.616", "action":"Pause", "object":"Waiting"} Example of merged events - 2017-03-27 01:03:55.599: ERROR - Message: A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - An existing connection was forcibly closed by the remote host.); InnerException: An existing connection was forcibly closed by the remote host; StackTrace: at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action 1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParserStateObject.SNIWritePacket(SNIHandle handle, SNIPacket packet, UInt32& sniError, Boolean canAccumulate, Boolean callerHasConnectionLock) at System.Data.SqlClient.TdsParserStateObject.WriteSni(Boolean canAccumulate) at System.Data.SqlClient.TdsParserStateObject.WritePacket(Byte flushMode, Boolean canAccumulate) at System.Data.SqlClient.TdsParser.TdsLogin(SqlLogin rec, FeatureExtension requestedFeatures, SessionData recoverySessionData) at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover) at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout) at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance) at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData) at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions) at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource 1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource 1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource 1 retry, DbConnectionOptions userOptions) at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource 1 retry) at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource 1 retry) at System.Data.SqlClient.SqlConnection.Open() at MyService.ServiceWork.GetDocumentHistory(String docId) at MyService.ServiceWork.ProcessKnowledgeArticle(KnowledgeArticleVersion kav) at MyService.ServiceWork.ProcessSalesforceObjects() at MyService.ServiceWork.ProcessSalesforce() at MyService.ServiceWork.TimerElapsed(Object sender, ElapsedEventArgs e) {"timestamp":"01:03:55.599", "action":"Waiting", "object":"Timer"} ... View more

Thanks SomeSoni. the second one worked as expected. ... View more

can you paste sample data ? ... View more

hi SomeSoni .. can you help me with similar problem - https://answers.splunk.com/answers/491154/how-to-configure-setting-the-host-field-during-ind.html Thanks ... View more

hi Hung_Nguyen , in which field are you capturing the widgetId or "Widget ID"? ... View more

in which field are you capturing the widgetId or "Widget ID"? you were able to produce results using the widgetId field using your first query right ? index=tto* Type=ERROR Message="Widget load failed" | stats count(eval(Type="ERROR")) as errors by widgetId The table shows up as Widget ID | count WidgetA | 45 WidgetB | 90 can you try this ? index=tto* (sourcetype=access method=GET uri_path="index.htm") OR (Type=ERROR Message="Widget load failed") | stats count(eval(uri_path="index.htm")) as UsersCount, count(eval(Type="ERROR")) as errors | eval ratio = round((errors/UsersCount)*100, 2) | table "Widget ID", ratio ... View more

Is your Splunk setup standalone or clustered ? if you're going to install the TA and App on the same instance then.. you just need to install the TA first , but skip the configure step. and then configure the app to pull data from ServiceNow rather. But since you've already configured the TA with inputs and updated the inputs.conf with index=snow, then it has to be allowed to search. servicenow TA creates some indexes as part of the installation, You should add these indexes to your main indexes.conf file with reference to Primary volume - snow is index I created. the rest were created by the TA and would through errors, for not referencing the primary volume. So I added it to the master indexes. conf file. example - [snow] homePath = volume:primary/snow/db coldPath = volume:primary/snow/colddb thawedPath = $SPLUNK_DB/snow/thaweddb maxTotalDataSizeMB = 750000 frozenTimePeriodInSecs = 31536000 [snow_sys_user_group_list_index] homePath = volume:primary/snow_sys_user_group_list_index/db coldPath = volume:primary/snow_sys_user_group_list_index/colddb thawedPath = $SPLUNK_DB/snow_sys_user_group_list_index/thaweddb maxTotalDataSizeMB = 750000 frozenTimePeriodInSecs = 31536000 [snow_cmdb_ci_list_index] homePath = volume:primary/snow_cmdb_ci_list_index/db coldPath = volume:primary/snow_cmdb_ci_list_index/colddb thawedPath = $SPLUNK_DB/snow_cmdb_ci_list_index/thaweddb maxTotalDataSizeMB = 750000 frozenTimePeriodInSecs = 31536000 [snow_incident_state_index] homePath = volume:primary/snow_incident_state_index/db coldPath = volume:primary/snow_incident_state_index/colddb thawedPath = $SPLUNK_DB/snow_incident_state_index/thaweddb maxTotalDataSizeMB = 750000 frozenTimePeriodInSecs = 31536000 ... View more

Hi Davebo, Is your add-on and app on the same Splunk instance ? If so , then do not run set-up to configure the add-on to enable the inputs , instead setup the servicenow App to collect data directly which will enable the inputs on app and they get saved in servicenowapp/local/inputs .conf. Then change the index=main to index=snow in inputs.conf, followed by restart of splunk . refer to this page - http://docs.splunk.com/Documentation/ServiceNow/4.0.3/Install/Setuptheapp ... View more