Getting Data In

How to remove a specific string from an events in splunk ?

Motivator

Hi All, currently we are facing an issue in removing a specific values from the event list starting with the word "at" as we do not want these in the splunk events.

Example :

5/16/17
8:57:04.674 AM

[2017-05-16T08:57:04.674-04:00] [TIM_server1] [ERROR] [] [db2.tam.platform.kernel.impl] [tid: [ACTIVE].ExecuteThread: '29' for queue: 'xxxx.kernel.Default (self-tuning)'] [userId: xxx_TIMPASSSYNC] [ecid: d87cb14ef99e9513:-133033ab:15c0857bfd3:-8000-000000000005cb77,0] [APP: TIM#11.1.2.0.0] [J2EE_APP.name: spml-dsml] [J2EE_MODULE.name: spmlws] [WEBSERVICE.name: TIMProvisioning] [WEBSERVICE_PORT.name: TIMProvisioningPort] Kernel Information: {0}[[
db2.tam.platform.kernel.ValidationFailedException: IAM-3030006:The following password policy rules were not met:Password must not be one of 8 previous passwords.
:
at db2.tam.passwordmgmt.eventhandlers.UserPasswordValidationHandler.validate(UserPasswordValidationHandler.java:96)
at db2.tam.platform.kernel.impl.TIMEvent.executeHandlers(TIMEvent.java:204)
at db2.tam.platform.kernel.impl.MonitoredTIMEvent.invokeExecuteHandler(MonitoredTIMEvent.java:99)
at db2.tam.platform.kernel.impl.MonitoredTIMEvent.executeHandlers(MonitoredTIMEvent.java:69)
at db2.tam.platform.kernel.impl.TIMEvent.execute(TIMEvent.java:157)
at db2.tam.platform.kernel.impl.ProcessImpl.executeStage(ProcessImpl.java:223)
at db2.tam.platform.kernel.impl.TIMProcess.doStageExecution(TIMProcess.java:38)
at db2.tam.platform.kernel.impl.ProcessImpl.execute(ProcessImpl.java:182)
at db2.tam.platform.kernel.impl.MonitoredTIMProcess.execute(MonitoredTIMProcess.java:33)
at db2.tam.platform.kernel.impl.Utils.manageSyncProcessing(Utils.java:73)

Kindly guide me on how to remove this value starting with "at" from the events list.

thanks in advance

0 Karma
1 Solution

Revered Legend

Try this

props.conf on indexer/heavy forwarder

[yoursourcetypehere]
..other configs..
SEDCMD-removeat = s/at \S+//g

View solution in original post

0 Karma

Revered Legend

Try this

props.conf on indexer/heavy forwarder

[yoursourcetypehere]
..other configs..
SEDCMD-removeat = s/at \S+//g

View solution in original post

0 Karma

Motivator

Hi Somesoni2 thanks for your effort, actually we are monitoring the identity management logs in splunk and we are using UF agent to get the events from remote machine to the splunk indexer. so in this case were do we want to configure the props.conf details in indexer or Heavy forwarder. And also do we need to configure transforms.conf along with props.conf

Props.conf : Is the below stanza correct
[sourcetype=IBM:AUT:TAM]
SEDCMD-removeat = s/at /s+//g

kindly guide me on this.
thanks in advance

0 Karma

Revered Legend

A UF is not heavy forwarder (Splunk Enterprise instance) so the configurations should go to Indexers. There are two methods, one involves transforms.conf and one (this one) doesn't so you don't need a transforms.conf with this method.

This should be your configuration in props.conf (no prefix for sourcetype and it's upper case S in the regex).

[IBM:AUT:TAM]
SEDCMD-removeat = s/at /S+//g
0 Karma

Motivator

Hi Somesoni2, I have tried the above stanza inprod environment but it is not working, we could still see the huge log details getting into splunk.

Props.conf details: Placed at Indexer instances.

[ibm:auth:identitymanagement]
SEDCMD-removeat = s/at /S+//g

Log details :

[2017-05-24T09:21:31.473-04:00] [itm_server1] [ERROR] [] [com.xxxxx.tam.itm.plugins.eventhandlers] [tid: [ACTIVE].ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: xxxxx] [ecid: ce05a10ae6311cf9:15640c05:15c2c928226:-8000-00000000000b19be,1:123289:21] [APP: itm#11.1.2.0.0] [J2EE_APP.name: itm_11.1.2.0.0] [J2EE_MODULE.name: workflowservice] [WEBSERVICE.name: CallbackService] [WEBSERVICE_PORT.name: CallbackServicePort] Exception Occurred.[[
ibm.tam.identity.exception.AccessDeniedException: tam-3054101:The logged-in user itminternal does not have viewSearchEntity permission on Role xxxxx Inactive itm
Users entity.:itminternal:viewSearchEntity:Role:xxxxx Inactive itm Users
at ibm.tam.identity.rolemgmt.impl.RoleManagerCommon.hasAccess(RoleManagerCommon.java:401)
at ibm.tam.identity.rolemgmt.impl.RoleManagerCommon.hasAccess(RoleManagerCommon.java:251)
at ibm.tam.identity.rolemgmt.impl.RoleManagerImpl.getDetails(RoleManagerImpl.java:531)
at ibm.tam.identity.rolemgmt.impl.RoleManagerImpl.getDetails(RoleManagerImpl.java:492)
at sun.reflect.GeneratedMethodAccessor4906.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)

kindly guide me how to remove the word starting with "at" from the events list from above actual log details.

0 Karma

Revered Legend

You're using foward slash before S+ where it should be backward slash, like in my answer.

0 Karma

Motivator

thanks somesoni2, can I update the below stanza will it fix the issue.

[ibm:auth:identitymanagement]
SEDCMD-removeat = s/at \S+//g

0 Karma

Revered Legend

It will not correct the data that is already ingested. It will do it for any new data that will come.

To validate where it works or not, (assuming you've some data that was ingested with those lines NOT removed), try like this

base search to select current full data 
| rex mode=sed "s/at \S+//g"

The search result should've those lines removed.

0 Karma

Motivator

hey somesoni2, it worked but at the same time we could see some space between the events now and the word "at" is removed. Now the events look like this

log details :
[2017-05-24T12:40:55.515-04:00] [ibm_server1] [ERROR] [] [ibm.iam.platform.kernel.impl] [tid: [ACTIVE].ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: xxxxxxx] [ecid: 061bf930d1319f9b:-30072ed4:15c2c8bbbd3:-8000-000000000009f634,0] [APP: ibm#11.1.2.0.0] [J2EE_APP.name: spml-dsml] [J2EE_MODULE.name: spmlws] [WEBSERVICE.name: ibmProvisioning] [WEBSERVICE_PORT.name: ibmProvisioningPort] Kernel Information: {0}[[
ibm.tam.platform.kernel.ValidationFailedException: IAM-3030006:The following password policy rules were not met:Password must not be one of 8 previous passwords.
:

 Source)


 Source)

 Source)
 Source)




     Method)

Caused By: ibm.tam.passwordmgmt.exception.InvalidPasswordException

kindly guide me on how to remove the space and the words from the events.

0 Karma

Revered Legend

Try this

 base search to select current full data 
 | rex mode=sed "s/at .+//g"
0 Karma

Motivator

somesoni2, can I update above regx to the stanza in props.conf, as the user wants the data not to be injected into splunk. User wants only the events not the content shown in " show all 124 lines details " so in this case can I update the props stanza with above regex mentioned for a base search

Props.conf details :
[ibm:auth:identitymanagement]
SEDCMD-removeat = s/at \s .+//g

Log details :
5/24/17
12:40:55.515 PM
[2017-05-24T12:40:55.515-04:00] [itm_server1] [ERROR] [] [ibm.tam.platform.kernel.impl] [tid: [ACTIVE].ExecuteThread: '22' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: xxxxxx] [ecid: 061bf930d1319f9b:-30072ed4:15c2c8bbbd3:-8000-000000000009f634,0] [APP: itm#11.1.2.0.0] [J2EE_APP.name: spml-dsml] [J2EE_MODULE.name: spmlws] [WEBSERVICE.name: itmProvisioning] [WEBSERVICE_PORT.name: itmProvisioningPort] Kernel Information: {0}[[
ibm.tam.platform.kernel.ValidationFailedException: tam-3030006:The following password policy rules were not met:Password must not be one of 8 previous passwords.
:

Show all 124 lines

Kindly guide me on this as user needs only the events content not the data which are under "show all 124 lines details"

thanks in advance

0 Karma

Revered Legend

Above regex will remove everything after the at (removed text won't be ingested). so yes, you can put above regex into props.conf (on indexer/heavy forwarder).

0 Karma

Motivator

somesoni2 thanks for your effort on this, could please correct me if the below stanza can be updated in props.conf in indexer instances.

Props.conf details :
[ibm:auth:identitymanagement]
SEDCMD-removeat = s/at \S .+//g

thanks in advance.

0 Karma

Revered Legend

Use this (exacly) s/at .+//g

0 Karma

Motivator

sure, thanks. I will update the below stanza in props,conf and validate it .

Props.conf details :
[ibm:auth:identitymanagement]
SEDCMD-removeat =s/at .+//g

0 Karma

Motivator

somesoni2 , we could see some of the events are still showing the blank space after updating the below props stanza in indexer instances.

Props.conf details :
[ibm:auth:identitymanagement]
SEDCMD-removeat =s/at .+//g

Log details:

5/24/17
3:39:30.000 PM

WatchData: DATE = May 24, 2017 3:39:30 PM EDT SERVER = itm_server1 MESSAGE = [ServletContext@1479415291[app:ibm.iam.console.identity.self-service.ear module:identity path:/identity spec-version:2.5 version:V2.0]] Root cause of ServletException.
java.lang.AssertionError: Assertion violated

SUBSYSTEM = HTTP USERID = SEVERITY = Error THREAD = [ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)' MSGID = BEA-101017 MACHINE = hit01.xxxxx.com TXID = CONTEXTID = 061bf930d1319f9b:-30072ed4:15c2c8bbbd3:-8000-00000000000a715f TIMESTAMP = 1495654770087

WatchAlarmType: AutomaticReset
WatchAlarmResetPeriod: 30000

Kindly help me to remove the space from the events t which are under "show all lines details.

0 Karma

Motivator

Hi Somesoni2, thanks for your much need help on removing the word at and the white space from the event content, though the above regex helped to remove most of the events contained the letter at and the white space but still we could see some of the events are having the white space under the "show all lines details" as shown on below event data.

Log details:
5/25/17
2:10:00.487 AM

[2017-05-25T02:10:00.487-04:00] [itm_server1] [ERROR] [] [XELLERATE.AUDITOR] [tid: [ACTIVE].ExecuteThread: '12' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: itminternal] [ecid: 0000Lkg^grQ3JBc5TjO5yc1P8V12000002,0] [APP: itm#11.1.2.0.0] Processor (com.thortech.xl.audit.auditdataprocessors.UserProfileRDGenerator) failed to process successfully generated audit data[[
com.thortech.xl.audit.exceptions.AuditDataProcessingFailedException: Failed to insert form profile record in table UPA_UD_FORMS

Caused by: com.thortech.xl.orb.dataaccess.tcDataAccessException

... 39 more

Caused by: java.sql.SQLIntegrityConstraintViolationException: ORA-02291: integrity constraint (PRD_itm.FK_UPA_UD_UPA_RES) violated - parent key not found

... 41 more

]]
Collapse

kindly guide me on removing the space from events under the "show all lines details"

thanks in advance.

0 Karma

Revered Legend

Can you paste both original event (full) and what Splunk is showing? May be a screenshot?

0 Karma

Revered Legend

Try adding additional SEDCMD for spaces. LIke this

[ibm:auth:identitymanagement]
SEDCMD-removeat =s/at .+//g
SEDCMD-removespaces = s/\s+\s+[\r\n]//g
0 Karma

Motivator

Somesoni2, the above Regex worked and now we are not seeing any white space in the events. thanks a lot for guiding me on this.

0 Karma

Motivator

Somesoni2 kindly guide me on how to remove a specific string from an event in splunk.

thanks in advance.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!