topic Re: How to break event 1 and 2 further? in Getting Data In

How to break event 1 and 2 further?

Sujithkumarkb — Mon, 15 Jul 2019 11:43:13 GMT

Each Realm entry should be an event, JSON is the source.

Event1:

{"realm":"/humapp","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328639","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"Application","info":{"authIndex":"module_instance","authControlFlag":"REQUIRED","moduleClass":"Application","ipAddress":"10.254.110.61","authLevel":"0"}}],"userId":"","principal":["HUMAppAgent"],"timestamp":"2019-07-15T11:29:36.221Z","trackingIds":["25ac5061b64b400902"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328643"}
{"realm":"/humapp","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328639","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"Application","info":{"authIndex":"module_instance","ipAddress":"10.254.110.61","authLevel":"0"}}],"userId":"id=HUMAppAgent,ou=agent,o=humapp,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["HUMAppAgent"],"timestamp":"2019-07-15T11:29:36.235Z","trackingIds":["25ac5061b64b400902"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328647"}

Event2 :

{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.274Z","trackingIds":["6cea414e7a464b4d02"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328624"}
{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.295Z","trackingIds":["6cea414e7a464b4d02"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328628"}

Re: How to break event 1 and 2 further?

Sujithkumarkb — Wed, 30 Sep 2020 01:22:37 GMT

I tried with should_line_merge=false and it works fine on local .But how can i map it to the index and sourcetype on production , as i am unable to find the sourcetype declared in production to update with new config

Re: How to break event 1 and 2 further?

woodcock — Mon, 15 Jul 2019 14:39:41 GMT

You need something like this:

[<your sourcetype here>]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=$|{"realm)

As far as finding the sourcetype declaration, you do not need to do that. Many people have an app called something like global_props or whatever and deploy configurations there. Splunk will merge them all together. You can see this using $SPLUNK_HOME/bin/splunk btool props list --debug.

Re: How to break event 1 and 2 further?

Sujithkumarkb — Tue, 16 Jul 2019 06:46:40 GMT

Hey Woodcock,
Thanks for the response , this works fine unless the each realm event starts from a new line as shown above in the preview . does not work when a new realm event starts on the same line as end of previous realm event line.

{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.274Z","trackingIds":["6cea414e7a464b4d02"],"_id"}{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-

Re: How to break event 1 and 2 further?

Sujithkumarkb — Wed, 30 Sep 2020 01:23:01 GMT

This is how it is indexing now. The first one is fine ,but second event has second half of first event and the half of second event
7/16/19
2:50:39.000 AM
{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411601","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T06:50:38.672Z","trackingIds":["3278ae96d06b64c602"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411607"}

7/16/19
12:28:36.000 PM
{"realm":"/healthcheck","transactionId":"25c79b89-329b-462e-950b-0f75fd67a3ae-72771235","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"LDAP","info":{"ipAddress":"10.223.108.29","authLevel":"0"}}],"userId":"cn=healthcheck01,ou=Users,ou=HealthCheck,ou=external,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T05:57:41.089Z","trackingIds":["bf6f5024a8b7f65f02"],"_id":"25c79b89-329b-462e-950b-0f75fd67a3ae-72771241"}{"realm*":"/healthch*eck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411601","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T06:50:38.653Z","trackingIds":["3278ae96d06b64c602"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411603"}
host = VDI-W10-13270 source = C:\Users\Sujith.Kumarkb.HUGHESTELEMATIC\Desktop\Forgerock.txt sourcetype = forgerock_16july

Re: How to break event 1 and 2 further?

woodcock — Fri, 19 Jul 2019 01:43:12 GMT

It should work for that case.