Getting Data In

How to break event 1 and 2 further?

Sujithkumarkb
Observer

Each Realm entry should be an event, JSON is the source.

Event1:

{"realm":"/humapp","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328639","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"Application","info":{"authIndex":"module_instance","authControlFlag":"REQUIRED","moduleClass":"Application","ipAddress":"10.254.110.61","authLevel":"0"}}],"userId":"","principal":["HUMAppAgent"],"timestamp":"2019-07-15T11:29:36.221Z","trackingIds":["25ac5061b64b400902"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328643"}
{"realm":"/humapp","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328639","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"Application","info":{"authIndex":"module_instance","ipAddress":"10.254.110.61","authLevel":"0"}}],"userId":"id=HUMAppAgent,ou=agent,o=humapp,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["HUMAppAgent"],"timestamp":"2019-07-15T11:29:36.235Z","trackingIds":["25ac5061b64b400902"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328647"}

Event2 :

{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.274Z","trackingIds":["6cea414e7a464b4d02"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328624"}
{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.295Z","trackingIds":["6cea414e7a464b4d02"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328628"}
0 Karma
1 Solution

woodcock
Esteemed Legend

You need something like this:

[<your sourcetype here>]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=$|{"realm)

As far as finding the sourcetype declaration, you do not need to do that. Many people have an app called something like global_props or whatever and deploy configurations there. Splunk will merge them all together. You can see this using $SPLUNK_HOME/bin/splunk btool props list --debug.

View solution in original post

0 Karma

woodcock
Esteemed Legend

You need something like this:

[<your sourcetype here>]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=$|{"realm)

As far as finding the sourcetype declaration, you do not need to do that. Many people have an app called something like global_props or whatever and deploy configurations there. Splunk will merge them all together. You can see this using $SPLUNK_HOME/bin/splunk btool props list --debug.

0 Karma

Sujithkumarkb
Observer

Hey Woodcock,
Thanks for the response , this works fine unless the each realm event starts from a new line as shown above in the preview . does not work when a new realm event starts on the same line as end of previous realm event line.

{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2328622","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-15T11:29:27.274Z","trackingIds":["6cea414e7a464b4d02"],"_id"}{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-

0 Karma

woodcock
Esteemed Legend

It should work for that case.

0 Karma

Sujithkumarkb
Observer

I tried with should_line_merge=false and it works fine on local .But how can i map it to the index and sourcetype on production , as i am unable to find the sourcetype declared in production to update with new config

0 Karma

Sujithkumarkb
Observer

This is how it is indexing now. The first one is fine ,but second event has second half of first event and the half of second event
7/16/19
2:50:39.000 AM
{"realm":"/healthcheck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411601","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T06:50:38.672Z","trackingIds":["3278ae96d06b64c602"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411607"}

7/16/19
12:28:36.000 PM
{"realm":"/healthcheck","transactionId":"25c79b89-329b-462e-950b-0f75fd67a3ae-72771235","component":"Authentication","eventName":"AM-LOGIN-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"LDAP","info":{"ipAddress":"10.223.108.29","authLevel":"0"}}],"userId":"cn=healthcheck01,ou=Users,ou=HealthCheck,ou=external,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T05:57:41.089Z","trackingIds":["bf6f5024a8b7f65f02"],"_id":"25c79b89-329b-462e-950b-0f75fd67a3ae-72771241"}{"realm*":"/healthch*eck","transactionId":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411601","component":"Authentication","eventName":"AM-LOGIN-MODULE-COMPLETED","result":"SUCCESSFUL","entries":[{"moduleId":"DataStore","info":{"authControlFlag":"REQUIRED","moduleClass":"DataStore","ipAddress":"10.254.110.18","authLevel":"0"}}],"userId":"id=healthcheck01,ou=user,o=healthcheck,ou=services,dc=openam,dc=verizontelematics,dc=com","principal":["healthcheck01"],"timestamp":"2019-07-16T06:50:38.653Z","trackingIds":["3278ae96d06b64c602"],"_id":"d9d6ba4e-c3bb-416e-b81b-3eb3afb3737a-2411603"}
host = VDI-W10-13270 source = C:\Users\Sujith.Kumarkb.HUGHESTELEMATIC\Desktop\Forgerock.txt sourcetype = forgerock_16july

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...