Extracting field from JSON data

JP — Thu, 08 Jul 2021 22:56:58 GMT

Hi,

New to Splunk, I am trying to create a Test Automation dashboard. I have the following JSON in my SPLUNK events which has come from Jenkins jobs. I would like parse the following fields from this JSON. I tried with spath but did not work. Can someone help?

Classname
Testname
Status
stderr (only the first line)
Nexis.Auto.Environment

{ [-]
build_number: 232
build_url: job/Dev/job/gob/TestAutomation/job/Regression/232/
event_tag: build_report
job_name: Dev/TestAutomation/Regression
job_result: UNSTABLE
metadata: { Nexis.Auto.Browser: chrome headless
Nexis.Auto.Environment: CERT1
Nexis.Auto.IsRemote: false
Nexis.Auto.Platform:
Nexis.Auto.Version:
RELEASE_KEY: r1080
SPECFLOW_BUILD_NUMBER:
TEST_FILTER: TestCategory=regression
}
page_num: 13
testsuite: {
duration: 1800.8512
errors: 0
failures: 9
passes: 16
skips: 0
testcase: [
{
classname: HomePage.#()::TestAssembly:TestAutomation
duration: 80.61503
failedsince: 0
groupname: MSTestSuite
skipped: false
status: PASSED
stdout: Given the user is on the L landing page
-> done: GivenTheUserIsOnTheLLandingPage() (1.8s)

And selects the source "CNN Wire" from "Home" page
-> done: GivenSelectsTheSourceFromPage("CNN Wire", "Home") (8.3s)

When searches for "crime"
-> done: GivenSearchesFor("crime") (3.1s)

Then the results page is displayed
-> done: ThenTheResultsPageIsDisplayed() (0.1s)

And the Narrow by filter section is displayed with "CNN Wire"
-> done: ThenTheNarrowByFilterSectionIsDisplayedWith("CNN Wire") (1.0s)

testname: Search results with Source
uniquename: gHomePage.#()::TestAssembly:TestAutomationSearch results with Source and Segment Search, Wire
}
{
classname: TestAutomationSpecs/DisableTransactionalAccessUsers.#()::TestAssembly:g.TestAutomation
duration: 51.68238
errordetails: Timed out after 30 seconds -> no such element: Unable to locate element: {"method":"css selector","selector":"li[class='navitemselected']"}
(Session info: headless chrome=91.0.4472.124)
(Driver info: chromedriver=2.39.562718 (9a2698cba08cf5a471a29d30c8b3e12becabb0e9),platform=Windows NT 6.3.9600 x86_64)
errorstacktrace: OpenQA.Selenium.WebDriverTimeoutException: Timed out after 30 seconds ---> OpenQA.Selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"li[class='navitemselected']"}
(Session info: headless chrome=91.0.4472.124)
(Driver info: chromedriver=2.39.562718 (9a2698cba08cf5a471a29d30c8b3e12becabb0e9),platform=Windows NT 6.3.9600 x86_64)
at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(String mechanism, String value)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElementByCssSelector(String cssSelector)
at OpenQA.Selenium.By.<>c__DisplayClass23_0.<CssSelector>b__0(ISearchContext context)
at OpenQA.Selenium.By.FindElement(ISearchContext context)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(By by)
at OpenQA.Selenium.Support.UI.ExpectedConditions.<>c__DisplayClass19_0.<ElementToBeClickable>b__0(IWebDriver driver)
at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult](Func`2 condition)
--- End of inner exception stack trace ---
at OpenQA.Selenium.Support.UI.DefaultWait`1.ThrowTimeoutException(String exceptionMessage, Exception lastException)
at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult](Func`2 condition)
at LexisL.TestAutomation.Core.WebDriver.WaitUntil[T](Func`2 expectedCondition, Int32 timeoutInSeconds) in D:\BuildAgent\_work\16\s\Core\Originals\WebDriver.cs:line 338
at g.TestAutomation.PageObjects.PageObjects.L.LLandingPage.SelectSecondarySearch(String searchType) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.PageObjects\PageObjects\L\LLandingPage.cs:line 707
at LexisL.TestAutomation.L.Spec.Specs.StepDefinition.NavigationSteps.GivenTheUserIsOnSecondLevelPage(String searchLevel) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\StepDefinitions\LNavigationSteps.cs:line 23
at lambda_method(Closure , IContextManager , String )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecRun.SpecFlowPlugin.Runtime.RunnerTestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.ScenarioCleanup()
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.DTAUser_PowerSearch_Filters_LegalContentResults(String hLCTList, String[] exampleTags) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\Features\LDisableTransactionalAccess.feature:line 44
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.DTAUser_PowerSearch_Filters_LegalContentResults_CasesStatutesAndLegislationAdministrativeAndAgencyMaterialsAdministrativeCodesAndRegulationsLawReviewsAndJournals() in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\Features\LDisableTransactionalAccess.feature:line 34
at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.ExecuteInternal(ITestThreadExecutionContext testThreadExecutionContext)
at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.Execute(ITestThreadExecutionContext testThreadExecutionContext)
at TechTalk.SpecRun.Framework.TestAssemblyExecutor.ExecuteTestNodeTask(TestNode testNode, ITaskExecutor task, TraceEventType eventType)
failedsince: 204
groupname: MSTestSuite
skipped: false
status: FAILURE
stderr: Timed out after 30 seconds -> no such element: Unable to locate element: {"method":"css selector","selector":"li[class='navitemselected']"}
(Session info: headless chrome=91.0.4472.124)
(Driver info: chromedriver=2.39.562718 (9a2698cba08cf5a471a29d30c8b3e12becabb0e9),platform=Windows NT 6.3.9600 x86_64)
OpenQA.Selenium.WebDriverTimeoutException: Timed out after 30 seconds ---> OpenQA.Selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"li[class='navitemselected']"}
(Session info: headless chrome=91.0.4472.124)
(Driver info: chromedriver=2.39.562718 (9a2698cba08cf5a471a29d30c8b3e12becabb0e9),platform=Windows NT 6.3.9600 x86_64)
at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(String mechanism, String value)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElementByCssSelector(String cssSelector)
at OpenQA.Selenium.By.<>c__DisplayClass23_0.<CssSelector>b__0(ISearchContext context)
at OpenQA.Selenium.By.FindElement(ISearchContext context)
at OpenQA.Selenium.Remote.RemoteWebDriver.FindElement(By by)
at OpenQA.Selenium.Support.UI.ExpectedConditions.<>c__DisplayClass19_0.<ElementToBeClickable>b__0(IWebDriver driver)
at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult](Func`2 condition)
--- End of inner exception stack trace ---
at OpenQA.Selenium.Support.UI.DefaultWait`1.ThrowTimeoutException(String exceptionMessage, Exception lastException)
at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult](Func`2 condition)
at LexisL.TestAutomation.Core.WebDriver.WaitUntil[T](Func`2 expectedCondition, Int32 timeoutInSeconds) in D:\BuildAgent\_work\16\s\Core\Originals\WebDriver.cs:line 338
at g.TestAutomation.PageObjects.PageObjects.L.LLandingPage.SelectSecondarySearch(String searchType) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.PageObjects\PageObjects\L\LLandingPage.cs:line 707
at LexisL.TestAutomation.L.Spec.Specs.StepDefinition.NavigationSteps.GivenTheUserIsOnSecondLevelPage(String searchLevel) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\StepDefinitions\LNavigationSteps.cs:line 23
at lambda_method(Closure , IContextManager , String )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecRun.SpecFlowPlugin.Runtime.RunnerTestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.ScenarioCleanup()
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.DTAUser_PowerSearch_Filters_LegalContentResults(String hLCTList, String[] exampleTags) in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\Features\LDisableTransactionalAccess.feature:line 44
at g.TestAutomation.L.Specs.Specs.L.Features.LLDisableTransactionalAccessUsersFeature.DTAUser_PowerSearch_Filters_LegalContentResults_CasesStatutesAndLegislationAdministrativeAndAgencyMaterialsAdministrativeCodesAndRegulationsLawReviewsAndJournals() in c:\jenkins\workspace\Dev\g\TestAutomation\g_L_Specflow_Build\g.TestAutomation.L.Specs\Specs\L\Features\LDisableTransactionalAccess.feature:line 34
at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.ExecuteInternal(ITestThreadExecutionContext testThreadExecutionContext)
at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.Execute(ITestThreadExecutionContext testThreadExecutionContext)
at TechTalk.SpecRun.Framework.TestAssemblyExecutor.ExecuteTestNodeTask(TestNode testNode, ITaskExecutor task, TraceEventType eventType)

stdout: Given the user is a DTA user with "legal" content
-> done: LDTAuserSteps.GivenTheUserIsADTAUserWithContent("legal") (0.0s)
-> [Fatal] Timed out after 30 seconds
-> [Fail] Current Web Driver URL: https://cdc1-signin.lexisL.com/lnaccess/AuthzDenied?aci=la
-> [Fail] Screen shot saved to: ScreenShots/LLDisableTransactional/DTAUserPowerSearchFilters_20210708_023332.png
-> [Fatal] Timed out after 30 seconds
-> [Fail] Current Web Driver URL: https://cdc1-signin.lexisL.com/lnaccess/AuthzDenied?aci=la
-> [Fail] Screen shot saved to: ScreenShots/LLDisableTransactional/TransactionalUserPowerSearchLegalContent_20210708_023333.png
-> [Fatal] Timed out after 30 seconds
-> [Fail] Current Web Driver URL: https://cert1-advance.lexis.com/search/?pdmfid=1519360&crid=32014007-3588-4f08-ac9f-b1dea68bde86&pdsearchterms=crime&pdstartin=urn%3Ahlct%3A16&pdcaseshlctselectedbyuser=false&pdtypeofsearch=searchboxclick&pdsearchtype=SearchBox&pdoriginatingpage=BisLPowerSearch&pdqttype=or&pdpsf=urn%3Ahlct%3A16&pdquerytemplateid=&indexsearch=false&ecomp=_x7hkkk&earg=pdpsf&prid=4739fc21-707f-4e2f-a94f-8858f7c2363b
-> [Fail] Screen shot saved to: ScreenShots/LLNewsletter/CreatenewsletterAddtextblockCancelSave_20210708_023335.png

testname: DTA User-Power Search-Filters-Legal Content Results, Cases;Statutes and Legislation;Administrative and Agency Materials;Administrative Codes and Regulations;Law Reviews and Journals
uniquename: TestAutomation.Specs.L/LDsUsers.#()::TestAssembly:g.TestAutomation.L.DTA User-Power Search-Filter
}
}
]
tests: 25
time: 1800.8512
total: 25
}
user: (timer)
}

Re: Extracting field from JSON data

venkatasri — Fri, 09 Jul 2021 03:19:44 GMT

Hi @JP Can you share the _raw json event, what you have shared is UI well formatted version.

Re: Extracting field from JSON data

JP — Fri, 09 Jul 2021 15:05:48 GMT

Hi @venkatasri

This is how the events look in my Splunk. The one I have is a huge file and I shown sample of a testcase that has failed.

Re: Extracting field from JSON data

JP — Fri, 09 Jul 2021 15:03:59 GMT

This shows the case of a passed test case

I would like to extract details like,

Classname

Testname

status

duration

failedsince

Errordetails etc

topic Extracting field from JSON data in Splunk Search

Extracting field from JSON data

Re: Extracting field from JSON data

Re: Extracting field from JSON data

Re: Extracting field from JSON data