topic Re: Why Can't I use a datamodel backwards? in Reporting

Why Can't I use a datamodel backwards?

snoobzilla — Tue, 26 Aug 2014 01:25:53 GMT

Error in 'SearchParser': The datamodel command can only be used as the first command on a search

Ok... more of theoretical discussion here..

Why oh why can't I push events into a data model and see where it lands in the datamodel?

Datamodels look awesome for big picture analytics. However, I am trying to build tools to help classify individual events(errors) through lookups and such so that people at the support desk know exactly what they are looking at.

Essentially we build datamodels to put events into buckets... why can't I put an event through to see which bucket it lands in? If I could throw individual or a small set of events at the at a complex datamodel it would be ideal for this purpose. However it seems like to filter for say an individual username in a datamodel schema I have to run the entire datamodel OR add it to the data model as a constraint? Why can't I pipe into a data model?

Thoughts from fellow Splunkers? Would anyone else find this useful?

Re: Why Can't I use a datamodel backwards?

Ayn — Tue, 26 Aug 2014 06:57:24 GMT

Not sure I follow your exact use-case. A datamodel is not a means of storage, it is a way of representing data already that already exists in your index. This model can then be used by at least pivot and tstats - you can add your filters there. Or, you can do it by adding new constraints in the model itself. I don't know what you mean by "running the whole model" - a regular search with a username constraint like "... username=foo" isn't looking at all the data in the timerange, it only grabs data which matches the constraint. It's the same with data models.

Perhaps if you elaborated a bit more on your exact use-case it would be possible to post a more meaningful response.

Re: Why Can't I use a datamodel backwards?

snoobzilla — Tue, 26 Aug 2014 08:37:17 GMT

Use case: Build a complex data model to bucket poorly standardized logs into meaningful buckets of distinct use cases for errors. Add a lookup of what it means. People at the service desk could use this info when they get a call. Build a form to allow them to do a search like...

username=foo earliest=now latest=-12h | datamodel complexdatamodel clienterrors search | fields _time username WhatThisMeans WhatToDoAboutIT WhenWillItBeFixed

They would know what we know without any expertise. Right now it looks like what I would have to do is run the entire datamodel and then search the results....

| datamodel complexdatamodel clienterrors search | fields _time username WhatThisMeans WhatToDoAboutIT WhenWillItBeFixed | search username=foo