Lookup Fields not updating in the Datamodel

[poojak2579]

I have built an accelerated datamodel with lookup fields.
There is a report that is scheduled to run everyday to populate the lookup.
The datamodel does not get updated when lookup file is updated. But if I disable acceleration, I can see the fields updated.
Do I have to rebuild the datamodel every time lookup file is updated or it is rebuild automatically?

[woodcock]

When you accelerate a datamodel, it goes through an additional indexing pass that creates index-time fields (in the case of your lookup-created fields, as metadata) and it is cooked into the tsidx as it is now. If you change the lookup, then anything that is cooked after the change will reflect the change but not the stuff already cooked. You can delete your datamodel acceleration and rebuild it, or you can use tstats on the field that is the key into the lookup and run the lookup after you pull stuff out of your ADM with the | lookup command.

[poojak2579]

Thanks for replying.
I thought the dynamic lookup attribute will pick up the value change because splunk automatically rebuild datamodels when it finds that those models are outdated.
Is there any way I can rebuild the datamodel everyday after the lookup is updated?

[woodcock]

What do you mean by dynamic lookup attribute? I am unaware of any such thing.

[woodcock]

How many days of data are you accelerating? If more than a few days, you'll never have it built in enough time to search it before it needs to get rebuilt!

[poojak2579]

I have a summary range of 1 month.
I thought fields in the datamodel are dynamically populated.
Does that mean we can not take benefits of accelerated data model if the lookup is updated every day?
Can we schedule it to rebuld everyday after the lookup is updated?

[woodcock]

You could figure out the REST API that turns off acceleration and the one that turns it back on and then add those 2 calls to the bottom of your search that updates the lookup so it would look like this:

... | outputlookup foo
| appendpipe [|rest/foo/bar/.../accelerate=0]
| appendpipe [|rest/foo/bar/.../accelerate=1]

This assumes that you are accelerating little enough data that it can get rebuilt before you come in to search it. It also assumes that you are not using continuous searches like ES does, because these will all fail and be "gapped" for the duration of the outage.

[poojak2579]

Thanks for the answer.
I will have to ask my administrator because I don't have right to enable or disable acceleration.

[cpetterborg]

I believe that the acceleration is created from the model at the time the data model is created or updated. So if you change the lookup, it changes the acceleration that it creates. Rebuilding the data model is the only way to update the acceleration, unless there is a way to trigger the creation of the acceleration that I am unaware of. Hopefully someone will correct me if I'm wrong, so I'm leaving this as a comment for now.

[poojak2579]

Thank you.