Cannot get data Index in Datamodel

[zksvc]

Hi there, 

I was ingest new server to new index (Ubuntu with UF) 

Let say my index is index=ABC 

I want to connect it in Datamodel, unfortunately im not the first who was create it. And when i check it i got error "This object has no explicit index constraint. Consider adding one for better performance." 

And when i check it in macros `cim_Endpoint_indexes` it only show () 

When i want to add my new index in that macros i got this 500 server error 

According from this question : https://community.splunk.com/t5/Knowledge-Management/Adding-index-to-accelerated-CIM-datamodel/m-p/5... it said 2 solution :

• if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
• if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Since i know it cannot add from macros, i create new Eventtype and Tag for my new index.

And that Eventtype also in Tag like this 

Eventtype	Tag
eventtype=ABC_endpoint_event	tag=endpoint, tag=asset, tag=network
eventtype=ABC_process_event	tag=process, tag=endpoint
eventtype=ABC_network_event	tag=network, tag=communication
eventtype=ABC_security_event	tag=security, tag=endpoint

One from base search in Datamodel Endpoint is using tag=process 

(`cim_Endpoint_indexes`) tag=process tag=report | eval process_integrity_level=lower(process_integrity_level)

From that query it calling tag=process 

But when i try to running it, it don't show my new index. 

Anyone can help me to solving this issue ? 

~Danke

[marnall]

If you do not specify an index= filter in your search, then Splunk will search your role's default indexes, which can be toggled in the role settings. If you have no default indexes or no data in your default indexes, then no results will appear.

The purpose of the cim_Endpoint_indexes macro is to list the indexes from which to find data to populate the data model, so you /should/ be able to list your index filters in there. E.g.
index=ABC
or
index IN (ABC,DEF)

The problem is that your Splunk instance is returning a 500 Internal Server Error when you try to edit the macro. In a working system it would not do that. Can you check the web_service.log to see what is causing the problem?

If you can access the shell of your splunk search head and it is Linux, then the log should be findable at:

/opt/splunk/var/log/splunk/web_service.log

[PickleRick]

Small correction. If you don't define the cim_* macros their contents will of course be empty and while searching without using the accelerated summaries in your ad-hoc or scheduled searches it will use indeed your user's role's default indexes, the datamodel acceleration summary building searches will be spawned with the system user's default indexes which is an empty list. You need to have explicitly defined list of indexes to have CIM acceleration built properly.

[zksvc]

i was download log file in `/opt/splunk/var/log/splunk/web_service.log` and i open with Notepad++ like this

When i search 500 ERROR it showed too much data, could you please give me specify keyword? Because when i want to search macros it not show anything.

Sorry very confuse about it 

[marnall]

Are you able to go to one of those 500 Error lines, then post 10-20 of the previous lines? If it contains sensitive data then you should sanitize it before posting.

[zksvc]

I think there is no sensitive data, so here the full error message

2024-09-23 21:06:30,760 INFO	[66f175e6c17fc8d479ab10] error:337 - 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request.
2024-09-23 21:06:30,760 ERROR	[66f175e6c17fc8d479ab10] error:338 - Traceback (most recent call last):
  File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 628, in respond
    self._do_respond(path_info)
  File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 687, in _do_respond
    response.body = self.handler()
  File "/opt/splunk/lib/python3.7/site-packages/cherrypy/lib/encoding.py", line 219, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/opt/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/htmlinjectiontoolfactory.py", line 75, in wrapper
    resp = handler(*args, **kwargs)
  File "/opt/splunk/lib/python3.7/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "</opt/splunk/lib/python3.7/site-packages/decorator.py:decorator-gen-2042>", line 2, in help
  File "/opt/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 41, in rundecs
    return fn(*a, **kw)
  File "</opt/splunk/lib/python3.7/site-packages/decorator.py:decorator-gen-2040>", line 2, in help
  File "/opt/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 102, in check
    if verify_session and not util.checkRequestForValidFormKey(requireValidFormKey=must_login):
  File "/opt/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/util.py", line 1481, in checkRequestForValidFormKey
    if not isValidFormKey(form_key) or not doesFormKeyMatchCookie(form_key):
  File "/opt/splunk/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/util.py", line 1452, in isValidFormKey
    logger.warn('CSRF form_key mismatch received=%s expected[redacted]=%s' % (key, getFormKey()[-4:]))
TypeError: 'NoneType' object is not subscriptable

Before that i would to say thankyou for reply this post and want to help me

[marnall]

As Picklerick suggests, this may resolve if you clear your cookies and/or cache.

[PickleRick]

OK. It seems to be CSRF-prevention cookie related. Try clearing your browser cache and cookies. Maybe your browser has some invalid cookie stored which it supplies with your requests.

[zksvc]

Hi @PickleRick @marnall 
Thankyou for your advice, but unfortunately i still can't change it even after i clear my cookies and/or cache.

Can this issue solved using another method ?

[marnall]

There are some troubleshooting steps you could try:

1. Use a different browser
2. Try to edit other macros
3. Try to add a new macro
4. Try to edit other knowledge objects, like field extractions, dashboards, etc
5. Make a new user with very high permissions (e.g. admin) and try editing the macro with it
6. Install a new search head, connect it to your indexers, then edit the macro

[PickleRick]

Well, not everything can be solved just by joined community wisdom. Sometimes you simply have to raise support case.