Access Finding/Investigation notes in ES 8

[ljvc]

Hi there,

we're currently migrating to ES 8 and need to see Work Notes (comments) provided by analysts in some dashboards/reports. Previously, the incident_updates_lookup contained the "comment" field, which held this information, and was easy to access in a search.

With ES 8, this was obviously mentioned as a limitation - "The Comments feature available in prior versions of Splunk Enterprise Security is now replaced by an enhanced capability to add notes."

How can we access those notes (KV Store/Lookup/...) outside of having to click through the Mission Control/Analyst Queue manually? Where are they stored?

[Christopherscha]

I got an other approach. 

It seems that this update events are getting stored into the audit trail . 

So you can find the content of the notes with the search:  

index="_audit" source=mc_notes

br 

[ljvc]

As usual, I figured it out shortly after finally asking.
Notes are kept in the mc_notes collection in the missioncontrol app, if anyone else was wondering...

[randoj]

Good afternoon @ljvc. Could you provide some direction on how you're accessing the mc_notes collection from within the Mission Control app? Struggling to find this.

[ljvc]

Hi @randoj ! 

We just created a lookup definition manually in a local/transforms.conf, as you would with any other KV Store lookup.
Additionally, we needed to do the same for the mc_incidents collection, as it is needed to correlate notable_ids and incident_ids, the latter of which are used in mc_notes.

It probably is easier to access the collections using the Python SDK and scripts, but this solution worked for us and required less setup.

Hope this helps!

[joelschaerhk]

Hi,

Thanks for your help in solving this issue. Here is my solution :

Create the lookup definition in missioncontrol/local/transforms.conf for mc_notes :

[mc_notes_lookup]
collection = mc_notes
external_type = kvstore
fields_list = ai_generated,content,create_time,files,incident_id,last_edited_by,source,source_type,title,update_time,_key

Comments can then be corelated with notables :

`notable`
|eval `get_event_id_meval`,rule_id=event_id
|lookup mc_findings_lookup notable_id AS event_id OUTPUT _key AS incident_id
|lookup mc_notes_lookup incident_id OUTPUT content AS comment

Hope this helps.

[randoj]

@ljvc I appreciate the information, this is helpful. Would you be able to share your transforms.conf files against the mc_incidents and mc_notes collections? I'd like to better understand how that correlation with incident_id is happening between the collections and dumping it into something readable in a lookup.

[ljvc]

@randoj unfortunately, I cannot share the exact files. However, you should be able to get the incident id for each finding using its calculated rule_id (compare the eval statement for rule_id/event_id in [Incident Review - Main] in SA-ThreatIntelligence/default/savedsearches.conf) via the mc_incidents collection, which has a field notable_id iirc. Then, use that id as a key against the mc_notes collection, and you can get notes for findings. Hope this clears things up a bit!

[randoj]

@ljvc I appreciate the information you were able to provide, this is helpful. On a side note I do have an active case open with Splunk support on this topic. Their latest update was that this has been a reported issue, and that they expect it to be addressed in ES 8.2 per an internal JIRA ticket.

[adrezende_splun]

Download Splunk App for Lookup File Editing app. Nn Lookups menu, select All and search for mc_notes. On Actions menu, click the magnifier button to search the mc_notes lookup. A prompt will show up asking you to create a lookup transform. Add the name that you want and click Create transform.

Open a new search and search | inputlookup mc_notes to show mv_notes content.

[hcpr]

Hi, 

Following up on the above discussion, has anyone else discovered that there are quite a few instances where the "incident_id" field is blank in the mc_notes lookup?
The other fields (autor.username, create_time and content) contain the correct information but there is nothing in incident_id.

Makes it a bit difficult to match the note to the corresponding incident 🙂

[TamasDubicz]

For some reason as I see in case of updates of the old events Splunk writes the change into the _audit index. You can access to it by this search

index=_audit source=mc_notes 
| rex "(?<timestamp>[\d.]+),(?<incident_id>[\w-]+),(?<user>[\w_]+),(?<model>[\w]+),(?<command>[\w]+),(?<diff>.+)" 
| eval time=_time 
| table time, user, incident_id, diff, command 
| rex field=diff mode=sed "s/\\\\//g" 
| rex field=diff mode=sed "s/^\"//g" 
| rex field=diff mode=sed "s/\"$//g" 
| spath input=diff

---

[ljvc · Accepted solution]

Hello hcpr,

we did run into the same issue shortly after my previous post, and I forgot to give an update in here.

The app "missioncontrol" exposes a few endpoints to do with incidents and investigations, and tracing the behavior of Enterprise Security when fetching comments led us to its OpenAPI spec which you can find at missioncontrol/mcopenapi.yaml.

The incidents endpoint, when fed a finding/notable ID, will return a list of comments. I recommend everyone to take a look at the requests in your browser's developer tools when interacting with finding notes on the Analyst Queue to see how the endpoint works. Ultimately, we went that way and implemented a custom command to perform the same requests at search time. This is now working flawlessly for us so far, getting even those notes which have no incident_id or source in mc_notes.

Hope this helps!

[swalker]

@ljvc are you able to share any more details on your solution? Struggling to figure out how to get all the comments in a single report. thanks in advance! 

[ljvc]

Hi there! We ultimately implemented a custom command to query the missioncontrol API based on the event_id for each finding. This returns all relevant notes. You can use the usual service object the custom command classes provide to formulate GET requests to the ES API using its service.get() function while passign along information from each event you're looking to report on.

As mentioned above, the best way to understand the necessary requests is to just follow along with the requests contents when using the ES UI with your browser, in addition to the documentation in the missioncontrol app. For custom commands and the service object, check the custom command examples in splunk dev and the splunk SDK documentation respectively.

Hope this helps, best of luck with your implementation!