Changes (regression?) to export activity audit logging between Splunk versions.

[D77]

In Splunk v7 we used to search index=_internal to find events that contained GET AND "/results/export?output" 

This provided us with information about who had performed the export, in which App, how long the download took, bytes downloaded and which fields had been download, etc
In Splunk 9 we have had to switch to searching index=_internal for POST and "/results/export". From reading it looks like GET has been deprecated, not that the HTTP method really matters to us. However the appended list of fields that were downloaded with the export activity no longer appears to be generated. For us this is an unfortunate regression as it limits our visibility to quickly assess what has been downloaded without needing to re-generate the search.
It would be useful to understand why the downloaded field set is no longer part of the export event. And whether there is something we can potentially configure to enable this to be visible in the events again.
On an aside we feel that these events would be better located in the index=_audit as the index=_internal rolls pretty quickly, so if not constantly forwarding elsewhere visibility of such export activity is transient (noting that there is also an export event generated in _audit but we find that log event of less utility in compliance checks).

[PickleRick]

I wouldn't call that a regression. A new API version uses POST request and therefore uses the POST parameters instead of the pathinfo or whatever it's called. It's not that the old method explicitly logged those parameters, it's just that they were a part of the URI which was logged in the normal access.log. You could try raising HTTP server logging level but you might drown in the amount of data.

Also I disagree about the _audit index. If you want those specific events to have different retention period, just route them to another index. But they are not as such audit events. Audit index contains information about a user invoking actions requiring a specific privilege. The quality of Splunk's audit logs can be of course discussed but just because you want to know who exported which fields, doesn't mean that that particular action is a general audit event.

[D77]

@PickleRick  - thanks for your reply.

It appears we have differing thoughts on this. If I lose something that has provided me utility in the past, I then to think or that as a regression.
Export events in my view are things an organisation should most care about from an intellectual property and security perspective.
We do actually forward pertinent events from _internal (including these exports events) to a collection engine for long term storage. However the export events in v9 are now less useful to us. Additionally, not every organisation would forward their events so having the audit that really counts in _audit still seems prudent to me.
I don't think that my desire for index=_audit to be a historical record for auditing and forensic purposes is excessive. 

[PickleRick]

Export is just a "technical" aspect of presenting the data. The main thing is that the user has searched the data in the first place.

Also, as I understand it, _audit is mostly about "points of decision" whether the user has the right to do something or not. See https://help.splunk.com/en/splunk-enterprise/administer/manage-users-and-security/9.2/audit-activity...

Exporting data AFAIR has no capability specific for this particular action so it's not auditable at this moment.

[isoutamo]

As said the reason why those aren’t logged anymore into internals, is how GET and POST methods works. You can read more e.g. from https://www.w3schools.com/tags/ref_httpmethods.asp

[livehybrid]

Hi @D77 

In recent version there was an update to the API spec and there’s now a series of “v2” endpoints which for export will use POST Instead of GET as you have found. 
It’s worth raising your concerns about the regression with Splunk Support and maybe also add an idea to ideas.splunk.com

In the meantime, you could try and change the logging of the ExportProcessor from Error to Info or debug to see if this provides more info (currently away from my laptop so cannot check):

https://yourSplunkInstance/en-US/manager/launcher/server/logger/ExportProcessor?action=edit&f_search...

let me know how you get on with it. 

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful

• Marking it as the solution if it resolved your issue

• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing.

[D77]

@livehybrid - Thanks for your response. I have passed to our engineering team to test.

I have previously added my thoughts to a splunk idea in the same thread (rather than create a new idea):
https://ideas.splunk.com/ideas/EID-I-1964
But it doesn't appear to get much traction.

I have however appended my new findings  to that idea so the concerns are not lost.