All Apps and Add-ons

Proofpoint Protection Server TA for Splunk: How do I get this field extracted from my sample event?

cumbers
Explorer

Hi all,

I am looking at using the Proofpoint Protection Server TA for Splunk, and having set it up, I am having some difficulty with field extraction in that the app is not doing what I expect.

2016-08-14T08:00:01.774397+01:00 dc1-pro-prp03 filter_instance1[7090]: rprt s=24sr7mgd5h mod=session cmd=disconnect module= rule= action= helo=<redacted_host> msgs=1 rcpts=1 routes=allow_relay,default_inbound,internalnet,outbound  duration=0.264 elapsed=0.547

I was hoping that Splunk would extract s=24sr7mgd5h as a field named s and a value of 24sr7mgd5h. This would then allow me to run transaction commands and get useful session data from the devices.

I see that this answer https://answers.splunk.com/answers/86461/search-proofpoint-logs.html shows a Splunk user using the s field in their transaction. I am wondering if they have done some Splunk magic to make this happen.

I have found that adding the following will give me what I need, but I am hoping to avoid having to have this for all searches:

| extract pairdelim=" ",kvdelim="=\,"

If anyone can help me with their Splunk Ninja skills, I would be very much appreciative!

0 Karma
1 Solution

Masa
Splunk Employee
Splunk Employee

In general Splunk behavior, s field will be extracted. When an app or Add-on would like to have their own custom field extractions in order to avoid unexpected fields populated by Splunk's auto field extraction. In such case, they use "KV_MODE = none" in props.conf. You can change it to KV_MODE = auto and see how it works.

However, for more detail of the reason why the developer of the add-on disabled or potential issue, I recommend to contact author of the add-on.

View solution in original post

eckolp2003
Path Finder

Proofpoint now has a beta app that will allow you report on and visualze your Proofpoint Protection Server and TAP data! Check out the new app here:

https://splunkbase.splunk.com/app/3727/#/details

Be sure to follow the instructions listed in the details to get all the needed TA's etc that the app needs to work correctly.

0 Karma

eckolp2003
Path Finder

The latest TA (1.06) has this functionality now. Search using: message_session_id=(your session ID you want results from)

This is a field extraction for session ID and will automatically group all of the logs into one transaction result from your query if you format the query like this example:

message_session_id=2b1wdr84m3 | transaction maxpause=3s

eckolp2003
Path Finder

Hello world,
Due to naming collisions, KV_MODE was turned off. It is best to create your own field extractions as needed for now.

For the scenario you are working with now, you can accomplish it like this:

  1. Go To:
    Fields » Field extractions

  2. Click New to create a new Field Extraction

  3. Fill in these details:
    Destination app: TA_pps
    Name: sid
    Apply to: sourcetype
    Named: pps_filter_log
    Type: Inline
    Extraction/Transform: \s+s=(?P[^ ]+)

***** you need to add in < sid >(without any spaces) between P and [ above.... It is not allowing me to past this in without changing the formatting. *******

  1. You may need to adjust the permissions depending on your setup. Make it read/write and available to all apps if you are not sure.

  2. Run a search like this to see all the processing details for a session ID:
    sid="254qq8142p"|transaction maxpause=2s

ppablo
Retired

Hi @cumbers

I noticed you upvoted the answer by @Masa. If his answer solved your issue, don't forget to resolve the post by clicking "Accept" directly below his answer. Thanks!

Patrick

0 Karma

cumbers
Explorer

Yes indeed, apologies. I was on a dodgy cell connection, and was unable to click the button. All done now 🙂

0 Karma

Masa
Splunk Employee
Splunk Employee

In general Splunk behavior, s field will be extracted. When an app or Add-on would like to have their own custom field extractions in order to avoid unexpected fields populated by Splunk's auto field extraction. In such case, they use "KV_MODE = none" in props.conf. You can change it to KV_MODE = auto and see how it works.

However, for more detail of the reason why the developer of the add-on disabled or potential issue, I recommend to contact author of the add-on.

cumbers
Explorer

Thank you! I am going to ask the author (ProofPoint) why they did this, as I can't see a good reason. I'll post back here once I have an answer!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...