Splunk Search

Possible to retrofit props.conf file


I have a situation in which Cisco Sourcefire files are being ingested into Splunk (v6.0.1) under different sourcetypes into one index (sourcefire), and whose fields are being extracted at search time.

Here's what the props.conf file looks like for one of the sourcetypes:

EXTRACT-Access_Policy = [^]\n]](?P<Access_Policy>[^ ]+)
EXTRACT-Access_Policy-Connection_Type = [^]\n]
](?P<Access_Policy>[^ ]+)[^:\n]:\s+(?P<Connection_Type>[^,]+)
EXTRACT-User = (?i) URL: (?P<User>[^,]+)
EXTRACT-Client = (?i) URL: (?P<Client>[^,]+)
EXTRACT-Application_Protocol = (?i) URL: (?P<Application_Protocol>[^,]+)
EXTRACT-Web_App = (?i) URL: (?P<Web_App>[^,]+)
EXTRACT-Access_Control_Rule_Name = (?i) Name: (?P<Access_Control_Rule_Name>[^,]+)
EXTRACT-Access_Control_Rule_Action = (?i) Action: (?P<Access_Control_Rule_Action>[^,]+)
EXTRACT-Access_Control_Rule_Reason = (?i) Reasons: (?P<Access_Control_Rule_Reason>[^,]+)
EXTRACT-URL_Category = (?i) URL: (?P<URL_Category>[^,]+)
EXTRACT-URL_Reputation = (?i) Reputation: (?P<URL_Reputation>[^,]+)
EXTRACT-URL = (?i) URL: (?P<URL>[^,]+)
EXTRACT-Interface_Ingress = (?i) Ingress: (?P<Interface_Ingress>[^,]+)
EXTRACT-Interface_Egress = (?i) Egress: (?P<Interface_Egress>[^,]+)
EXTRACT-Security_Zone_Ingress = (?i) URL: (?P<Security_Zone_Ingress>[^,]+)
EXTRACT-Security_Zone_Egress = (?i) .
?: (?P<Security_Zone_Egress>\w+/\w+)(?=,)
EXTRACT-SI_Matching_IP = (?i) IP: (?P<SI_Matching_IP>[^,]+)
EXTRACT-SI_Category = (?i) Name: (?P<SI_Category>[^,]+)
EXTRACT-srcip = (?i){.?} (?P<srcip>\d+.\d+.\d+.\d+)(?=:)
EXTRACT-src_port = (?i)^(?:[^.]
.){6}\d+:(?P<src_port>[^ ]+)
EXTRACT-dst_ip = (?i)^[^>]>\s+(?P<dst_ip>[^:]+)
EXTRACT-dst_port = (?i)^(?:[^.]

Reviewing the Sourcefire log files I see they are delimited via a comma with headers, some with key/values. Here is a sample log file:

[hostip of sourcefire] SFIMS:
[Primary Detection Engine (d9fd69ee-b1c9-11e4-ade9-c9e0fcb0c479)]
Connection Type: Start,
User: Unknown,
Client: Unknown,
Application Protocol: Unknown,
Web App: Unknown,
Access Control Rule Name: Malware,
Access Control Rule Action: Block,
Access Control Rule Reasons: IP Block,
URL Category: Unknown,
URL Reputation: Risk unknown,
URL: Unknown,
Interface Ingress: [NAME-NAME]/[NAME-NAME],
Interface Egress: [NAME-NAME]/[NAME-NAME],
Security Zone Ingress: Unknown,
Security Zone Egress: N/A,
Security Intelligence Matching IP: Source,
Security Intelligence Category: Malware,
Client Version: (null),
Number of File Events: 0,
Number of IPS Events: 0,
TCP Flags: 0x0,
NetBIOS Domain: (null),
Initiator Packets: 1,
Responder Packets: 0,
Initiator Bytes: 66,
Responder Bytes: 0,
Context: [NAME-NAME],
SSL Rule Name: N/A,
SSL Flow Status: N/A,
SSL Cipher Suite: N/A,
SSL Certificate: 0000000000000000000000000000000000000000,
SSL Subject CN: N/A,
SSL Subject Country: N/A,
SSL Subject OU: N/A,
SSL Subject Org: N/A,
SSL Issuer CN: N/A,
SSL Issuer Country: N/A,
SSL Issuer OU: N/A,
SSL Issuer Org: N/A,
SSL Valid Start Date: N/A,
SSL Valid End Date: N/A,
SSL Version: N/A,
SSL Server Certificate Status: N/A,
SSL Actual Action: N/A,
SSL Expected Action: N/A,
SSL Server Name: (null),
SSL URL Category: N/A,
SSL Session ID: 0000000000000000000000000000000000000000000000000000000000000000,
SSL Ticket Id: 0000000000000000000000000000000000000000,
{TCP} x.x.x.x:51645 -> x.x.x.x:443

Is there a way I can retrofit the props.conf file (and I'm confused as I thought I would need to configure data transformations in transforms.conf) to extract fields based on the conf file during search time?

Looking at the transforms.conf examples I see this:

Extract comma-delimited values into fields:

DELIMS = ","
FIELDS = "field1", "field2", "field3"

I would appreciate any help for this - thx

0 Karma


From your example result... I'm guessing that

... | extract kvdelim=":" pairdelim="," 

does what you want on a search. Now if I'm reading the transforms doc correctly, then what you would want on your search head is:


REPORT-kv = sourcefire_kv 


DELIMS = ",", ":"

This should get you most of the way there but there may be other fiddling that you'd want to do as well.

0 Karma


"Is there a way I can retrofit the props.conf file to extract fields based on the conf file during search time?"

That's exactly what those EXTRACT-foo lines in props.conf are supposed to do, so I'm not sure what the question is...?


I was under the assumption I could enter the following:

DELIMS = ","
FIELDS = "Connection Type", "User", "Client" - etc

and the fields will automatically be extracted without the need for regex?


0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...