How to parse CEF files with key value pairs that contain white space?

[brent_weaver]

Hello all!

I am a Splunk "newb" when it comes to parsing out files for ingestion. Here is my situation. I have a CEF formatted file, which onto itself is not a problem. Splunk handles that just fine... What is a problem is at the end of lines there are key/value pairs, but the values have white spaces in it and splunk (or better yet I) cannot figure out hot to parse that out. Here is an example:

CEF|ProdManuf|ProdName|Vervion|Timestamp| key1=this is key1 key2=this is key two  key3=another random length string 

So my question is how to I deal with the tail end of this string to get these as full key value pairs. I would assume a regular expression that will capture the pairs.

Thanks!

[dshpritz]

Hey Brent,
Check out my answer here.

You may need to adjust some of the regular expressions, but generally it works pretty well. You may also need to add additional extractions for some edge cases.

Thanks,

Dave

[brent_weaver]

This data will be coming in tailing a file. But as I stated in my post I am currently uploading the file via the web interface for testing.

[dshpritz]

You may want to look into using the oneshot command, something like:
splunk add oneshot "/path/to/host1/file" -index myidx -sourcetype mytype

[brent_weaver]

Dave -

Thank you for taking the time to ans this post. I am new to splunk and cannot figure out how to make this become a sourcetype. For testig purposes I am just taking my CEF file and uploading it through the we portal. I have a TA for imperva that does most of the job except the multi-word key value pairs.

So how do I take the article you referenced and make it a sourcetype that I can apply when reading in the file? Perhaps I am missing an app.conf file?

[dshpritz]

The best place to set the source type is usually on the input. Is this data coming in via syslog? Or is this coming in by tailing a file?