Extract fields fomr Python Dictionary text file

[Claw]

How do I create an extract that handles variable numbers of fields.

I am generating events that are time stamped Python dictionaries. The list of items are emitted to the event using Python logging where everything in curly brackets was from a single Python dictionary. The dictionary is rendered using built-in rend(d) where d is a dict. The list of items in the {} brackets are variable, and the order of items is arbitrary.

2013/07/18 19:51:00.090 UTC [DssSplunkHostAgent] INFO: {'rtime': 31556.303133832, 'ABI': 32, 'egid': 20000, 'uid': 101, 'mrcv': 0, 'pctcpu': 0.0, 'pctmem': 0.5523681640625, 'args': 'python -W ignore::DeprecationWarning -m DssCore', 'pid': 9452, 'Taskname': 'DssCore', 'start': datetime.datetime(2013, 7, 18, 11, 5, 5, 785854), 'gid': 20000, 'euid': 101, 'fname': 'python', 'time': 1.244510313, 'oublk': 0, 'dmodel': 1, 'inblk': 0, 'ppid': 9289, 'msnd': 0, 'ctime': 256.16}

[Claw]

Here is a very clean answer. It assumes that the sourcetype is called python dictionary. Change this to meet your needs.

The oddball issue here is the "start" field so we set up automated extractions for every thing else using DELIMS and handle the exceptions with regular extracts.

Props.conf

[pythondictionary]
EXTRACT-PD_MessageLevel = (?i)[.?] (?P\w+)(?=:)
EXTRACT-PD_Agent = (?i)^[^[][(?P[^]]+)
REPORT-pythondictionary = PD-Extract
EXTRACT-PD_Message = {(?P.+)(?=})
EXTRACT-start = \'start\':\s(?.*?),\s\'

Transforms.conf

[PD-Extract]
SOURCE_KEY = PD_Message
DELIMS = ",", ":"

Kudos for this solution goes to Dritan Bitincka in Splunk ps.