Archive

FIX message protocol with Splunk

Path Finder

Does anyone have any recommendations of how to use Splunk with FIX trading messages logs and in particular is there anything that understand's / translates FIX tags?

Tags (1)

Path Finder

I'm using translatefix too. To set your expectations: In my experience, the translated fields are not subsequently extracted and indexed. For example, I can search "MsgType=Execution" as a string, but I can't search "MsgType!=Heartbeat" because it's not extracted as a key/value pair. I discussed it with a Splunk Sales Engineer, he had a trick to dump the translated fields back into the raw index (?) but I've lost the notes I took that day (arrrgh!).. I've not had the time or talent to revisit the problem, but I would be grateful for anybody who could.

0 Karma

Path Finder

Found the issue with FIX Log Parser: it turned out to be missing values after the stanza in commands.conf;

[translatefix]
filename = translatefix.py
streaming = true
enableheader = false
retainsevents = true

0 Karma

Path Finder

Within seach I believe I should just be able to run:

index=test_index Execution* 10:19:37 826 | translatefix

To convert the above into readable tagged format?

0 Karma

Path Finder

I've installed FIX Log Parsing by Glenn but am not having much joy:

20/05/2013 10:19:37.826 2013-05-20 10:19:37,826 INFO in.GSFUT_FILCRD - <231 ExecutionReport (8=FIX.4.2\x19=330\x135=8\x149=GSFUT\x156=FILCRD\x1142=FUSNYQAC\x157=A396051\x134=231\x152=20130520-09:19:37\x137=FUSNYQAC15120130516\x111=10301529\x141=10301523\x117=F5193780920130520\x120=0\x1150=4\x139=4\x11=C0795408\x163=0\x155=HCK3\x148=HCEIK3\x122=5\x1167=FUT\x1200=201305\x154=1\x138=13\x140=1\x115=HKD\x159=0\x147=A\x132=0\x131=0\x130=XHKF\x1151=0\x114=0\x16=0\x175=20130516\x160=20130520-09:19:37\x1120=HKD\x121=3\x110=255\x1)

0 Karma