Monitoring EDI transactions

[pradeepiyer2024]

Hi, this is probably a product related question. I have a requirement to monitor EDI files (834 - Enrolment file in Healthcare terms) end to end. I would like to see number of EDI files received, processed and saved, analyse the file processing failures. Which Splunk product(s) best suits my need?

[tscroggins · Accepted solution]

@pradeepiyer2024,

I would approach this from the perspective of either your EDI/X12 gateway or your payer platform. I only have past experience with TriZetto Facets--and it's been a minute--but in general, the batch jobs themselves should log the output you need. Absent that, your gateway may help you track 834 requests and 999 responses but not failures within the system.

I've personally used Splunk Enterprise and Splunk IT Service Intelligence to track X12 transactions end-to-end, but the best fit depends on the software components in your solution and how those components log transactions and store data.

[gcusello]

Hi @pradeepiyer2024,

probably Splunk Enteprie can help you:

has the tool you're using a log to read? if yes, you can read it using Splunk.

did you enable file logging in your Windows sustem? if yes, youcan read the file system og with Spunk,

your EDI fiels are in plain text, so you could read them with Splunk, even if I don't know how this could help.

Ciao.

Giuseppe

[pradeepiyer2024]

@gcusello 
My intention is not to read or parse the file, instead, to make sure, for example, if 10 EDI files were consumed in a batch, all those 10 files make it to the target database. Part of it would be understand how many failed, failed at which stage etc. 

[PickleRick]

@gcuselloWell, this ain't that easy 😉

While the format itself is relatively easy, it is - to some extent - a structured data. So it's problematic to parse it properly preserving the relationship between entities (there might be - for example - several instances of a "name" or "address" field, each related to another person within the same record.

So it's more complicated than it looks.

Sure, it can be ingested but it's more complicated to parse it properly to not make a big mess of it.

Also @pradeepiyer2024 - what do you mean by "monitor end to end"?

[pradeepiyer2024]

@PickleRick 
End to end means from the moment the files are picked up, to the point they hit the target database. My intention is not to read or parse the file, instead, to make sure, for example, if 10 EDI files were consumed in a batch, all those 10 files make it to the target database. Part of it would be understand how many failed, failed at which stage etc. 

[PickleRick]

That's not a question about Splunk or its products. It's about how your whole process is organized, what, where and how can you monitor and so on. It might not be related to EDI documents at all - just monitoring and logging from whatever procass you have making sure that unique identifier of a EDI document is stored.

It might be a matter of choosing proper Splunk tools for your process already in place and maybe doing some slight adjustments to it. In such case your local Splunk Partner will happily help you choose right products/services for you (might be Splunk Enterprise, might be Splunk Cloud, might be O11y Cloud, depending on what you have and how it's done).

But it might be a bigger consulting project to help you architect the whole process and environment, possibly using Splunk tools.

It's beyond the scope of this forum. You can't just throw in "some Splunk" and hope for the best. If your process is OK, you'll probably fit in one of the solutions, if it's not, there are no wonders - GIGO.