On the provider side of healthcare integration, HL7 (particularly v2) is a critical message type to understand. While it is standardized and heavily used by various EHRs/EMRs, it’s used in slightly different ways. There are efforts to further standardize and normalize its use across the board (such as with v3/FHIR), many EHRs and EMRs continue to use 2.x messages. Common HL7 messages include admissions/transfer/discharge (ADT), scheduling (SIU), lab orders and results (ORU, ORM), and medical reports (MDM). Choosing the right platform can be challenging.
Some of the challenges of HL7 2.x messages include:
The ability to add non-standard custom segments or additional data anywhere in the message (whether they are completely custom “Z Segments” or other segments that aren’t typically part of the message, such as IN1 segments in an ADT message to include additional insurance information).
A myriad of parsing and manipulation libraries…
This error is typically harmless, but can result when the XML Disassembler encounters an empty Envelope message instance that’s formatted like this:
instead of this:
BizTalk chooses to make a semantic difference between these two instances, and process the second one fine (publishing no messages), but raising an exception like this for the first:
There was a failure executing the response(receive) pipeline: “…” Source: “XML disassembler” Send Port: “…” URI: “…” Reason: Unexpected event (“eos”) in state “processing_header”.
This can happen particularly when using an XmlProcedure or XmlPolling from SQL – if the resultset is empty, the adapter will publish this message. While this behavior may be desirable (and can frequently be avoided by ensuring you have good Data Available statements on your polling ports, and only call XmlProcedures with valid parameters/at valid times), it can also generate a lot of alarming errors….
MSDN provides an example of INSERTing large data into SQL Server, leveraging the WCF-SQL adapter’s built in FILESTREAM capabilities. However, it’s also possible to leverage the transaction enlisted by the WCF adapter in a custom pipeline to pull FILESTREAM data out of SQL Server more efficiently than the more common SELECT … FOR XML query which simply grabs the FILESTREAM content and stuffs it into an XML node in the resulting document.
Imagine, for example, you had a large document of some sort (XML, Flat File, etc.) to store in SQL Server that BizTalk would need to process from a FILESTREAM table defined like so:
The tradeoff here is losing the typed XML data in favor of more efficient storage and access to larger file objects (especially when the data will, on average, be large). This can make a vast difference if you have…
With the ESB Toolkit, BizTalk provides an excellent framework for handling exceptions that occur throughout the ESB. There are many built in facilities that are as simple as checking off a box to route failed messages to the portal, and within orchestrations you can easily build ESB Exception messages in catch blocks and route them to the portal as well.
However, these only apply if a message actually makes it to a pipeline or orchestration. For WCF SQL Polling receive locations, it’s possible that no message will ever make it to the pipeline – for example, if the procedure causes an exception to occur (perhaps by a developer intentionally using THROW or RAISERROR), the adapter will write the exception to the event log without providing a message for any pipeline or orchestration processing. Checking “suspend message on failure” doesn’t offer any…
I was working on a message flow that involved routing the response from a Request-Response send port directly to another send port. The ultimate send port had failed message routing turned on so that messages would get properly routed by the ESB ToolKit to the EsbExceptionDb for further analysis or processing. In this particular case, the send port was an SFTP port with a misconfigured username/password combination, which would result in failed messages. However, when the message failed I’d get the following exception in the EventLog:
Value cannot be null.
Parameter name: guid
Method: Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext, Microsoft.BizTalk.Message.Interop.IBaseMessage)
Error Source: Microsoft.Practices.ESB.ExceptionHandling
Error TargetSite: System.String GetApplication(ServiceType, System.String, System.String, System.String, System.String)
Error StackTrace: at Microsoft.Practices.ESB.ExceptionHandling.Utility.GetApplication(ServiceType serviceType, String mgmtDBServer, String mgmtDBName, String guid, String serviceName)
at Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.ProcessFault.WriteHeaderFailedMessageRouting(XmlTextWriter writer, IBaseMessage pInMsg, Object& portName)
at Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.ProcessFault.WriteHeader(XmlTextWriter writer, XmlTextReader reader, FaultSource faultSource, IBaseMessage pInMsg, Object& portName)
at Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.ProcessFault.Execute(IPipelineContext pContext, IBaseMessage…
An integration scenario requires a unique incrementing numeric identifier to be sent with each message (or each record in a message). These identifiers cannot be reused (or cannot be reused over certain ranges, or cannot be reused over certain periods of time). A GUID is not suitable because it will not be sequential (not to mention that many legacy systems and data formats may have trouble handling a 128 bit number!).
Integration platforms will have a hard time meeting this on their own – GUIDs work well because they guarantee uniqueness on the fly without needing to worry about history. Messaging platforms typically deal in terms of short executions, and BizTalk is no exception. While persistence of a message might be handled (such as BizTalk does with the MessageBox), persistence of the entire execution process is usually not guaranteed. Deployments,…
Aggregation is a common pattern used in Enterprise Integration. System (or systems) A sends many messages that System B expects in a single message, or in several messages grouped on a particular attribute (or set of attributes).
The most common way to approach this in BizTalk is using a Sequential Convoy orchestration to aggregate the message – Microsoft provides a sample of how to do this in the SDK. This is a powerful pattern, but has a few downsides:
Sequential Convoys can become difficult to manage if they’re expected to run for a long time
Complex subgrouping can multiply the headaches here – for example, if you have to aggregate messages for hundreds of destinations concurrently
The destination message may become very large, to the point where BizTalk cannot optimally process it anymore – particularly if it is a large flat file message.
Modifying the aggregate…
Databases are very frequently at the heart of an enterprise integration. EAI tasks frequently involve polling databases, calling stored procedures in databases, as well as ETL and basic CRUD work on databases. The database backing a particular application is often a natural source to integrate with – and if no on premise database exists for a particular source, creating one for local OLTP purposes can help increase insight and decrease chattiness between on premises and off premises applications.
It’s no wonder then that both BizTalk and Mulesoft ESB offer database connectivity out of the box. In this post, I’ll compare Mule and BizTalk’s SQL capabilities, primarily focusing on Oracle and SQL Server. Why these two? They’re the most popular enterprise grade database engines. MySQL is also very popular, but lacking in some enterprise features that this blog will be examining. PostgresSQL…
Microsoft Azure BizTalk Services (MABS) has a lot to offer for companies looking for a PaaS Middleware solution. EAI bridges provide B2B communication as well as LoB access functionality for EDI, XML, and flat file interchanges. The new mapping system offers some exciting and powerful new functionality as well, vastly simplifying certain tasks that previously required several functiods, and opening up new possibilities and enhanced performance with lists.
However, it is a new technology, and certain tasks that have been very straightforward in BizTalk Server 2013 require a different way of thinking for MABS. For example, it is a fairly trivial task to create an orchestration that accesses a LoB adapter (using, for example, WCF slqBinding) to do data validation or enhancement, and publishing this orchestration as a web service for client consumption. If your SQL database is SQL Azure, there…
The ESB Toolkit 2.1 Exception Handling Framework provides the ability to easily report and and alert on both routed failed messages (where Failed Message Routing is enabled on a Receive or Send Port), as well as ESBFaults created from exceptions within an Orchestration.
However, if you try routing failed messages on a Send Port that is bound to a Logical Port that has Delivery Notification enabled, you’ll get a Send Port failure with the following exception: