This post is about how to implement Dead Letter Channel Pattern with ESB Pack.
The Dead Letter Channel is well known EAI pattern that solves the problem when a messaging system determines that message cannot be delivered and redirects it to another channel. The default option is provided by ESB Toolkit exception management framework to route failed messages to a central database. To address the problem specified by this pattern with only BizTalk Server, developers typically implement orchestrations to handle ACK and NACK emitted by send ports and implement logic for alternative routing. After solution evolves, it becomes more difficult to maintain orchestration logic for handling message delivery errors especially if alternative flows tend to change over time due new business requirements.
Let’s take an example from insurance industry: the car estimate rating. In this scenario, the estimate is sent to an external system to get a score for the claim routing. When external scoring system fails, claims are assigned for manual review in another system: we ask humans to do it manually. The alternative flow for handling failed estimates can easily get more complex, involving more than one system of record.
Can it be done with BizTalk and/or ESB Toolkit alone? Unfortunately, BizTalk ESB Toolkit does not support starting one or more itinerary instances from an orchestration in the same way as in on-ramp that uses ItinerarySelector pipeline component. In addition, IT operations cannot easily track such scenario at runtime, which makes this case even more complex to maintain and measure in production.
ESB Pack was designed to address these challenges and includes a sample, which is discussed through the remainder of this post. The first itinerary shown below represents the “ideal” flow with services for setting retry settings and routing to non-existent folder using BizTalk FILE adapter.
Note that AdvanceWithAck orchestration service indirectly uses resolver defined in the off-ramp to determine message destination and transport properties before message is being dispatched to the message box database via direct bound send port. It does handle acknowledgements from direct bound port types, which is a bit more complex. This service also has been configured with its own resolver (ResolveNackItinerary), to specify itinerary that implements “Dead Letter Channel” flow. This is done using itinerary resolver extension included in the ESB Pack. After NACK has been received from dynamic send port, this service starts compensating itinerary named "TestNackItinerary" shown below:
This is very simple itinerary that routes message to a specific folder using orchestration routing service from BizTalk ESB Toolkit. After both itineraries are deployed, BizTalk operators can easily track their performance at runtime. The below picture shows main itinerary Grafana dashboard:
As you can see, the itinerary events show that itinerary has been completed twice and one itinerary failure event has been recorded. Two completion events occur because one way dynamic send port completes original itinerary before failed routing takes place. The completion event will not be emitted if this example would use two way send port scenario. The itinerary failure event has been emitted because dynamic send port triggered by the first itinerary has been configured with enabled failed message routing. Thus, simply disabling this setting in dynamic send port would eliminate itinerary failed event and no failed message will be sent to external ESB Toolkit database.
The Grafana dashboard shown below includes full set of events executed for this itinerary instance:
As you can see, ESB Pack allows not only spawn the itinerary from orchestration but also gives developers a choice whether to track it as new instance or continuation of an existing one. Its instances can be also tracked separately as shown below:
The following benefits should be considered for using ESB Pack for BizTalk Server to develop complex flows, including compensation sub-flows for failed messages:
- Itineraries for “Dead Letter Channel” flow can be changed at any point of time without coding as long as you have necessary itinerary services.
- Handling acknowledgements and re-routing becomes reusable while orchestration service is decoupled using direct bound send ports
- Easy test automation using monitoring data without BAM
- Full BizTalk ESB DevOps in production
The ESB Pack has even more powerful features for managing and tracking itineraries to enable cost effective DevOps. If you are interested to know more about ESB Pack for BizTalk Server, please feel free to get in touch.