Migration Challenges in BizTalk Migration
Integration migration projects are humbling in their complexity and over-whelmingly communication-intensive. You are more likely to run into communication, management and operational challenges which can conspire to defeat your march than any serious technical challenges. An adaptive, agile approach where assumptions are questioned daily and where close anticipation and planning discipline is instilled has a greater chance of success.
Here are pointers and advice about the challenges of an EAI migration project – right from the trenches.
Project Management lessons
1. First, if the “Replicating Behavior” migration approach is chosen, a pilot implementation of an interface would be beneficial. This allows learning a lot about the application scope, get the “big picture”, peek into many technical details and eccentricities early, and foresee upcoming challenges – not to mention planning ahead for infrastructure and performance. This Proof Of Concept is not required to be fully functional and can contain only limited amount of artifacts but should a variety of processes, workflows, operations, adapters etc. allowing to sound out different eccentricities.
2. In all large scale migration projects, it is very important to recreate the environment early. Solution partitioning, project structure, build, server topology, packaging and deployment guidelines should not be left for the last – we cannot overemphasize this. A controlled environment where SeeBeyond and BizTalk application can be run in isolation on the control data set. This is critical as there’s no other means to validate migration quality in this approach except to compare output for the known input. It takes considerable amount of discipline to stick to this plan as features get discovered, complexity grows and the team is crushed under this load – but it is important that this stays in focus as an early deliverable, not late one.
3. Early and continuous integration of the processes along with testing mentioned in 3) above helps to discover problems early and take mitigation actions. This is a key assumption that bedevils many teams who assume so-called isolated systems will not impact each other when run together on the same application server. Make sure to get the latest version of the shared code to avoid unpleasant surprises.
4. Some BizTalk features must be evaluated carefully before applying in implementation. There might be critical limitations that would slow down or even halt migration. For example, the Oracle DB adapter for BizTalk cannot return results of the joins from multiple tables. So, if you require this kind of result set then it’s time to turn back to the more flexible ADO.NET and bid goodbye to a code-free approach. Native SQL schema can be used instead of adapter generated schemas to generalize data access out of orchestrations and make them cleaner and easier to understand and maintain. When creating data access intensive orchestrations it often makes sense to consider delegating Oracle adapter task to a .Net component to get rid of large amount of transient messages and reduce performance overhead.
5. Do a careful analysis of the current code base before committing any timelines or budget. Do not assume the solution you are inheriting is either perfect or well-architected. We have found messy implementations where the management is delegated to UNIX/WMI scripts instead of being controlled from the GUI/e-Ways/Collaboration rules, resulting in business logic being distributed over areas not conducive to maintenance or good practice. When these problems are compounded by lack of documentation, then you need to increase project times by a significant amount to accommodate the pain and increased efforts of first collating logic in the code base into a sensible picture before the analysis and design can even begin.
Posted in: Software Programming| Tags: Microsoft WCF Challenge B2B BizTalk BizTalk Server EAI flagship integration solutions Migrating Windows Communication FrameworkRoll Out Strategy in BizTalk Migration
Rollout in an EAI migration project is always tricky – the goal should always be to cause as little disruption and downtime as is necessary. In order to achieve this goal of minimal disruption, an incremental phase approach is recommended.
To further illustrate our phased deployment approach, consider the following scenario which is typical in most EAI implementations in large enterprises - The existing SeeBeyond implementation has a history of having been built incrementally; new programs or upgrades were placed in service individually. Now compound that complexity with migrating the actual EAI logic to a new tool (BizTalk). Deploying such a migrated solution which includes a large number of sub-systems in service all at the same time, even if they are well tested, is an incredibly risky step for any business. Do not be deceived if each subsystem individually has a low bug rate – a single problem can manifest itself in ways across other systems that may not have been tested or accounted for. What typically occurs is that the number of such problems quickly overwhelms and cross-pollutes other systems, crippling the ability of the organization to deal with them. In other words - High Conversion Risk.
Incremental migration strategies entail the decomposition of multi-system implementations in phases which can be independently and selectively addressed and replaced. This reduces the costs and risks of a migration program. The original SeeBeyond sub-systems are retained and used until the BizTalk functions achieve an acceptable level of reliability, then they are replaced. In the beginning the smaller interfaces are piloted as the learning curve is steeper and the number of unknowns greater, so we do not want to overload with a complex sub-system. As we gain greater expertise and vital insights, we attempt to bite of larger pieces and tackle them until we meet with complete success.
In an incremental rewrite methodology, the SeeBeyond system continues to run as the new system is built, tested, and deployed. You will want to try to build the new system in a way that seamlessly integrates with the old system. The side-by-side environment maintains redundancy as functionality moves from the old to the new system.
Posted in: C# and .NET| Tags: Microsoft WCF B2B BizTalk BizTalk Server EAI flagship integration solutions Migrating Windows Communication FrameworkDiscovery and Development Strategy of BizTalk Migration
With any migration approach, discovery of SeeBeyond code and artifacts is necessary due to the invaluable wealth of information there which has survived due its robustness or been refined to its current state. Not to use it would be an enormous waste. Although a huge amount of it is redundant in BizTalk due to the pre-built infrastructure (ports, adapters, configuration) you can mine a lot of it outright – some of it a simple cut and paste – such as database queries. Hence knowledge of SeeBeyond is inescapable.
To delve into SeeBeyond or to have a meaningful exchange with the SeeBeyond analyst, it would benefit the BizTalk professional greatly if could gain a first look inside SeeBeyond with the concepts and mental models he is already familiar with. Getting sufficient time from the harried, expensive and stretched thin SeeBeyond analyst is likely not going to be easy to come by, so the problem gets worse.
We have painstaking compiled a reference section to help with this comprehension – you will find this in the Appendix A of this paper.
Once the migration is complete, it is necessary to ensure that the new destination data is semantically equivalent to the original destination data (that has been processed with the SeeBeyond implementation). For this to be so, the results of all operations on the new Biztalk solutions must be identical to those on the old SeeBeyond solutions. There is no better way to gain such assurance than to exhaustively perform the same operations on both solutions and to compare the results. We recommend a Cross-Talk approach for verification testing.
In the Cross-Talk QA approach, the data source is flipped from SeeBeyond to BizTalk receivers and the results compared. When verified with a credible set of test queries, a high degree of assurance can be obtained from this approach that the new data store is semantically equivalent to the old one.
Posted in: Software| Tags: Microsoft WCF B2B BizTalk BizTalk Server EAI flagship integration solutions Migrating Windows Communication FrameworkReArchitecture in BizTalk Migration
This rearchitecture approach is ideal in circumstances where the original implementation is sub-optimal (has performance issues or has been overtaken by new developments) or was developed in multiple layers laid on top over time of one another without an inherent design.
Even systems with well-defined architectures are prone to structural erosion. The relentless onslaught of changing requirements that any successful system attracts can gradually undermine its structure. Systems that were once tidy become overgrown as piecemeal growth gradually allows elements of the system to sprawl in an uncontrolled feature-creep. If such sprawl continues unabated, the structure of the system can become so badly compromised that it must be abandoned. The only way to sustain this system is by patches, and over time layers develop, until no shred of any original architecture or pattern remains.
Such systems are usually very difficult to maintain, document and perform poorly, making them good candidates for complete re-writes. In this case rearchitecture in a new system (BizTalk) from the ground up makes most sense as looking under the hood will confuse and provide no net benefit. There will also be many subtle, and not so subtle, differences between the old and new platforms and these make manual translation very difficult. A developer can quickly get led astray by the many subtle differences and become unproductive due to the overhead of constantly switching back and forth between old and new tools and platforms.
The newly designed system will accommodate new features, use the latest standards, and follow best practices for component reuse. The lessons from the previous implementation and hindsight will always be available, making the second generation solution a robust and optimized implementation.
Here are some tips for a rearchitecture approach
· Make use of accepted standards like HL7, HIPAA, RosettaNet, cXML, GS1, etc where applicable
· Loose coupling enabling other applications to be plugged in later
· Common components should be separated out into frameworks for reusability
· Avoid using custom connectors as these are difficult to develop and maintain
· Insist on automation of testing process to check for regression
· Run a performance lab and overload tests to make sure the application scales
· Ask for load balancing and fail over strategies for deployment
Posted in: Software Software Programming| Tags: Microsoft WCF B2B BizTalk BizTalk Server EAI flagship integration solutions Migrating Windows Communication FrameworkWhy migrate from SeeBeyond to BizTalk?
Migrating from a large EAI platform to another is always a tough decision and cannot be made in vacuum. Whatever EAI platform an enterprise has in place forms the backbone of their enterprise when it comes to their data communication needs. The EAI platform is what one would analogously consider the “plumbing” of a house. If the plumbing fails, the house is unlivable and alternate arrangements have to be made.
Given the business criticality of an EAI solution, it is essential for an enterprise to choose a tool that is scalable, robust, extensible and most importantly maintainable. Typically these are qualities one would look in any software tool, but nowhere are they more important then in an EAI tool.
Historically, the problem with EAI tools in the enterprise space has been their lack of ability to continue to evolve to the new standards and advancements in technology and also the ability to scale with the growing demands of business. The tools that did manage to provide patches to the tools for such advances were usually “kludgey” solutions that were not truly integrated with the tool. Given these challenges that are bound to occur with any EAI tool now or in the future, it is prudent to move with a tool that is supported by a financially stable company which will be there for the long haul and has the deep pockets to support the advancements in technology. The reasons discussed about inability to evolve and scale has plagued the EAI industry until now because typically all EAI vendors have been small companies with their own agendas dictated more by wall street than the needs of their customers and as a result virtually all the key players have either almost vanished (Vitria) or have been acquired by larger corporations to fill a niche in their overall solution portfolio (SeeBeyond being purchased by Sun is a prime example) when the focus of the company may not be EAI.
Biztalk is one of the few tools that has grown internally within Microsoft and has been nurtured over the past few years for it to mature and achieve enterprise status. It is supported by the most financially stable software corporation on the planet and has the deep pockets and resources to keep evolving BizTalk to the ever changing needs.
Regardless of what tool an enterprise may be utilizing currently, chances are the enterprise will hit a point where the tool is unable to evolve or scale as needed and the sooner an enterprise makes a decision to move with an industry leader, the safer its investment in time, resources and money will be.
Having discussed some benefits of moving to BizTalk as a tool for choice, here are some points to consider why it is beneficial to move away from other EAI tools such as SeeBeyond.
· Expensive to maintain and increases TCO (Total Cost of Ownership)
· Is no longer a market leader in EAI tools and will eventually phase out and disappear where components are cannibalized and utilized in the J2EE frameworks from Sun
· Product is not backwards compatible with older versions
· Lack of backward and forwards compatibility requires constant re-tooling of team
· Upgrade in SeeBeyond is resource intensive and time consuming
· Takes longer to make changes to functionality
Posted in: Software Software Programming| Tags: Microsoft WCF B2B BizTalk BizTalk Server EAI flagship integration solutions Migrating Windows Communication FrameworkCons of SeeBeyond to BizTalk Migration
· Cannot be decoupled from the proprietary data storage layer – Microsoft SQL Server. This means a deployment must also acquire SQL Server licenses even though it may be using a different DB solution.
· Debugging the main business process logic - orchestrations – is non-standard, circuitous, and discouragingly complex (stepping through code debugging involves understanding the XLang library) – although nearly every error message has profuse help available from the online community.
· BizTalk development involves little coding but a lot of configuration. Configuration errors are cryptic, difficult to trap, trace or debug – resulting in once again defaulting to help from the community rather than the software itself
· UML compliance for modeling the business processes (Orchestrations) is missing
· Recovery of failed messages is manual and not automated
· Monitoring is dependent on additional software like Microsoft Operations Manager and is not available out of the box
· Runs only on Windows and not on other server operating systems like UNIX flavors. Although the Microsoft server platform is the fastest growing server system, this is still limiting to its adoption base for non-Windows deployments. The product is also reliant on core OS services like security, scale out and authentication.
Posted in: Software Software Programming| Tags: Microsoft WCF B2B BizTalk BizTalk Server BizTalk Server 2006 EAI flagship integration solutions Windows Communication FrameworkPros of SeeBeyond to BizTalk Migration
Business Value
· Fastest growing integration software, with 5500+ customers in the past 4 years
· Supported by the largest and most stable software company on the planet –course change which is common among smaller players is unlikely
· Lower TCO – on average about half but sometimes as much as one-sixth. This is in part due to the large user community of developers and support communities (average SeeBeyond consultant (independent) is $150 / hour whereas the average BizTalk consultant costs $65-70 / hour)
· Powerful, customizable end to end process management, application integration and trading partner management
· A broad array of technology and application adapters are freely available – connecting everything from SAP, Oracle and Siebel to BizTalk
· Business vertical accelerators are available for faster implementation – HL7, HIPAA, RosettaNet, SWIFT, GDS, etc
· Highly SOA capable – native support for web services in web ports. Now its tightly integrated with the Windows Communication framework supporting all WS* standards
· Hybrid of Hub and Spoke and Message Bus architecture – the “Hub Bus” architecture avoiding the weaknesses of both
· 64-bit support
· Built on the Windows platform – the largest and most broad-based platform integrating a host of other widespread applications - from office to collaboration portals
Development value
· Tightly integrated with Visual Studio – the most widely used and well-known integrated development environment in the World
· Bigger Adapter Community – help and a breadth of knowledge is easily available
· Reduced Custom Coding
· Common Type System Languages like C# (no MONK)
· All internal processing and communication based on open XML standards
· Simplified setup and deployment
Posted in: Software Software Programming| Tags: Microsoft WCF B2B BizTalk BizTalk Server BizTalk Server 2006 EAI flagship integration solutions Windows Communication FrameworkA Perspective on SeeBeyond to BizTalk Migration
BitTalk is the flagship product of Microsoft for EAI and B2B integration solutions. BizTalk Server allows diverse software islands to exchange information. It comprises of a messaging component that provides the ability to communicate over a range of transport and application protocols and data formats by using pluggable adapters. BizTalk supports graphically modelling the business process logic to schedule that information exchange. The product also lets information workers monitor running processes, interact with trading partners, and perform other business-oriented tasks.
Several other technologies can also be used in concert with the engine, including:
· A Business Rules Engine that allows evaluating sets of rules and policies.
· A Health and Activity Tracking tool that lets developers and administrators monitor and manage the engine and the orchestrations it runs.
· An Enterprise Single Sign-on facility, providing the ability to map authentication information between Windows and non-Windows systems.
On top of this foundation, BizTalk Server 2006 provides a group of technologies that address the more business-oriented needs of information workers. Those technologies are:
· Business Activity Monitoring, allowing information workers to monitor a running business process. The information is displayed in business rather than technical terms, and what gets displayed can be controlled directly by business people.
· Business Activity Services, allowing information workers to set up and manage interactions with trading partners.
From its initial roots, BizTalk Server has grown into the foundation for supporting a range of business processes since 2004 when it was the first Microsoft server product to be re-developed in .NET. As the change to a service-oriented world rolls on, BizTalk Server 2006 will continue to play an important part in the ongoing SOA wave through an array of WS* compliant technologies like Windows Communication Framework.
Posted in: Software Software Programming| Tags: Microsoft WCF B2B BizTalk BizTalk Server BizTalk Server 2006 EAI flagship integration solutions Windows Communication Framework