The next step was to start work on automating the build and deployment process. A perfect opportunity to leverage the TFS and MSBuild support introduced in BizTalk 2009.

In order to build a BizTalk 2010 project, I need to install the BizTalk Project Build Components on the TFS build agent. Unfortunately, the client was using Windows Server 2003 R2 for the existing TFS 2008 Build Agent (BizTalk 2010 requires Windows Server 2008 and above.)

To work around this, we installed TFS 2008 Build on the BizTalk 2010 Development machine and added it as an Agent to the TFS server. (This is a short-term work-around until the client creates a dedicated VM for this purpose.)

I set-up my build definition and got the build working. Hooray!

I then used BTSTask to write a simple batch file to Create the Application, Add the Resource and Import Bindings.

The next time I ran the same build, it failed with the following error: “SGEN : error : Cannot generate serialization assembly because it already exists. Use /force to force an overwrite of the existing assembly.”.

I was somewhat puzzled why the build failed when I haven’t made any changes to the source code.

The MSDN docs for SGEN provides the following info…

The XML Serializer Generator creates an XML serialization assembly for types in a specified assembly in order to improve the startup performance of a XmlSerializer when it serializes or deserializes objects of the specified types.

Since this appears to be a performance optimization step, my initial approach was to disable Generate serialization assembly by changing it from Auto to Off in the BizTalk project properties (build tab.)

This setting had no affect on the build and did not resolve the build error. I took a break from resolving the build error and worked on automating the deployment and un-deployment of the Biztalk application.

Around this time, I noticed that the build succeeded only when the BizTalk application was not deployed!

Looking into the issue more, I noticed that the error was due to my BizTalk project assembly being in the GAC. This was mentioned as an issue in the following MSDN forum post and I verified that the build failed when the assembly was in C:\Windows\assembly and succeeded when it was removed.

So if SGEN does not like the assembly being in the GAC, I will just modify the build script to  remove it from the GAC before compilation and add it back in afterwards. Messy, but a necessary step due to the TFS Build Agent residing on the DEV BizTalk Server.

To implement this, I started by adding the Un-GAC step to the build script…

<Target Name="BeforeCompile" Condition="‘$(IsDesktopBuild)’!=’true’">
    <Exec Command="gacutil /u MyBizTalkProject" WorkingDirectory="C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin"/>
</Target>

Around the same time, I resumed working on the deployment process – now using the BizTalk Deployment Framework (BTDF). At this point, I deployed a new version of the BizTalk application using the server MSI provided by BTDF.

The next build succeeded and I wanted to verify that my build customization was working properly. When I examined the build results, I noticed that it was not finding the assembly in the GAC:

Target "BeforeCompile" in file "…\BuildType\TFSBuild.proj":
Using "Exec" task from assembly "Microsoft.Build.Tasks.v3.5, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a".
Task "Exec"
  Command:
  gacutil /u MyBizTalkProject
  Microsoft (R) .NET Global Assembly Cache Utility.  Version 3.5.30729.1
  Copyright (c) Microsoft Corporation.  All rights reserved.
 
  No assemblies found matching: MyBizTalkProject
  Number of assemblies uninstalled = 0
  Number of failures = 0
Done executing task "Exec".
Done building target "BeforeCompile" in project "TFSBuild.proj".

I was a little confused – I was sure that assembly was deployed to the GAC as the application was up and running in BizTalk. Looking into this a bit more, I realized what was going on…

In .NET, assemblies compiled targeting the 4.0 CLR are now deployed to

C:\Windows\Microsoft.NET\assembly

whereas assemblies targeting the 2.0 CLR (which includes .NET 2.0, 3.0 and 3.5) are kept in

C:\Windows\assembly

This was a BizTalk 2010 project and I verified that .NET 4 is the target framework – so it should be using the  .NET 4.0 GAC. Why were my assemblies being deployed to the .NET 2.0 GAC earlier (C:\Windows\assembly)?

It was at this point that I realized that I forgot to configure the TFS Build Agent to build using the .NET 4 compiler. This was confirmed by checking the build log…

Project file contains ToolsVersion="4.0", which is not supported by this version of MSBuild. Treating the project as if it had ToolsVersion="3.5".
Building with tools version "3.5".

TFS 2008 Build Agent was compiling it using the .NET 3.5 compiler. And when I deployed using BTSTask, I was deploying a .NET 3.5 assembly to the GAC and the 3.5 version of SGEN was generating an error during the next build.

When I switched over to using BTDF, it was compiled on my desktop as a .NET 4.0 MSI and was deployed the assembly on the DEV server to the 4.0 GAC. Since the .NET 3.5 tools and utilities cannot see the 4.0 GAC, SGEN does not generate an error during the build and  gacutil is not able to find the assembly.

Side Note: the .NET 4 gacutil searches both the v4 and v2 GACs

I incorrectly assumed that it was compiled as .NET 4 since the TFS builds were succeeding and the build server already had the v4 framework and SDK installed.

Old BizTalk habits die hard and after working with BizTalk 2004-2009 for so long, I didn’t notice anything wrong when the assemblies were in the old GAC (C:\Windows\assembly).

The following table provides an overview of when my builds were failing.

Resolution

Modify the TFS Build Agent to default to the .NET 4 MSBuild. This is easily done by…

1. Stopping the Visual Studio Team Foundation Build service

net stop VSTFBUILD

2. Edit the following file:

C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\tfsbuildservice.exe.config

and modify the following setting…

<!– MSBuildPath
         Set this value to the full path to the directory of MSBuild.exe to use
         a location other than the default. This should only be needed if a new
         version of the .NET Framework is installed. –>
<add key="MSBuildPath" value="C:\Windows\Microsoft.NET\Framework\v4.0.30319" />

Note: the build will fail with the following error if you try to point to the X64 version of MSBuild:

C:\Windows\Microsoft.NET\Framework64\v4.0.30319

MSBUILD : error MSB1021: Cannot create an instance of the logger. Could not load file or assembly ‘file:///C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\Microsoft.TeamFoundation.Build.Server.Logger.dll’ or one of its dependencies. An attempt was made to load a program with an incorrect format.

3. Start the build service

net start VSTFBUILD

Now that TFS Build is using the proper MSBuild version, the BizTalk assembly is being compiled as a v4 CLR image.

The last step is to modify the build script to reference the correct version of gacutil in order to GAC and un-GAC the assemblies. If you aren’t paying careful attention, you may not realize which .NET version tools you are using. Compounding this problem is the subtle difference in the file paths. For example, these are the directories for the .NET 3.5 tools (depending on whether you want to run the 32-bit or 64-bit versions):

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\x64

These are the directories for the .NET 4.0 tools:

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools

C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools\x64

Going back to my build script, changing the Working Directory of the Exec task completes the last bit of changes.

<Target Name="BeforeCompile" Condition="‘$(IsDesktopBuild)’!=’true’">
   <Exec Command="gacutil /u MyBizTalkProject" WorkingDirectory="C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\NETFX 4.0 Tools"/>
</Target>

Lessons Learned

1. Just because a build succeeded on the first try, doesn’t mean you shouldn’t examine the text log. If I did that early on, I would have seen countless references to .NET 3.5. It is very easy to see the green success icon in the Build Results page and quickly move to the next task.

2. Don’t use TFS Build on the same server as BizTalk. Having to UnGAC an assembly during a build is a pain and it disrupts BizTalk processing.

3. Try to unlearn old habits – such as jumping to C:\Windows\assembly to look for an assembly in the GAC.

• WIF runtime
• WIF SDK
• Identity Developer Training Kit

Note that there is a version of the SDK and Training Kit for Visual Studio 2008 SP1 (.NET 3.5) and one for Visual Studio 2010 (.NET 4.0.)

Note: Side by side installation of the WIF 3.5 and 4.0 SDKs is not recommended. Visual Studio templates will be properly installed for side by side installations of Visual Studio 2008 and 2010, the included samples will not work in such a configuration without modification.

This post is about a problem I found with the 2008/3.5 version of the Identity Training Kit and how you can fix an error in the lab set-up scripts.

Each training lab has a SetupLab.cmd file that needs to be run in order to install and configure the code snippets and certificates for that lab. For the first hands-on lab, Web Sites and Identity, this can be found at \IdentityTrainingKit\Labs\WebSitesAndIdentity\Source\Setup

When you run this batch script (with elevated administrator permissions), it runs a configuration wizard that checks for all prerequisites: PowerShell, Visual Studio, SQL, IIS7, WCF HTTP Activation, WIF.

Unfortunately, it complains about the WIF SDK not being installed even though it is.

I verified that the SDK was properly installed in Control Panel > Programs and Features. I tried reinstalling the SDK but that didn’t fix the error.

I did some investigating in the content of the TrainingKit and followed a trail of folders (IdentityTrainingKit\Assets\DependencyChecker\scripts\dependencies\check) to this set of PowerShell scripts…

Open up CheckWifSdk.ps1 and you will find the culprit

function SearchUninstall($SearchFor, $SearchVersion, $UninstallKey)
{
$uninstallObjects = ls -path $UninstallKey;
$found = $FALSE;
[reflection.assembly]::LoadWithPartialName("System.Version");
foreach($uninstallEntry in  $uninstallObjects)
{
   $entryProperty = Get-ItemProperty -path registry::$uninstallEntry
   if($entryProperty.DisplayName -like $searchFor)
   {
       $vc1 = new-Object System.Version($entryProperty.DisplayVersion);
       $vc2 = new-Object System.Version($searchVersion);
       if($vc1.CompareTo($vc2) -ne -1)
       {
         $found = $TRUE;
         break;
       }
   }
}
$found;
}
$res1 = SearchUninstall -SearchFor 'Windows Identity Foundation SDK' -SearchVersion '6.1.7600.113' -UninstallKey 'HKLM:SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\';
$res2 = SearchUninstall -SearchFor 'Windows Identity Foundation SDK' -SearchVersion '6.1.7600.113' -UninstallKey 'HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\';
(($res1 -eq $TRUE) -or ($res2 -eq $TRUE))

The culprit is in the last section. Change 6.1.7600.113 to 6.1.7600.16436 AND Windows Identity Foundation SDK to Windows Identity Foundation SDK 3.5 in both SearchUninstall statements (one checks for the x86 version while the other looks for the x64 version of the SDK)

You can verify the version number and display name by checking the detail info in Control Panel > Programs and Features (see screenshot above) or by checking the registry paths the PowerShell script uses.

HKLM:SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\
or
HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\

Re-run setupLab.cmd and you should be able to proceed to the remainder of the set-up. Follow the recommended steps described in the Overview of each Lab.

Tallan is providing a complimentary executive lunch around the topic of Microsoft Integration. It includes a talk by Tom Stickle, Program Manager at Microsoft for the Connected Systems Division, about the new integration features in BizTalk. In addition, there will be a real-life integration case study from a CT-based company.

The event will be held from 11:00 AM until 1:00 PM on June 10th at Lake of Isles in North Stonington, CT (part of the Foxwoods Resort), where an executive lunch will be served.

Here is the link to more information about this event and how to register.

Here is a link if you are interested in downloading the presentation (powerpoint 97-2003 format).

BizTalk Server Database Optimization White Paper – This is an excellent MSDN write-up on how to optimize your BizTalk systems. The title is somewhat incorrect as it describes how to optimize the hardware (which RAID level to use, turn off hyperthreading), the Network (usage of subnets), the OS (Registry and other settings) , IIS and SQL Server tuning. This is an extensive and through document that includes lab performance tests and results, recommended configurations and links to helpful tools and utilities.

Some of the suggestions can be helpful even if you aren’t using BizTalk.

Enterprise Service Bus (ESB) Guidance Documentation and Source Code – This is the CodePlex community site for the latest release (CTP3) of the ESB Guidance package for BizTalk. Current plan is to release the final version in October.

Add a new C# class library project to your BizTalk solution and call it MyCompany.BizTalk.Functoids. Create a C# class in the project and use the same project name for the namespace.

 Example:

Add the method to perform the conversion:

Add the close braces for the class and namespace, save the file and compile the project.

In order to use the scripting functoid in your map, you will need to first add the assembly to the GAC:

gacutil /i MyCompany.BizTalk.Functoids.dll

In order to use it on a map, you will need to drop the Scripting Functoid onto the map. It looks like a orange box with a S label. Select the scripting functoid and in the properties, click on the Configure Functoid Script option. A Configure Functoid Script dialog box will appear and the following entries should be filled in as follows:

Script Type: External Assembly

Script Assembly:  MyCompany.BizTalk.Functoids, version=…

Script Class: MyCompany.BizTalk.Functoids.Utils

Script Method: ConvertFromEBCDIC

After this, all you would need to do is connect the input parameter and the one output parameter.

In order to complete the exercise of creating the archive component, we need to add the code to actually perform the archival of the message. For this example, the requirements of the archive component is to persist an exact copy of the message coming into BizTalk (whether flat file, xml, binary or even an encrypted) in it’s original form to the local filesystem.

To start off, we need to create a method to write an inputstream to the filesystem outputstream. The following CopyStream method does this…

 The next method needed is one that takes a BizTalk IBaseMessage and uses the CopyStream method from above to write the message to the filesystem.

The final step is to  implement the Execute method that utilizes the WriteToFile method we just created. We will use the InterchangeID as the filename, and since it is basically a GUID it guarantees the filename will be unique. This is a string automatically set by the Messaging Engine for each message that arrives on the server and it defines the unique ID that is used to group the documents that resulted from the same interchange message.

We also want to the archive file to include the original filename, otherwise it might be difficult for a user to determine which message came from which file when a directory is filled with just GUIDs. So what we will do is look at the original filename property and prepend it to the InterchangeID with a underscore between them. You will notice that depending on the adapter (FILE, FTP) the file comes in on, we use a different schema. In addition, if the file comes in from any other adapter, the filename property won’t be available, so we skip the prepend step. 

When used, the archive component will create files with filenames similar to the following examples:

If Transport Type = FTP  or FILE:

MyTestFile.txt.pgp_{2E50A0DD-395B-4629-A345-7CB0FDDE4727}.out

Any other Transport Type:

{B9D33010-BE49-490B-ABFA-E3B63CAFCDAE}.out

 The next article in this series goes over the steps to using the custom Archive pipeline component in your BizTalk application.

The next interface we will implement is the IPersistPropertyBag interface. Before we do that, we need to add two generic utility methods to read to & write from property bags. A property bag is an object which can persistently save its properties. MSDN provides more info on the Microsoft.BizTalk.Adapter.Framework.PropertyBag class.

Now we can implement the IPersistPropertyBag interface which only requires the following four methods:
• GetClassID – Gets the class ID of the object to persist.
• InitNew – Initializes a new class.
• Load – Loads data from the persisted object into the new object.
• Save – Saves the new object. Notice that the Save and Load methods persist the custom properties (path, IsActive) we created in the first post.

Note: the guid in this example should be replaced by your custom derived value (see the first post of this series)

 The last part of the interface we need to implement is the Microsoft.BizTalk.Component.Interop.IComponent interface. We just need to define one method – Execute - which is called by the pipeline to process the input message and get the resulting message. The method accepts the current pipeline context and the message to process, and it returns the modified (or not) message.

For now, we will not add the actual “archive” code – that comes in the next post. We will just return the same message passed in. The reader should be able to compile this class and generate a DLL.

A custom pipeline component is just a plain .NET class that implements several BizTalk interfaces. I recommend creating a seperate c# project to contain just your custom pipeline classes.

In this post, I will go through creating a sample pipeline component that archives the incoming message to the file system at the Decode stage

To get started, create a standard C# Class Library project.

Add references to the following:
• Microsoft.BizTalk.Messaging (C:\Program Files\Microsoft BizTalk Server 2006\Microsoft.Biztalk.Messaging.dll)
• Microsoft.BizTalk.Pipeline (C:\Program Files\Microsoft BizTalk Server 2006\Microsoft.BizTalk.Pipeline.dll)
• System
• System.Data
• System.Drawing
• System.XML

 Add your namespace and the following namespace directives…

Add the class name and the following attributes. Note that you should replace the guid value with a value generated for your class

Note that the class implements the following interfaces:
• Microsoft.BizTalk.Component.Interop.IComponent - All pipeline components except assemblers and disassemblers implement this interface to get messages from the BizTalk Server engine for processing and to pass processed messages back to the engine.
  NOTE: this needs to be a fully qualified reference, otherwise it will cause a compiler error: ‘IComponent’ is an ambiguous reference between ‘System.ComponentModel.IComponent’ and ‘Microsoft.BizTalk.Component.Interop.IComponent’
• IBaseComponent – All pipeline components need to implement this interface to provide basic information about the component.
• IPersistPropertyBag –  Pipeline components need to implement this interface to receive its configuration information.
• IComponentUI – Pipeline components must implement this interface to be used within the Pipeline Designer environment.

The archive component has two properties: an archive folder location (set as a string) and a boolean to indicate whether the archive component is turned on. 

 The following properties are required to implement the IBaseComponent interface. Basically, the name, version and description of the component returned as string.

Another alternative to hard-coding the strings is to use a resource file (.resx) by adding a new class member…

And replacing return “string” with

Now, we will implement the ComponentUI interface. This consists of an icon property and a method called Validate. Validate is called by the compiler to verify all of the configuration properties are set correctly.  For now, to indicate successful property validation, the method should return an empty enumerator.

This ends the first part of the Custom Pipeline Components posts. This is continued in part two.

For security purposes, BizTalk never shows the existing values of any passwords – whether those passwords are entered during configuration, setting up SQL or FTP adapters, etc. Instead all the user will see is *************  (the # of asterisks does not correspond to the length of the password).

So if you export bindings to “save a copy” of the current configuration, don’t forget to track the passwords somewhere. One approach is to edit the exported files and fill in the empty password fields, but this should only be done if you can ensure the xml binding files will be tightly secured.

Here is an example of the password output for a SQL Server database connection. Just replace the stars with your real password…

(BTW, this post has another purpose – to test the blog post capabilities of Live Writer, including uploading images)