Tallan's Technology Blog

Tallan's Top Technologists Share Their Thoughts on Today's Technology Challenges

BizTalk Mapper file format (.btm) documented

Dan Field

The BizTalk mapper is a phenomenal tool and part of what sets BizTalk apart as an enterprise middleware platform.  Developers with little knowledge of XSLT can rapidly and accurately develop complicated transformations, and like all other aspects of the BizTalk engine it can be extended and heavily customized when necessary.  This more than makes up for any limitations in the mapping engine.  At the same time, sometimes the designer interface goes wrong and it’s difficult to figure out why: a functoid that has placeholders on a multipage map, or the desire to delete all functoids of a specific type from a specific page.  On smaller maps, this usually isn’t a problem, but when dealing with larger documents (like HIPAA 837 claim or 834 enrollment files), understanding the file structure of .BTMs can be a huge time saver.  It’s also possible to manually correct changes in your schema that would otherwise break the map (and perhaps cause a significant loss of development time and effort).  For demo purposes, I’ll use a fairly simple map with just a few links, using the 837I schema.overview

This map has two pages, makes use of a couple functoids, and only has a few links.  Ordinarily it wouldn’t be very difficult to refactor if needed, but in real life cases maps for this file type can have dozens of pages and thousands of links.  If we right click the map in the solution explorer, we get the option to open it with the XML editor:

open with

With the exception of scripting functoid values, the map is saved as one long line of text.

before formatting

You can easily fix that by clicking “Format document” or Ctrl+K, Ctrl+D.  (If you wish to edit the .btm file in another editor such as Notepad++, you’ll have to change the encoding attribute to “utf-8″ before XML Tools will pretty print it.)

After formatting

Once formatted properly, the document is actually fairly easy to work with.

The mapsource root node has a few attributes that can be customized and are already well documented here: https://msdn.microsoft.com/en-us/library/aa561485.aspx.  Some of these attributes can only be changed by directly editing the map file.

The SrcTree and TrgTree nodes specify the source and target schemas.  For this demo, my map and schema are in the same project – but ordinarily they’d be compiled into separate projects/assemblies.  If a fully qualified name or file name of a map changes, it can be easily fixed by editing this node.  Note that the mapper will allow you to replace a schema (and will prompt you to do so if a FQN or file name changes), but it sometimes has mixed results on preserving links and nodes.

ScriptTypePrecedence allows you to enable or disable Scripting functoid options.  These parameters can be controlled in the properties window in VisualStudio as well, and don’t usually have to be changed.

TreeValues allows you to see/edit default values for testing (set on the source schema) or constants (set on the destination schema).  This is immensely useful, as there is no obvious visual key about default values set in the mapper.  It’s easy to forget where you set a default value if you use this feature, but using this method can quickly identify, clear, or change those values.  The format is:

<Value value="123" Query="/*[local-name()='&lt;Schema&gt;']/*[local-name()='X12_00501_837_I']/*[local-name()='ST']/*[local-name()='ST01_TransactionSetIdentifierCode']" />

The value attribute sets the value, and the Query attribute specifies an XPath to the node to set. Note, however, that the XPath has an additional pseudo-element the mapper uses: <Schema> (encoded as &lt;Schema%gt;). This must be present in any XPaths used in the map.

Finally, we come to the Pages node.  Under the Pages node, there are Page nodes, which have name attributes.  These correspond to the visible pages in BizTalk.  Each Page Element has two children: Links and Functoids.

Links contain Link elements.  Each Link element has three attributes: LinkID, LinkFrom, and LinkTo.

  • The LinkID is unique among other LinkIDs throughout the entire file, so if you manually add or change any you must update this LinkID.  The LinkID is also used by functoid elements to reference their inputs and outputs.
  • The LinkFrom is either a number (in which case it corresponds to a FunctoidID) or an XPath (with the <Schema> prefix).  This is the source of the link.
  • The LinkTo is either a number (in which case it corresponds to a FunctoidID) or an XPath.

Functoids contain Functoid Elements, which have four attributes: FunctoidID, Functoid-FID, X-Cell, and Y-Cell:

  • FunctoidID is similar to LinkID, but for functoids.  They are kept unique among other FunctoidIDs throughout the file.  This is what a LinkFrom or LinkTo attribute refers to when it’s numeric in a Link element.
  • Functoid-FID is an integer that specifies what kind of functoid it is.  In the screen shot, 424 is looping.  I have not fully documented what each one does, but it’s possible to figure out what functoid you’re looking at from the Link values coming into and out of it and their XPaths.  Fromt his you can derive the meaning of the Functoid-FID value
  • X-Cell: the X position of the functoid (horizontal) on the grid
  • Y-Cell: The Y Position; these values should not be altered in a way that would cause two functoids to occupy the same cell for obvious reasons.

Functoid elements have up to two children: Input-Parameters and ScripterCode (for Scripting Functoids)

Input-Parameters has Parameter elements one or more children.  Parameter nodes have four attributes: Type, value, linkIndex, Guid.

  • Type: either “link” or “constant”.
  • Value: if Type is “link”, then the value refers to a LinkID in the map; if the Type is “constant”, then Value is a constant value parameter for the functoid.
  • linkIndex: numerical index for the parameter/value.  0 based.
  • Guid: a GUID used by the mapper.  These can be changed without breaking the map, but they must be valid GUIDs.  At the end of this post, I’ll include a PowerShell script that can regenerate GUIDs in a file, which is handy if you copy and paste functoids.

ScripterCode nodes contain a child called Script with a Language attribute, which specifies which language is used for this scripting functoid:

scripter

In the CDATA node, there is the code for my scripting functoid.  Again, this is immensely useful when using inline Script functoids: by directly editing the file, you could search for code in these functoids and even edit that code without having to find the exact scripting functoid.  You can also manually edit external assembly script functoids.    For external assemblies, the language is “ExternalAssembly”  and three additional attributes are used:

  • Assembly: the FQN of the Assembly
  • Class: The namespace qualified name of the class in the assembly
  • Function: the name of the function

external assm

So when else is this useful?

  • Lets say you (or another developer) refactors the schema for a map you’ve been working on.  While this shouldn’t happen, it certainly does (sometimes as a direct result of finding out how challenging it is to map that particular structure!).  While this could potentially break your entire map, it doesn’t have to this way.  You can manually add the part of the XPath that would be missing to relevant LinkTo or LinkFrom nodes.
  • You get a horribly unhelpful error message that Functoid X has placeholders and that functoid occurs on many pages in your map.  Look for functoids with that FID that don’t have the right number of Parameter nodes (too many or too few), or that have a GUID of 00000000-0000-0000-0000-000000000000 (sometimes this happens when the map gets corrupted).
  • You need to quickly find or change a constant value, or a value in a scripting functoid
  • You need to map a nearly identical structure in another part of the schema (such as the Loop1 loops in the 837 schemas): I was able to save a lot of time on this by copying and pasting entire pages in the text and using search/replace (sometimes with regexes) to update the XPaths on the new pages.

Finally, here’s a PowerShell script to regenerate GUIDs; I wish I could take credit for this, but I only slightly modified a version that was found on StackOverflow (http://stackoverflow.com/questions/2201740/replacing-all-guids-in-a-file-with-new-guids-from-the-command-line):


## GuidSwap.ps1
##
## Reads a file, finds any GUIDs in the file, and swaps them for a NewGUID
##
param (
[string]$inputFilename = $( Read-Host "Enter input filename" ),
[string]$outputFilename = $( Read-Host "Enter output filename")
)

$text = [string]::join([environment]::newline, (get-content -path $inputFilename))

$sbNew = new-object system.text.stringBuilder

$pattern = "[a-fA-F0-9]{8}-([a-fA-F0-9]{4}-){3}[a-fA-F0-9]{12}"

$lastStart = 0
$null = ([regex]::matches($text, $pattern) | %{
$sbNew.Append($text.Substring($lastStart, $_.Index - $lastStart))
$guid = [system.guid]::newguid()
$sbNew.Append($guid)
$lastStart = $_.Index + $_.Length
})
$null = $sbNew.Append($text.Substring($lastStart))

$sbNew.ToString() | out-file -encoding utf8 $outputFilename

Write-Output "Done"

Note that it uses utf8 encoding even though the file says it’s utf-16 – because the file actually has a BOM indicating it’s utf8.

1 Comment. Leave new

Satya B. Kolachana
May 22, 2015 1:18 pm

This and another feature for .btm files is a nice “Validate map” in the solution explorer, when u right click the file. It generates the actual xslt that biztalk engine is excuting using XslCompileTransform in BizTalk 2015. This has on many occassions saved my life when trying to figure out namespace resolutions.

ciao

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

\\\