Friday, October 10, 2014

Capturing BizTalk Perfmon Values with PowerShell - Part 1

A couple of posts ago I commented on the recently released BizTalk Health Monitor snap-in that comes with BizTalk 2013.  Remember, this can be installed on just about any version of BizTalk.  It's nice to see some common BizTalk tools being integrated into the BizTalk Administration Console.

One of the things leadership tends to ask I.T. is what kind of value is BizTalk bringing to the company.  In my case, of course, the business leadership is asking: how many, how often, when, etc.

In my case, I've been tasked to do a few things:  determine what kind of load we have on the BizTalk servers.  A couple of reasons why:
  1. Throughput/load.  Management wants to understand "do we have a 'right-sized' environment?"  Keep in mind that Microsoft overhauled their licensing model not too long ago.  Instead of license per CPU, you now are being charged by the core.  This may cause a huge jump in licensing costs.  If BizTalk is constantly under 10% utilization, it may make sense to downsize the environment (by core, at least).  Or, in a worse-case scenario, show how much is really going through BizTalk to justify scaling up your BizTalk environment further.
  2. Understand which processes, applications, ports, etc. within BizTalk are being highly taxed or under-taxed.  In our case, we 'lease' our BizTalk environment to different entities within our organization.  We need to capture data to make sure we are properly charging our internal customer/entity base.
One of the things that the BizTalk Health Monitor has exposed for me are the NUMEROUS Instances that are available.

Also, within each Instance is the actual Counter you may want to monitor.


So knowing and understanding what counters exist is great, but viewing the BTHM Performance counters is a bit inefficient and really only gives you a general snapshot of about (default) 100 seconds .  Why not capture some performance counters and store it somewhere, either as a file or directly into a database?
Side Note: Light reading for those who are interested in documentation on some of these counters.
I recently came across the very excellent PowerShell extension SQLPSX on CodePlex.  All I can say is that the contributors here have done many people a huge favor with these modules.  In this particular case, Laerte Junior really deserves kudos.  I came across his Gathering Perfmon Data with PowerShell post, which led me to the CodePlex site.  Most of this entry follows Laerte's blog to the 'T'.  Credit also goes to Allen White, as one of his posts kicked this whole thing off.  I try to give credit where credit is due.  :)
Note: You will likely need to force PowerShell version 2 for the extensions to work.
Moving on.  You can either install SQLPSX or just copy the psm1 module to your environment.  For me, I just copied the module over and used the Import-Module command.

I wanted a quick list of ALL BizTalk counters and save them to a file.  Easy to do:

Opening the text file shows a rather comprehensive list.

Taking that one step further, I can now create an XML configuration file by using the command

And the results:

Warning: Before one gets too crazy here, make sure you either pair down the configuration file or add some extra filters to the PowerShell command when you generate a config file.  Please be selective with what you really want to monitor.

With that out of the way, we now can go to the next step (and the next blog) - gathering BizTalk performance data.

Monday, September 8, 2014

WCF-SQL Transport Component is not registered correctly

You may have come across the message "WCF-SQL Transport Component is not registered correctly" when setting up a receive location or send port in BizTalk.  Some blogs suggest re-installing the BizTalk Adapater Pack.

However, in the scenario that the company I work for, this isn't the case.

The architecture of the environment was designed to be a multi-server environment with 6 BizTalk server front-ends.  Three of the servers were built to be used as Orchestration servers, and the other three were built as Receive and Send servers.

At a high-level, this sounds acceptable (pending you are configuring these at the host level - more on that later).  

However, the 6 servers aren't built alike.  The adapter packs (among other things as well) were only installed on the Receive/Send servers.  One main reason this was done were that some third-party adapters had a per-server licensing fee (i.e. special ftp adapter), and the company wanted to keep costs down while still being compliant.  ("We aren't going to use them on the Orchestration servers, so why install them on ALL servers?").  

Unfortunately, the policy above bleeded over to other adapters as well, regardless if a licensing issue was at hand or not.  In this case, the WCF Adapter Pack was not installed across all 6 servers.  

When you open up the BizTalk Administration Console (regardless from which server you remoted to), you do see the WCF adapters which were installed.

Back to the issue at hand.  If you happen to create a send or receive port onto a server that does NOT have the adapters installed on, you will get the error.  You may think that the BT Admin Console is a database-driven configuration, and you would be correct with that assumption.  However, if you click on the OK button, BizTalk tries to verify, at a minimum, the configuration settings with the locally-installed artifacts that are needed for that type of transport.

Resolving the issue was to log onto the Receieve/Send Port servers and create the ports from those servers, rather then the Orchestration servers.  No need to install/uninstall anything.

Two takeaways from this:  
  1. It may be in your best interest to keep all servers as similar as possible and let the host instances dictate which servers run what BizTalk hosts.
  2. Even though the settings within the BizTalk Administration Console are stored in the BizTalk databases, it does indeed verify a few things to the local server.  Make sure you are logged onto the proper server.

Monday, August 4, 2014

BizTalk Health Monitor - Using Older Versions of BizTalk

BizTalk 2013 R2 was just released not long ago.  One of the new tools that comes with BizTalk 2013 R2 is the BizTalk Health Monitor:




To install:
http://blog.jeroenmaes.eu/2014/06/enable-biztalk-health-monitor-biztalk-2013-r2/

In a nutshell, the BHM incorporates:
  • Integration of the MessageBox Viewer tool
  • Performance analysis thanks to integrated PerfMon
  • Easy SQL Job Status monitoring
  • Automatic Archiving of Health reports

Finally, some consolidation of common BizTalk tools (and a few other things as well)…

If you look at the second link from above, you will notice the following:

“Note that BHM will work also with previous versions of BizTalk targeting any versions of BizTalk groups since BizTalk 2004 ;-) “

Cool…  So of course I had to try it against my BizTalk 2010 instance (using the last link for install instructions).  And... it works!

I downloaded the BizTalk 2013 R2 iso file via MSDN.

Update:  The BHM is now available as a separate download.  See http://blogs.msdn.com/b/biztalkhealthmonitor.

Friday, July 11, 2014

Reserved words in BizTalk Orchestrations - unexpected keyword

Some of you have come across the "unexpected keyword: " issue with the use of XML schemas within BizTalk, and it's fairly well posted:


This solves many things.  However, you may also find issues with parts that do not really relate to the schema itself.  See the below screenshot.


I have selected the actual xsd file within the Visual Studio Solution Explorer and took a look at the properties.  You may notice the the Namespace property of the file is BizTalk_Server_Project2.response.

This is a project-level value, as you won't see this value in the schema file itself.  So let us say we have a 'MySchemas' project and a 'MyOrchestrations' project within a BizTalk solution.  You reference the MySchemas to the MyOrchestrations.  When you compile the MySchemas, you will end up with a .NET class with the .NET namespace of the above.  You may likely now get an error when trying to build the MyOrchestrations.

Sandro Pereira eludes to the same issue in a slightly different way, but the root of the issue is the same - reserved words (whether .NET or XLANG or whatever the technology):




As always, put some thought into ensuring you have unique naming conventions within your solution.

Monday, April 7, 2014

XML Schema Deep Dive

I came across this link (http://social.technet.microsoft.com/wiki/contents/articles/19692.biztalk-server-deep-dive-in-schema-design.aspx) which does a nice job of giving a deeper overview of XML schema capability and design.

Although this comes across as BizTalk-centric, my thoughts are that the overall information is W3C compliant. This means that Microsoft, in their own way (in this case, using BizTalk as the schema editor) which does try to conform to XML standards. This is a good thing, of course.  Any tool could be used, for that matter, such as Altova's XMLSpy, to develop XML schemas (.xsd file).

I've seen many developers load in an XML document and immediately using xPath to get information.  Depending on the scenario/environment, this may lead to issues that are difficult to debug, especially when working with complex (and sometimes slightly different) XML documents.  It's much better to bring in and XML document and use it as a 'type' via a schema, rather than treating it similar to a text file with 'some extra functionality'.

Also, another site (http://www.xfront.com/BestPracticesHomepage.html) which discusses some best practices...

Friday, January 3, 2014

The .NET way of changing a namespace within an XML Document

I'm on a project which receives messages and 'normalizes' the namespace of the XML message. You can do this easily in BizTalk by using the ESB pipeline components (ESB Add Namespace and ESB Remove Namespace).

If you aren't using BizTalk, you can do this in .NET as well, using the RemoveAttribute and SetAttribute methods. You may also wish to attempt the NamespaceURI.Replace method as well.

If you look closely at the documentation for the RemoveAttribute and SetAttribute link, at the bottom you will see
Therefore, changing xmlns attributes will have no affect until you save and reload the XmlDocument object.
Put a trace or watch in the Locals viewer and step through within Visual Studio debug mode. You will find out quickly that the xml document doesn't change if you try to modify the XmlDocument directly in code.

The way to accomplish this is by creating a placeholder XmlDocument variable, copy the contents of the original XmlDocument, and at then replace the namespace.  This can be done with a few lines of code: