Friday, January 8, 2021
Automate BRE Xml File Import - PowerShell Code, Part 2
Friday, October 30, 2020
Automate BRE Xml File Import - PowerShell Code
There isn't a lot of information on how to automate a BRE xml file import. Many suggest using the PowerShell provider that used to be available on Codeplex, but unfortunately it's now a dead page. There are other resources (QuickLearn) that explain using assemblies in the BizTalk SDK folder. These have been available since BizTalk 2013. It appears that what was available in Codeplex has been lifted and shifted to the BizTalk installation process (to the SDK subfolder) on your local server.
This is all good, but there are some limitations. First, Microsoft doesn't offer any real documentation for the PowerShell SDK. I had to revert back to a different site to get documentation (version 1.4.0.1, published back in 2014 by Randal van Splunteren and Maxime Labelle) for the PowerShell Provider documentation.
Second, upon looking at this documentation, the BizTalk PowerShell provider doesn't allow for the creation (or importing) of certain artifacts. See screenshot below, highlighted area.
So I came across this on the Microsoft website which had a C# snippet of doing what I wanted. I decided to convert it to PowerShell, since the rest of my processes use PowerShell for builds, releases, and deployments. I now can now script this out to automate deployments of BRE xml files. In my case, I use Azure DevOps to perform this using a 'PowerShell on Target Machines' task.
So here's my version of that - use and modify at your own risk. Enjoy!
Tuesday, April 9, 2019
TFS Release Notes Write-Verbose continued
One of the issues we ran into was executing a command but we weren't getting the output. In my case we were trying to call BizTalk's btstask.exe command in a PowerShell script, which was saved to the local remote server. Every now and then we'd get an error, but it wasn't properly displaying in the TFS Release log. So we would need to manually run the command by remoting into the server and then look at the output to verify what the issue was (typically we didn't delete existing suspended messages, so we couldn't update the application).
Initially I thought capturing the output via a variable would work, like so:
$myBtsTaskOutput = BTSTask.exe AddResource /Source:$srcPath /ApplicationName:$appName /Type:$resType /Overwrite /Destination:"%BTAD_InstallDir%\$outputFile" $gacOptions
However, when I tried to capture the output by using:
Write-Verbose $myBtsTaskOutput -Verbosethe command would fail with an object error. Even using .ToString() failed.
The proper way to get the output is to immediately cast the type of the variable to a string, like so:
[string]$myBtsTaskOutput = BTSTask.exe AddResource...
Now the Write-Verbose command will give you the proper output. Super simple :)
Thursday, October 25, 2018
Notes on building a BizTalk Build and Release Server using TFS
Here are the background/requirements/setup:
- Visual Studio 2015 is being used to develop the BizTalk 2016 solutions
- We use two NuGet repositories: The main NuGet repository, and our own company-based repository. I'll discuss the NuGet setup in a later post.
- We are using the MSBuild. XAML is not being used.
- Deployment Framework for BizTalk 5.7 is being used.
- Use of some simple, custom PowerShell scripts on local server is secondary method when BTDF doesn't make sense.
- Right now I have just one agent set up. I really haven't come upon a need to add more. It's easy enough to add more agents, so there's no need to fret on this.
I did follow a good portion of what Vikas Bhardwaj put in his blog - it's a fantastic resource and got me going in the right direction. Please take the time to read all parts, as there is good information in each one.
- You will need to ensure you have opened ports HTTP, HTTPS, and Windows Remoting (80, 443, 5985, and 5986) between the build server and the destination servers when using the build server to do Releases/Deployments. The port for PowerShell is needed only if you are going to perform a 'PowerShell on Target Machines' task. This has been a huge plus, as I'll explain later.
- Since we are using BizTalk 2016, I installed Visual Studio 2015. Make sure you install all the necessary components (i.e. C#, VB.NET, whatever) you require in order to do a proper solution build. For me, I left the default values alone.
- Install BizTalk 2016, but select only the following:
- Just the Developer Tools and SDK
- Server Runtime (and all sub-categories) - you need this to compile a BizTalk solution.
- Administration Tools And Monitoring (and all sub-categories)
- Additional Software
- Enterprise Single Sign-On Administration
- Enterprise Single Sign-On Master Server
- BAM Client
- Project Build Component
- Allow the installation wizard to install the Prerequisites
- DO NOT CONFIGURE BIZTALK !!!!!
- Note on the above: my interpretation is that a BizTalk license isn't necessary. See BizTalk Pricing. Send me a note if that is incorrect.
- We installed the Deployment Framework for BizTalk, version 5.7. Select everything but the samples.
- We found that we came across two scenarios for building/deploying BizTalk artifacts:
- Large, complex applications worked very well under the BTDF
- Small projects (single map, receive location, etc) worked just fine running the btstask.exe command via a PowerShell on Local Server task. This is where opening port 5985 was critical. PowerShell remoting (WinRM) is VERY powerful.
- Installing an Agent wasn't difficult. However, having the proper rights/permissions was difficult. Make sure you have a good relationship with your server admins. ☺☺
Wednesday, December 20, 2017
Microsoft TFS Build and Release Notes - Write-Host errors in PS Script
There is, however, a good and bad way to log in a PowerShell script in VSTFS 2015/2017.
In your PowerShell tasks, do NOT use Write-Host, as this will fail when you try to run a Build or Release.
Instead, use Write-Verbose in the PS script and you will get the results you need.
Tuesday, October 3, 2017
BizTalk 2016 - WCF-Custom Transport: Unrecognized attribute 'ApplicationName'
I have a send port that uses the WCF-Custom/sqlBinding type as a send port on my local dev machine. Everything works as expected, so I exported the MSI and bindings to use on our Test environment.
When I imported the bindings file, I received an error: "Error loading properties. (System.Configuration.ConfigurationErrorsException) Unrecognized attribute 'ApplicationName'.
I compared the settings on my local dev to the Test environment and noticed some differences. This screenshot is from our Test environment
and this is from my local dev environment
Thursday, August 17, 2017
Cannot delete a send handler that is used by a send port
- Create a new Host and Host Instance with the desired name.
- Once that was done, add the Host to the appropriate adapter. For us it was adding it to the FILE adapter.
- Then re-configure all appropriate send ports to use the new Host Instance.
Restart Host Instances, and the send ports were properly using the newly created Hosts.
Once that was done, the thought was that the old host could be deleted. We came across a few things (some are obvious, others aren't so):
- In the BizTalk Admin Console, select Applications/All Artifacts
/Send Ports - Sort by Transport Type. Here is where you will see some that don't have the Transport Type as blank. These are your Dynamic Ports.
- For each Dynamic Port, open one up and, in the General tab, select 'Configure'.
- You will see a list of Send Handlers associated to each Adapter. You will need to make your changes in this list according to how you made your changes with your static Ports.

FYI - This was tested in BizTalk 2016. I don't believe you can navigate the same way in BizTalk 2010.
Tuesday, May 23, 2017
TFS Build Error - works in Command Prompt, doesn't work in TFS Build Definition
I was able to successfully build a project off a DOS prompt (it's now called the "Developer Command Prompt" in more modern Windows Servers... ☺) using the MSBuild.exe command. However I was not able to get a successful build when using the MSBuild step. I even went a little further and decided to use the Command Line step in the definition and typed in the exact same command line that I used off the dos prompt above. Even that didn't work.
The errors I kept on seeing in the log files were ambiguous to me - a whole bunch of ICE and LGHT errors. Almost all of them referred to an "incorrectly registered scripting engine":
The errors didn't make much sense to me. So instead of getting too focused on the error messages, I took a step back and reviewed what is going on: I was using the same exact command line - one using via the local server Command Prompt (success), and the other being called within TFS Build Command Line step (fail).
Conclusion: permissions
I talked with a co-worker about this (he is a bit more seasoned with TFS Builds) and one of the first question he asked was: "How is your Agent set up?"
Looking at a few things we (he) figured it out. My settings.json file in the appropriate agent folder was not properly set up. In particular, the "WindowsServiceName" setting was blank. I *think* that if it's blank, it uses 'Local Service' or something like that. Either way, it's not a best practice.
I reconfigured the agent to use the appropriate service. I reviewed the updated configuration to make sure that the setting was now populated. After that my MSBuild step was running without a hitch.
Wednesday, April 20, 2016
Side by side SQL versions, Entity Framework, and SQL Aliases
I have been developing an application which has an app.config file with a SQL Server connection string (code first Entity Framework). Since the solution I'm working on it in source control (TFS), it's shared; so I decided to make an alias for use in the connection string. That way, others developing on this project can use their own local database without changing the configuration file (and then checking their connection string changes back in and irritating the rest of the dev group). The alias that I created for my local laptop points to the SQL 2012 instance.
I opened up SQL Server Configuration Manager and created an alias to my 2012 instance (.\SQL2012). However, when the update-database command is executed, the database was deployed to the root instance (SQL 2008), and not the 2012 instance that I wanted it deployed to (as identified in my SQL Alias). I restarted the SQL Server service, but still got the same result when I called the update-database command again.
What I didn't realized is that I was using the SQL Server 2008 version of SQL Server Configuration Manager, not the 2012 version. The 2008 version won't show the 2012 services but 2012 version will show all services.
To better explain, this screenshot is what I see when I run the 2008 version:
and this is the screenshot from the 2012 version:
Notice that 2012 version displays the SQL Server 2012 instance of the services. Because I was using the 2008 version of the Configuration manager, I was unknowingly restarting the root server (2008) service, not the 2012 service.
After I restarted the 2012 instance, all worked just fine.
As far as Entity Framework is concerned, it seems to me that should expect an error that an instance can't be found when using the update-datebase command. It would have been nice to see an error at a minimum (i.e. "Cannot find SQL instance, defaulting to root").
See https://msdn.microsoft.com/en-us/library/ms174212.aspx for exact locations of the specific version of SQL Server Configuration Manager.
Thursday, February 11, 2016
BizTalk BAM XML Import/Export Gotcha
I needed to make some updates to one of the BAM Activities, so I opened up a blank Excel document and selected "Import XML..." The BAM Activities and Views load up just fine.
However, there was one big issue that I came across. Sadly, the PivotTable and RealTimeAggregation information did not get loaded. If I didn't take the time to review and compare my updates with the last version of the file (we use TFS as a code repository), I would have never known that I might have pushed an incomplete BAM process (compared to the previous version) into our database. See the new output below.
As another test, I decided to do an import and then a direct export without any modification. Sure enough, the RealTimeAggregation and PivotTableViews within the exported XML document were missing.
To correct this, I needed to do some reverse-engineering/trial and error with creating a new PivotTable within the Excel document and re-export the XML file out. Then I needed to compare the old XML with the new XML to make sure I had the PivotTables and Aggregations the same. At this point I decided to put the Excel document into source code as well... :)
Not sure if this is a bug, an oversight, or not the intent of the Import XML feature within Excel, but I would think that one could get 'mildly' burned if the source xlsx file isn't saved to your code repository.
Thursday, June 4, 2015
BizTalk HTTP Adapter has NOT been deprecated
Microsoft has several postings which document deprecated adapters:
- What's New in BizTalk Server 2013 and 2013 R2
- BizTalk 2010 Deprecated Features, Tools, and APIs
- BizTalk 2009 Deprecated Features, Tools, and APIs
- BizTalk 2006R2 Deprecated Features, Tools, and APIs
None of the above Microsoft links discuss HTTP. I would suspect that there is some confusion because the SOAP adapter is indeed deprecated.
That being the case, I decided to open a ticket with Microsoft Support and get the story directly from Microsoft. Here's the reply:
Hello Bruce,
Thank you for contacting Microsoft Support. My name is Nikhil Jain. I am the Support Professional who will be working with you on this Service Request. You may reach me using the contact information listed below, referencing the SR number (xxxxxxxxxxx).Regarding your query I would like to inform you that Microsoft never deprecated HTTP adapter. We still support it. It is the SOAP adapter which is deprecated not the HTTP. I tried calling you @ +1 (xxx)xxx-xxxx however reached voice mail.If you have any questions or concerns, please let me know.
Thanks and RegardsNikhil Jain| Support Engineer | Microsoft Developer Support | BizTalk Server, WFM and MSMQ
So this puts the question to bed. The legacy HTTP adapter is still supported and is NOT deprecated.
Although it does make sense to migrate to WCF, there isn't a need to rush the migration from HTTP to WCF as some would think.
Friday, October 10, 2014
Capturing BizTalk Perfmon Values with PowerShell - Part 1
One of the things leadership tends to ask I.T. is what kind of value is BizTalk bringing to the company. In my case, of course, the business leadership is asking: how many, how often, when, etc.
In my case, I've been tasked to do a few things: determine what kind of load we have on the BizTalk servers. A couple of reasons why:
- Throughput/load. Management wants to understand "do we have a 'right-sized' environment?" Keep in mind that Microsoft overhauled their licensing model not too long ago. Instead of license per CPU, you now are being charged by the core. This may cause a huge jump in licensing costs. If BizTalk is constantly under 10% utilization, it may make sense to downsize the environment (by core, at least). Or, in a worse-case scenario, show how much is really going through BizTalk to justify scaling up your BizTalk environment further.
- Understand which processes, applications, ports, etc. within BizTalk are being highly taxed or under-taxed. In our case, we 'lease' our BizTalk environment to different entities within our organization. We need to capture data to make sure we are properly charging our internal customer/entity base.
So knowing and understanding what counters exist is great, but viewing the BTHM Performance counters is a bit inefficient and really only gives you a general snapshot of about (default) 100 seconds . Why not capture some performance counters and store it somewhere, either as a file or directly into a database?
Side Note: Light reading for those who are interested in documentation on some of these counters.
Note: You will likely need to force PowerShell version 2 for the extensions to work.Moving on. You can either install SQLPSX or just copy the psm1 module to your environment. For me, I just copied the module over and used the Import-Module command.
I wanted a quick list of ALL BizTalk counters and save them to a file. Easy to do:
Opening the text file shows a rather comprehensive list.
Taking that one step further, I can now create an XML configuration file by using the command
And the results:
Warning: Before one gets too crazy here, make sure you either pair down the configuration file or add some extra filters to the PowerShell command when you generate a config file. Please be selective with what you really want to monitor.
With that out of the way, we now can go to the next step (and the next blog) - gathering BizTalk performance data.
Monday, September 8, 2014
WCF-SQL Transport Component is not registered correctly
- It may be in your best interest to keep all servers as similar as possible and let the host instances dictate which servers run what BizTalk hosts.
- Even though the settings within the BizTalk Administration Console are stored in the BizTalk databases, it does indeed verify a few things to the local server. Make sure you are logged onto the proper server.
Monday, August 4, 2014
BizTalk Health Monitor - Using Older Versions of BizTalk
http://blog.jeroenmaes.eu/2014/06/enable-biztalk-health-monitor-biztalk-2013-r2/
- Integration of the MessageBox Viewer tool
- Performance analysis thanks to integrated PerfMon
- Easy SQL Job Status monitoring
- Automatic Archiving of Health reports
Update: The BHM is now available as a separate download. See http://blogs.msdn.com/b/biztalkhealthmonitor.
Friday, July 11, 2014
Reserved words in BizTalk Orchestrations - unexpected keyword
- http://bencode.wordpress.com/2008/02/13/reserved-words-and-distinguished-fields/
- http://www.power-programming.co.uk/post/2009/03/05/BizTalk-Orchestration-error-unexpected-keyword-request.aspx
- http://brucetank.blogspot.com/2011/02/biztalk-xml-schema-invalid-type-name.html
This is a project-level value, as you won't see this value in the schema file itself. So let us say we have a 'MySchemas' project and a 'MyOrchestrations' project within a BizTalk solution. You reference the MySchemas to the MyOrchestrations. When you compile the MySchemas, you will end up with a .NET class with the .NET namespace of the above. You may likely now get an error when trying to build the MyOrchestrations.
Sandro Pereira eludes to the same issue in a slightly different way, but the root of the issue is the same - reserved words (whether .NET or XLANG or whatever the technology):
As always, put some thought into ensuring you have unique naming conventions within your solution.
Monday, April 7, 2014
XML Schema Deep Dive
Although this comes across as BizTalk-centric, my thoughts are that the overall information is W3C compliant. This means that Microsoft, in their own way (in this case, using BizTalk as the schema editor) which does try to conform to XML standards. This is a good thing, of course. Any tool could be used, for that matter, such as Altova's XMLSpy, to develop XML schemas (.xsd file).
I've seen many developers load in an XML document and immediately using xPath to get information. Depending on the scenario/environment, this may lead to issues that are difficult to debug, especially when working with complex (and sometimes slightly different) XML documents. It's much better to bring in and XML document and use it as a 'type' via a schema, rather than treating it similar to a text file with 'some extra functionality'.
Also, another site (http://www.xfront.com/BestPracticesHomepage.html) which discusses some best practices...
Friday, January 3, 2014
The .NET way of changing a namespace within an XML Document
If you aren't using BizTalk, you can do this in .NET as well, using the RemoveAttribute and SetAttribute methods. You may also wish to attempt the NamespaceURI.Replace method as well.
If you look closely at the documentation for the RemoveAttribute and SetAttribute link, at the bottom you will see
Therefore, changing xmlns attributes will have no affect until you save and reload the XmlDocument object.Put a trace or watch in the Locals viewer and step through within Visual Studio debug mode. You will find out quickly that the xml document doesn't change if you try to modify the XmlDocument directly in code.
The way to accomplish this is by creating a placeholder XmlDocument variable, copy the contents of the original XmlDocument, and at then replace the namespace. This can be done with a few lines of code:
Friday, September 6, 2013
PowerShell for BizTalk Administration: Suspended Message Counts
Sometimes, one of our internal departments expects a message to come in at a particular time. There are times, however, that the message was never sent to BizTalk for processing. Either way, the "I don't see an expected message, can you see if it failed?" routine is communicated our way. Keep in mind that we already have put mechanisms in place to notify if something failed... :)
At any rate, I've developed a quick PowerShell script to see if anything indeed has suspended. This one uses SQL, and it hooks into the BizTalk MessageBox Database; make sure permissions are set accordingly. This post is rather code agnostic, but the approach is to use PowerShell to be consistent with the other scripts which our group has put in place.
The query is a common, read-only script that you may have seen elsewhere:
The next step is to do the traditional SQL routine, create the SQL connection, open it, put it into a SQL adapter, and so on.
Now you have a list of all suspended messages in a typical .NET DataSet. From there we can slice and dice the information. I like sorting and grouping everything out, and PowerShell does a fantastic job of that.
The result is a list of unique BizTalk applications in your environment with a suspended message count for each application. If there aren't any suspended messages, obviously you won't have any rows in your DataSet, and you can notify your customers that everything is operating as expected in BizTalk. ;)
Saturday, August 31, 2013
Friday, August 30, 2013
PowerShell for BizTalk Administration: Is this thing on? Part 3
Looking at the Host Instances (not Hosts) at a high level, they are nothing more than a Windows Service on a server. So two ways to get information on a service via PowerShell is use use either the Win32_Process or Win32_Service objects using the PowerShell Get-WMIObject command. However, there is a unique WMI object specifically for BizTalk: MSBTS_HostInstance. I chose the last one for my script. This does assume you have this class installed on the server you are making the call from.
Instantiating the object is a one-liner (no need to establish the SQL Server, as in the previous post). At that point you can loop through each host instance and get the state of the Host Instances.











