Submit XML message to BizTalk WCF Receive Web Service

I have been exposing orchestrations and schemas as WCF Web Services and I've been struggling recently in submitting messages to those web services. 
The usual approach I use is to create a test harness (a console app or win forms) and create a service reference to it. Then I go through the steps of creating the request message. My particular issue was trying to create an envelope with an embedded message.
I already had a sample XML message that worked and I found this excellent post which explains how to post an XML message to a WCF Web Service in BizTalk.  No service reference is required and there are very few lines of code. I'm going to use this method a lot more particularly where I have big messages because setting the value of each element can be time consuming.

Follow the steps in the post because they worked for me.  I'm reposting because I wanted to include the code for submitting to a WCF with basicHttp binding. Note the all important MessageVersion which must be set to Soap11.

public class SendBizTalk

{
[ServiceContract()]
private interface IBizTalkSubmission
{
         [OperationContract(Action = "*", ReplyAction = "*")]
         void Submit(Message msg);
}

static void Main(string[] args)
{
     XmlTextReader xmlrdr = new XmlTextReader(@"C:\Projects\Test\FedESB.Test1\Envelope2.xml");
    Message msg = Message.CreateMessage(MessageVersion.Soap11, "*", xmlrdr);

    string uriLocationEsbOnRamp = "http://localhost/FedESBWcfService2/Receive.svc";

     BasicHttpBinding b = new BasicHttpBinding();
     b.Security.Mode = BasicHttpSecurityMode.None;
     EndpointAddress epa = new EndpointAddress(uriLocationEsbOnRamp);
     IBizTalkSubmission proxy = ChannelFactory<IBizTalkSubmission>.CreateChannel(b, epa);
     proxy.Submit(msg);
}
}

BizTalk Deployment Framework

I've been using the BizTalk Deployment Framework in particular for highly available environments where there are 2 Receive hosts, 2 Send hosts and 2 Process hosts.  The 2 receive hosts are clustered which works well when you have orchestrations exposed as WCF web services because you can install the virtual directories on both servers and not have a single point of failure. 

The Deployment Framework allows you to deploy your BizTalk assemblies to the GAC and optionally to install them to the BizTalk management database.  So in this 6 server arrangement you select to deploy to the BizTalk management database on one of the servers and deploy just the assemblies to the GAC on the other 5.

Now the Deployment Framework provides the ability to install your virtual directories as part of the deployment but here is where you hit your first problem. It only does so when you select "deploy to management database" as true which means you only get them deployed onto one server.  And your deployment instructions need to be clear about which server you install the virtual directories on. 

I've deployed a lot of web services so I have experience in creating virtual directories and application pools.  The APPCMD is my best friend and can be called using

%systemroot%\system32\inetsrv\APPCMD

So I started to create a Windows CMD file called CreateVDir.cmd.  And it occurred to me I can use the %COMPUTERNAME% to specify which server I want to execute commands on.

IF NOT "%COMPUTERNAME"=="%RECEIVEHOST1%" GOTO END
IF NOT "%COMPUTERNAME"=="%RECEIVEHOST2%" GOTO END

In this case I am passing RECEIVEHOST1 and RECEIVEHOST2 as parameters to the CreateVDir.cmd along with the Virtual Directory name, the physical path and the user name and password for the Application Pool account.  And the great thing is I can have these as environment settings in the SSO and pass them into CreateVDir.cmd as $(VDIR_UserName) $(ReceiveHost1) etc.

So I just need to create a Target in BTDFPROJ as
<Target AfterTargets="" Name="CustomDeployTarget">
<Exec command="..\CreateVdir.cmd <vdirname> <physpath>  $(VDIR_UserName)  $(VDIR_UserPass) $(ReceiveHost1)  $(ReceiveHost2)"/>
</Target><

I also need to create a CustomUndeployTarget which calls my DeleteVdir.cmd file and then I have all the control I need. I like this approach. I keep my BTDFPROJ file simple and I have CMD files that I can create and test independently of the deployment process.

Good old DOS commands!

Create IIS Virtual Directory using APPCMD

The command line script below allows you to create a Virtual Directory from an existing physical directory and set up an application pool.  In this case the application pool will be running under a domain service account. 

To use an identity of Network Service then replace
/processModel.identityType:SpecificUser  ...
with
/processModel.identityType:NetworkUser


@echo OFF

IF "%1"=="" GOTO Syntax
IF "%2"=="" GOTO Syntax
IF "%3"=="" GOTO Syntax
IF "%4"=="" GOTO Syntax

set VDIRNAME=%1
set PHYPATH=%2
set USERNAME=%3
set PASSWORD=%4

REM Create Application Pool

%systemroot%\system32\inetsrv\APPCMD add apppool /name:%VDIRNAME%AppPool

%systemroot%\system32\inetsrv\APPCMD set apppool "%VDIRNAME%AppPool" /managedRuntimeVersion:v4.0

%systemroot%\system32\inetsrv\APPCMD set apppool "%VDIRNAME%AppPool" /managedpipelineMode:Classic

%systemroot%\system32\inetsrv\APPCMD set apppool "%VDIRNAME%AppPool" /processModel.identityType:SpecificUser /processModel.userName:%USERNAME% /processModel.password:%PASSWORD%

REM Add Virtual Directory

%systemroot%\system32\inetsrv\APPCMD add app /site.name:"Default Web Site" /path:/%VDIRNAME% /physicalpath:"%PHYPATH%"

%systemroot%\system32\inetsrv\APPCMD set app "Default Web Site/%VDIRNAME%" /applicationpool:%VDIRNAME%AppPool

:SYNTAX
ECHO.
ECHO VDir Name and Physical Path Required
ECHO.
ECHO CreateVDir.CMD VDirName C:\PhysPath Domain\UserName Password
ECHO example CreateVDir TEST c:\inetpub\wwwroot\test buildx\btsuser Password1

Execute Permission denied on object 'bts_ProcessHeartbeat_....'

It's always worth reposting a useful tip. This one was written by Paul Edwards in 2010 and I came across the problem recently. It occurs when you are using a Host Instance for an Isolated Host.  The full text of the error is

The following stored procedure call failed: " { call [dbo].[bts_ProcessHeartbeat_BizTalkServerIsolatedHost]( ?, ?, ?)}". SQL Server returned error string: "EXECUTE permission denied on object 'bts_ProcessHeartbeat_BizTalkServerIsolatedHost', database 'BizTalkMsgBoxDb', schema 'dbo'.".


The background is when you create a new Host then not only is the Host added to the BizTalk Management database but it also creates a stored procedure called 'bts_ProcessHeartbeat_'.
A new Database Role is also created called _USERS which includes the AD Group Name you specified when creating the Host.  When you create a Host Instance the Login account you use needs to be a member of this group in order to be granted Execute permissions.

But here's the issue: when using an Isolated Host (for recieve handlers of SOAP and HTTP), the stored procedure is called by the Application Pool account that your web site or virtual directory is running under. So you either need to make this account the same as the Host Instance login account or add it to the group you used when creating the Host. 

BizTalk Hosts, Host Intances and Groups

People new to BizTalk struggle with the concept of Hosts and Host Instances. I've tried to find a simple article that I can direct people to but the articles I've read IMHO don't seem to provide a simple explanation.  So let me have a go.

When you create a new Host in BizTalk all you are doing is creating an entry in a table in the BizTalk Management database and nothing more.  You can attach an Orchestration to this Host, but it won't run until you create a Host Instance.  When you create a Host Instance you actually create a Windows Service and this is what does the work.  So you can start and stop Host Instances but Hosts have no such concept.

So what is the point of a Host? The answer is that its the way BizTalk offers us scalability. Let me provide the analogy of load balancing web servers. As you know you create 2 web servers that have different IP addresses.  And then someone sets up a load balancer for you and gives you the load balanced IP address (or the equivalent DNS entry).  When I point to the web server I use the load balanced IP address.  Same thing with BizTalk Hosts.  The Host entry is my load balanced "address".  Now it should make sense as to why I bind my Orchestrations to the Host name and NOT the Host Instance.  Its so I can get load balancing.  Now obviously with just 1 BizTalk Server this doesn't make a lot of sense so the next thing to understand is BizTalk Groups.

You can have several BizTalk Servers that belong to the same BizTalk Group.  So is it analogous to a load balanced cluster? No, not exactly, its more of a mechanism for distributing the processing across multiple servers.  They key point to understand is that there is only 1 copy of the BizTalk databases sitting on the SQL Server.  They are created when I configure my first BizTalk Server. 

When I add the second BizTalk Server and configure it I select the option to "Join an existing Group".  When configuration is complete and I open the BizTalk Admin console on my second server I see the Hosts that are created on Server 1.  Remember a Host is just a database entry and members of a BizTalk Group share the same database. 

Now here comes the magic bit. I can create a Host Instance on each of these two BizTalk Servers that will run my orchestrations.  So each now has a Windows Service running and BizTalk does the load balancing for me. 

Actually that is NOT how you should start out along the scalability route. There are some good articles on best practices for scaling BizTalk Server. The first configuration is 3 BizTalk Servers in a Group, one configured as a Process Host, one as Receive Host and one as the Send Host.  That's the first step in scalability.  You can then go on to add more servers to the group and use BizTalk's load balancing capability.  But this is where I'm going to stop because I'm not going to repeat what is in those excellent articles. 

Hope that helps.

Replacing tempuri.org in a WCF Web Service

I keep forgetting how to do get rid of tempuri.org from a WCF web service.  And I also forget how to remove "Field" from the end of the properties on my class. This post deals with both.

Firstly, removal of tempuri.org I got form this blog.
The steps again are

1.) In your ServiceContract attribute constructor define the Namespace property.

[ServiceContract(Namespace = "http://myservice.com")]
public interface IMyService

2.) For your ServiceClass create the ServiceBehavior attribute with Namespace property

[System.ServiceModel.ServiceBehavior(Namespace = "http://myservice.com")]
class MyService : IMyService

3.) In all your bindings set the bindingNamespace property

<endpoint binding="basicHttpBinding" bindingNamespace="http://myservice.com".... 

4.) To remove the word "Field" from the end of the properties

In your ServiceContract attribute constructor define the XmlSerializerFormatAttribute  property.

[Systme.ServiceModel.XmlSerializerFormatAttribute()]
public interface IMyService

5.) If you have an array of objects defined in your class and you return that array in your ServiceContract the previous step will give you an "Array" prefix.  To remove this in your ServiceClass comment out the line

//[System.Xml.Serialization.XmlTypeAttribute(AnonymousType = true, Namespace = http://myservice.com)]

 

 

 

WCF Web Services called from BizTalk

I have a BizTalk orchestration which is calling a WCF Web Service.  Now I'm processing lots of messages at once (about 8,000 at a time) but the web service was a bottle neck.
 
During testing we noticed that only 10 connections were opened to the web service at any one time.  As a result we were getting timeout issues and orchestrations were getting dehydrated.  We tinkered around in IIS but try as we might we could not figure out a way of increasing the number of connections. 

Then my colleaguse found this article which reports much better performance if you use net.tcp rather than http and the former has binding properties that can control the number of connections. You need to be cautious because there are quite a lot of factors to consider before changing protocols.  The advice is always to perform tests to determine which is the right protocol to use. 

I've not used net.tcp before so there are a couple of things you need to do first.
1. This will only work on Windows Server 2008 and R2

2. You need to install the feature of .Net 3.5 Framework WCF Activation to support non-HTTP protocols.  By default this is not selected, so you will need to go and install it from Server Manager.

3. One you have done that, you can add the net.tcp biding to the web site where your WCF web service is installed.  Note: keep both protocols: net.tcp and http for the web site.  Do this by editing the bindings and adding net.tcp with Binding Information like 80:*. But also go to Advanced Settings and change Enabled Protocols to htp,net.tcp.

4. Next you need to add the binding to the web.config of the WCF web service.  I did this using NotePad.  Keep the original basicHTTP binding in place because that way you can still check the the web service will load in the browser and you can see the WSDL. I've provided an extract from my web.config below.

<bindings> <netTcpBinding> <binding closeTimeout="00:01:00" hostNameComparisonMode="StrongWildcard" maxBufferPoolSize="524288" maxBufferSize="65536" maxConnections="100" maxReceivedMessageSize="65536" name="NetTcpBinding_IADService" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" transferMode="Buffered"> <readerQuotas maxArrayLength="16384" maxBytesPerRead="4096" maxDepth="32" maxNameTableCharCount="16384" maxStringContentLength="8192"/> <security mode="None"> <transport clientcredentialtype="None" protectionLevel="None"/> <message clientCredentialType="UserName"/> </security> </binding> </netTcpBinding> </bindings>

<service name="WCFADService.ADService" > <host> <baseAddresses> <add baseAddress="net.tcp://localhost/WCFADService/ADService.svc"/> <add baseAddress="http://localhost/WCFADService/ADService.svc/mex"/> <add baseAddress="http://localhost/WCFADService/ADService.svc"/> </baseAddresses> </host> <endpoint address="" binding="netTcpBinding" contract="WCFADService.IADService" bindingConfiguration="NetTcpBinding_IADService" /> <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" /> </service>

As far as the endpoint url for a net.tcp web service is concerned you can just replace the http:// with net.tcp://

You can test this in the usual way with a console application by using the net.tcp endpoint in the service reference. 

As far as consuming the net.tcp web service in BizTalk is concerned you can use the same consume web service wizard.  That will give you a Custom Binding xml file which you can import into BizTalk.  The magic is then when you click on Configure of the Send Port you get the ability to change the number of connections.

There are two properties here maxConnections and listenBacklog and we set them both to 50.  Sure enough when we tested this out I got 50 simultaneous connections. 

One last point is that running the web service like this hammered the CPU.  You would be well advised to put your web service on another server separate from the BizTalk Server.