When trying to write a scalable web service, you need to be aware of a few properties that affect how the WCF runtime will dispatch requests to your service: InstanceContextMode and ConcurrencyMode. In a nutshell, InstanceContextMode controls when a new instance of your service type is created, and ConcurrencyMode controls how many requests can be serviced simultaneously. The default InstanceContextMode is InstanceContextMode.PerSession, and the default ConcurrencyMode is ConcurrencyMode.Single.

Others have covered the details of these two knobs in detail, you can check them out for more background. Here I’m simply going to explain the affect these settings can have on your threading behavior.

If you have set ConcurrencyMode == ConcurrencyMode.Single, then you don’t have to worry about your Service instances being free-threaded (unless you are doing concurrent code within your methods). The only time multiple calls are allowed is when there are multiple instances. For InstanceContextMode.Singleton, you will get one method call at a time since you only have a single instance. For InstanceContextMode.PerCall or PerSession, ServiceModel will spin up extra threads up to a throttle in order to handle extra requests. There’s one possibly unexpected twist here. That is, when using a session-ful binding there will only be one outstanding instance call per-channel. Even with InstanceContextMode.PerCall. This is because WCF strictly maintains the in-order delivery guarantees of the channel with ConcurrencyMode.Single. So when using a session-ful binding (i.e. the default NetTcpBinding or NetNamedPipeBinding) + ConcurrencyMode.Single, InstanceContextMode.PerCall and InstanceContextMode.PerSession will behave the exact same way from a server-side threading/throttling perspective with regards to a single channel.

ConcurrencyMode == ConcurrencyMode.Reentrant is similar, except you can trigger another call to your instance from within your service (let’s say you call into a second service who calls back into you before it returns).

When you have ConcurrencyMode == ConcurrencyMode.Multiple, threading comes heavily into play. WCF will call into your instance on multiple threads unless you are using InstanceContextMode.PerCall. Throttles again will come into play (that’s a topic for another post).

To summarize, here are some basic scenarios for what happens when 100 clients simultaneously hit a service method:

Scenario 1: InstanceContextMode.Single+ConcurrencyMode.Single
Result: 100 sequential invocations of the service method on one thread

Scenario 2: InstanceContextMode.Single+ConcurrencyMode.Multiple
Result: N concurrent invocations of the service method on N threads, where N is determined by the service throttle.

Scenario 3:InstanceContextMode.PerCall+Any ConcurrencyMode
Result: N concurrent invocations of the method on N service instances, where N is determined by the service throttle

Here are some more “under the hood” details in response to the following question:

How is one supposed to interpret the NetTcpBinding.MaxConnections property on the client? My assumption has been that setting this property at the client only allows this number of concurrent connections, and further connection attempts will be queued until an existing connection is released.

MaxConnections for TCP is not a hard and fast limit, but rather a knob on the connections that we will cache in our connection pool. That is, if you set MaxConnections=2, you can still open 4 client channels on the same factory simultaneously. However, when you close all of these channels, we will only keep two of these connections around (subject to IdleTimeout of course) for future channel usage. This helps performance in cases where you are creating and disposing client channels. This knob will also apply to the equivalent usage on the server-side as well (that is, when a server-side channel is closed, if we have less than MaxConnections in our server-side pool we will initiate I/O to look for another new client channel).

The reason that we don’t have a hard and fast limit on your connection usage is that you already can control the connection usage through your usage of the WCF objects. That is, if you don’t want to use more than two connections, don’t create more than two client channels 🙂 Any additional knobs at the lower layer would only impede debuggability and predictability.

Note that MaxConnections applies across channels. When sending messages over a single channel, you can only send out one message at a time. That is, your second Send() call will not be initiated until your first Send() completes. In this manner, our TCP binding can guarantee in-order delivery. Also, practically speaking there would be a significant amount of complexity (and overall a negative performance hit) if we allowed interleaving of data from multiple messages, as each “chunk” would need to be annotated with a scatter/gather message marker.

Lastly, all of the above comments apply to TransferMode.Buffered (the default). When using Streaming mode, we “check out” a connection for each in-progress send (and not per-channel). So all the above statements will apply to simultaneous sends rather than simultaneous channels. Streaming TCP is a datagram (not a session-ful) channel, and so simultaneous sends are supported since each send will use a separate TCP connection. This is more similar to HTTP’s usage of TCP connections (where each in-flight request-response pair is using a separate TCP connection).

For those interested in all the low-level intricacies of networking, check out the Windows Core Networking Blog. For the non-bit-heads out there, you will also find a bunch of Vista configuration tidbits useful to any home network user.

Using the TCP and Named Pipe bindings give you a very clean mapping between IDuplexSessionChannel and the underlying network resource (socket or pipe). Namely, you can effectively treat a channel as 1-1 to a socket (I will use socket as shorthand for the generic “network resource” for the remainder of this post :)).

That being said, the lifetime of the underlying socket is not necessarily 1-1 with the lifetime of the channel. Due to our connection pooling feature in WCF, a connection can be reused over the lifetime of multiple channels. We perform connection pooling for both buffered and streaming channels. Our connection pool is configurable through TcpConnectionPoolSettings/NamedPipeConnectionPoolSettings. These settings include a GroupName that we use for isolation, an upper bound on our cache size (MaxOutboundConnectionsPerEndpoint), and timeout values for reliability and NLB support

The way connection pooling works on the client is as follows:

• When you open a channel we will first look for a connection in our pool. This lookup is performed based on IP+port for sockets and based on endpoint Uri name for Pipes.
• If we find an available connection in our pool then we will attempt our open handshake using .Net Framing. If this succeeds then we will associate the connection with the new channel and return from Open. If it fails then we’ll discard the connection. If we have not yet exceeded the binding’s OpenTimeout then we will repeat the “look in pool” process.
• If no [valid] connections are found in our pool then we will establish a new connection (again, using up to the time remaining in OpenTimeout).
• When you close a channel, after we perform our close handshake we will consider returning the connection to our pool. If we already have reached MaxOutboundConnectionsPerEndpoint, or the connection’s lifetime has exceeded LeaseTimeout then we will close the connection instead. The connection that is returned to the pool is the “raw” connection (the one that was initially accepted, prior to any security upgrades). In this way we can provide a transparent pool without leaking any security or other information.

I was going to cover the server-side usage of connection pooling in this same post, but the process of accepting and reusing connections on the server is worthy of its own topic next time 😉

Part of the process involved in writing a custom MessageEncoder is hooking your encoder up to a binding stack. This is accomplished using a subclass of MessageEncodingBindingElement. The way our build process works is that we will take the first BindingElement within a Binding, and call [Can]CreateChannel[Factory|Listener]() on it. By default, these 4 calls will simply delegate to the BindingElement “below” them in the stack.

This default behavior is not exactly what you want in your MessageEncodingBindingElement. While most binding elements add an item to the channel stack, Message Encoders are an “adjunct” binding element. That is, they provide parameters to other binding elements (in this case the transport). The way that parameters are provided to other parts of the build process is through BindingParameters. For example, a MessageEncodingBindingElement can add itself to the BindingParameters collection:

context.BindingParameters.Add(this);

And later the TransportBindingElement can fish out the encoder when it is building the relevant factory/listener:

Collection<MessageEncodingBindingElement>messageEncoderBindingElements
= context.BindingParameters.FindAll<MessageEncodingBindingElement>();

Unfortunately, the base MessageEncodingBindingElement does not take care of adding itself to the BindingParameters collection in its default [Can]Build methods. Therefore, every subclass needs to add the following four boilerplate overrides. Furthermore, it will be a breaking change to fix this in V2, so it’s unclear if we will be able to fix this in the next release 🙁

public override IChannelFactory<TChannel> BuildChannelFactory<TChannel>(BindingContext context)
{
  if (context == null)
  {
    throw new ArgumentNullException("context");
  }

  context.BindingParameters.Add(this);
  return context.BuildInnerChannelFactory<TChannel>();
}

public override IChannelListener<TChannel> BuildChannelListener<TChannel>(BindingContext context)
{
  if (context == null)
  {
    throw new ArgumentNullException("context");
  }

  context.BindingParameters.Add(this);
  return context.BuildInnerChannelListener<TChannel>();
}

public override bool CanBuildChannelFactory<TChannel>(BindingContext context)
{
  if (context == null)
  {
    throw new ArgumentNullException("context");
  }

  context.BindingParameters.Add(this);
  return context.CanBuildInnerChannelFactory<TChannel>();
}

public override bool CanBuildChannelListener<TChannel>(BindingContext context)
{
  if (context == null)
  {
    throw new ArgumentNullException("context");
  }

  context.BindingParameters.Add(this);
  return context.CanBuildInnerChannelListener<TChannel>();
}