I haven’t posted here for awhile as I’ve been preparing for, and then starting, my sabbatical. While I will occasionally cross post here, those interested in following along over the next year should tune into http://lawolf.net/.
I’ve been pretty quiet recently on the technical blog front, mostly because my work was in the dark depths of development. A few weeks ago, we released Beta 1 of Visual Studio 2010 which includes all of the technologies I’ve been working on for the past 3 years 🙂
One of the big components included in .Net 4.0 Beta1 is the WF4 framework that I unveiled at PDC. The team is blogging here, and I’ll be including ongoing tidbits for WF and WCF to help smooth out speed bumps encountered by our customers.
For those of you who want to see what our marketing team has been up to 🙂
More at www.microsoft.com/net/wcf/champ.
Nicholas explains the protocol we use for messaging over TCP (net.tcp) and Windows Named Pipes (net.pipe) in a nice series of blog posts:
- Framing Protocol Overview
- Preamble Records
- Encoding Records
- Upgrade Negotiations (security, compression, etc)
- Message Data
#3-5 are the details behind the "handshake" I alluded to here.
Matt posted a great description of the WCF (and WF) talks we’re giving at PDC.
In particular there are two sessions that I’d like to call out.
The first is Ed Pinto’s session, where you’ll find out about the significant investments we’ve made to improve the WCF authoring experience:
WCF 4.0: Building WCF Services with WF in Microsoft .NET 4.0.
Eliminate the tradeoff between ease of service authoring and performant, scalable services. Hear about significant enhancements in Windows Communication Foundation (WCF) 4.0 and Windows Workflow Foundation (WF) 4.0 to deal with the ever increasing complexity of communication. Learn how to use WCF to correlate messages to service instances using transport, context, and application payloads. See how the new WF messaging activities enable the modeling of rich protocols. Learn how WCF provides a default host for workflows exposing features such as distributed compensation and discovery. See how service definition in XAML completes the union of WF and WCF with a unified authoring experience that simplifies configuration and is fully integrated with IIS activation and deployment.
Once you’ve built your services, you will need to deploy, host, and manage them. Windows Server "Dublin" handles this complexity, and Dan Eshner will unveil the details here:
Hosting Workflows and Services
Hear about extensions being made to Windows Server to provide a feature-rich middle-tier execution and deployment environment for Windows Workflow Foundation (WF) and Windows Communication Foundation (WCF) applications. Learn about the architecture of this new extension, how it works, how to take advantage of it, and the features it provides that simplify deployment, management, and troubleshooting of workflows and services.
I’ve gone dark on the technical side of this blog for two main reasons:
- Most of what I’m working on hasn’t been publicly disclosed
- I’m prepping for PDC 2008 where I can finally discuss the past 2 years of my life without an NDA 🙂
If you want to see the latest and greatest Microsoft technologies that we’ve been cooking up, register now, and then mark your schedule for my talk:
Windows Workflow Foundation 4.0: A First Look
Programs coordinate work. The code for coordination and state management often obscures a program’s purpose. Learn how programming with Windows Workflow Foundation (WF) 4.0 provides clarity of intent while preserving the functional richness of the .NET framework. See how easy it is to build workflows with the new Visual Studio workflow designer. Learn about text-based authoring options for WF. Hear how WF integrates well with other Microsoft technologies (WCF, WPF, ASP.NET). If you’ve looked at WF before, come and see the changes to data flow, composition, and new control flow styles. Significant improvements to usability, composability, and performance make Workflow a great fit for a broad range of solutions on both the client and the server.
41 days and counting…
I’ve recently encountered some confusion around the behavior of one-way operations in WCF that I’m going to try and clear up. In particular, developers are under the impression that a one-way operation == a non-blocking call. However, this is not necessarily the case. A one-way operation means that we will call the underlying channel in a "one-way manner". For IOutputChannel/IDuplexChannel, this maps to channel.Send(). For IRequestChannel this maps to channel.SendRequest() followed by a check for a null response.
Now, sometimes the underlying channel can complete immediately (UDP will drop the packet if the network is saturated, TCP may copy bytes into a kernel buffer if there’s room, etc). However, depending on the amount of data transmitted and the number of simultaneous calls to a proxy, you will often see blocking behavior. HTTP, TCP, and Pipes all have throttling built into their network protocols.
If this isn’t desirable, there are a few alternatives depending on your application design. First off, if you want a truly non-blocking call, you should call channel.BeginSend/client.BeginXXXX (i.e. generate an async proxy). This is step one if you want non-blocking calls. With an asynchronous proxy, we will always be non-blocking from a thread perspective (which is always my recommendation for middle tier solutions, though there’s some quota coordination necessary to avoid flooding outgoing sockets).
For one-way operations, when your async callback is signaled it means that the channel stack has "successfully put the message on the wire". When this happens depends on your channel stack:
- TCP signals when the socket.[Begin]Send() of the serialized message has completed (either because it’s been buffered by the kernel or put onto the NIC)
- Pipes are a similar process (NT Named Pipes work similarly to TCP under the covers but without the packet loss)
- MSMQ signals when the Message has been transferred successfully to the queue manager
- HTTP signals when we’ve received an empty response. The only other alternative would be to remove all guarantees (and have to arbitrarily propagate the exception through some other call or thread). Trust me, this is better
- UDP will complete when the UDP socket send completes (which is effectively instantaneous)
For two-way operations, when your async callback is signaled it means that the channel stack has "successfully put the message on the wire and then received a correlated response".
Asynchronous operations can be tricky, and can often get you into flooding trouble when used incorrectly. So be careful, use quotas to manage your flow control, and always remember that the internet is not a big truck; it is a series of tubes and sometimes they get clogged. And you shouldn’t try to fight the clogs by pouring more data down the tubes 🙂
The next step in our custom transport walk-through is to create our channel managers. This includes an
IChannelFactory for our client side channels, and an
IChannelListener for our service-side channels.
Both IChannelFactory and IChannelListener derive from
IChannelManager. IChannelManager is responsible for tracking channels that are created or accepted. IChannelManager also has a property to control the MessageVersion supported for Messages sent or received on its channels. Lastly, it requires you to specify a Scheme to use for message addressing.
Which leads me to a crucial aspect in defining a channel, namely: what is your addressing model?
For layered channels, the addressing model is simply that of the channels they are layered on top of. Such implementations look like:
public override stringScheme
public override void SetUri(Uri uri)
public override void SetUniqueUri()
For transport channels, it’s an area that takes some thought. While HTTP has a pre-defined syntax and semantic, when writing a new transport you need to define both of these. Concretely, this includes a scheme and its associated URI syntax. This work is closely related to how you bind your protocol to SOAP.
The first item to consider is how your client Channels will resolve the URI. For example, HTTP takes the host name and TCP port of the URI (based on the defined URI grammar), uses DNS to resolve the host name into an IP Address, establishes a TCP connection to the resulting (IP Address, port), and sends an HTTP request using the path portion of the URI in our POST request.
The second factor in your URI design is how to match an incoming URI to an IListenerFactory (which will dispatch to a service). IListenerFactory URIs are broken down into 2 parts: a base address and a relative address. The base address is defined by your hosting environment (ServiceHost<T>, IIS vroot, etc). Individual endpoints then define a simple relative address. Note that an endpoint can be configured with a full URI, but this is only necessary in a few corner cases. This model allows for the host to be decoupled from its endpoints, and also allows the transport to optimize the network resources it uses (i.e. we only require at most one per base address, and can share that resource among relative endpoints).
Given these requirements, we define our URI syntax for UDP as:
soap.udp:// host [":" port] path-absolute
where host, port, and path-absolute are defined in the ABNF of RFC 3986.
Our UdpOutputChannel will take the hostname (localhost) and convert it to an IP Address:
IPHostEntry hostEntry = Dns.GetHostEntry(remoteAddress.Uri.Host);
if (hostEntry.AddressList.Length > 0)
remoteIP = hostEntry.AddressList;
We will then use the resulting IP Address along with the UDP port from the URI (7000), construct a UDP socket to that remote endpoint, and set:
When our UdpListenerFactory (which is on the receiving end of the UDP socket) deserializes the message, it will compare the incoming message.To against its Uri and only accept the message if the two values match. More advanced implementations could factor out the listening socket into a shared app-domain wide resource that dispatches to the appropriate UdpListenerFactory based on the To of the incoming message.
With our transport addressing model crisply defined, we continue on to actually sending and receiving Messages over UDP!
A co-worker just reminded me that I promised to discuss the effects a home network could have on your Indigo Service, and how that could differ based on the transport you are using.
The first thing to realize about a home network is that your computer is likely behind a NAT. Most DSL or Cable routers will act in this role. The router is assigned an IP Address from your service provider via DHCP, and therefore your router is the only member of your network that is globally addressable. Which means that while you can connect to other machines on the internet, they cannot connect to you.
The practical upshot of this situation is that you cannot use the “dual” standard bindings (a.k.a. WsProfileDualHttpBinding and NetProfileDualTcpBinding) to access services located outside of your home network. Which won’t affect you unless you are using a duplex service contract. I can already hear you saying “but duplex contracts are incredibly useful!” Yes they are. And if you want to use them from behind a NAT you have 2 options:
- Use NetProfileTcpBinding. This will allow you to be the client for duplex communication with another Indigo endpoint.
- Host a router service on the Internet. This is a complicated solution that entails bridging the TCP connection from your home machine to a dual HTTP connection with the service. I call out HTTP specifically, because the only scenario where NetProfileTcpBinding will prove insufficient is if your service is a non-Indigo service and needs to be contacted through HTTP. I will try to comment further on this solution in a future post.
The second thing to note is that your network may not be configured with DNS. Which means that even though machines within your network are all addressable, you may need to identify them by their IP Address. To do this, simply use your machine’s IP Address where you would normally use a hostname (for a Service’s URL or a Proxy’s remote endpoint).
And now back to our (somewhat) regularly scheduled Channels/layering tutorial…
A major source of confusion with Indigo Beta 1 is how to shut down our communication objects, so I’m going to drill further into this topic. First, I must note that this area is under review for Beta 2. We understand there are some inconsistencies between our usage of
IDisposable and its usage by other parts of the .Net Framework. But the CTP bits are what you have to work with for now, so I will do my best to explain them 🙂
When I talked about the common state machine in Indigo, I neglected to point out that
ICommunicationObject derives from
IDisposable. This gives you a third option (in addition to our Close() and Abort() methods) for shutdown. It also allows you to use any ICommunicationObject with the C#
In the CTP bits, Dispose == Abort. Or put another way, Dispose != Close. This is a point worth re-iterating. Dispose will rudely abort your object.
So when you are done with an ICommunicationObject, what should you do?
First you should call Close(), which will flush any outstanding buffers, acknowledge outstanding data, and provide a graceful shutdown. This is similar to what happens when you call Stream.Close() or XmlWriter.Close(). Then you should make sure that Abort/Dispose is called if anything fails (i.e. throws an exception). This way any heavyweight network resources are released in an eager fashion.
Which leads to code like the following: