Tag Archives: transports

Channels 103: Addressing the World

The next step in our custom transport walk-through is to create our channel managers. This includes an IChannelFactory for our client side channels, and an IChannelListener for our service-side channels.

Both IChannelFactory and IChannelListener derive from IChannelManager. IChannelManager is responsible for tracking channels that are created or accepted. IChannelManager also has a property to control the MessageVersion supported for Messages sent or received on its channels. Lastly, it requires you to specify a Scheme to use for message addressing.

Which leads me to a crucial aspect in defining a channel, namely: what is your addressing model?

For layered channels, the addressing model is simply that of the channels they are layered on top of. Such implementations look like:

public override string Scheme
{

get { return InnerListenerFactory.Scheme; }

}

public override void SetUri(Uri uri)
{

return InnerListenerFactory.SetUri(uri);

}

public override void SetUniqueUri()
{

return InnerListenerFactory.SetUniqueUri();

}

For transport channels, it’s an area that takes some thought. While HTTP has a pre-defined syntax and semantic, when writing a new transport you need to define both of these. Concretely, this includes a scheme and its associated URI syntax. This work is closely related to how you bind your protocol to SOAP.

The first item to consider is how your client Channels will resolve the URI. For example, HTTP takes the host name and TCP port of the URI (based on the defined URI grammar), uses DNS to resolve the host name into an IP Address, establishes a TCP connection to the resulting (IP Address, port), and sends an HTTP request using the path portion of the URI in our POST request.

The second factor in your URI design is how to match an incoming URI to an IListenerFactory (which will dispatch to a service). IListenerFactory URIs are broken down into 2 parts: a base address and a relative address. The base address is defined by your hosting environment (ServiceHost<T>, IIS vroot, etc). Individual endpoints then define a simple relative address. Note that an endpoint can be configured with a full URI, but this is only necessary in a few corner cases. This model allows for the host to be decoupled from its endpoints, and also allows the transport to optimize the network resources it uses (i.e. we only require at most one per base address, and can share that resource among relative endpoints).

Given these requirements, we define our URI syntax for UDP as:

soap.udp:// host [":" port] path-absolute

where host, port, and path-absolute are defined in the ABNF of RFC 3986.

For example: soap.udp://localhost:7000/service/endpoint

Our UdpOutputChannel will take the hostname (localhost) and convert it to an IP Address:

switch (remoteAddress.Uri.HostNameType)
{

// …
case UriHostNameType.IPv4:
case UriHostNameType.IPv6:

remoteIP = IPAddress.Parse(remoteAddress.Uri.Host);
break;

case UriHostNameType.Basic:
case UriHostNameType.Dns:

{
IPHostEntry hostEntry = Dns.GetHostEntry(remoteAddress.Uri.Host);
if (hostEntry.AddressList.Length > 0)
{
remoteIP = hostEntry.AddressList[0];
}
// …
}
break;

}

We will then use the resulting IP Address along with the UDP port from the URI (7000), construct a UDP socket to that remote endpoint, and set:

message.To = RemoteUri (soap.udp://localhost:7000/service/endpoint).

When our UdpListenerFactory (which is on the receiving end of the UDP socket) deserializes the message, it will compare the incoming message.To against its Uri and only accept the message if the two values match. More advanced implementations could factor out the listening socket into a shared app-domain wide resource that dispatches to the appropriate UdpListenerFactory based on the To of the incoming message.

With our transport addressing model crisply defined, we continue on to actually sending and receiving Messages over UDP!

Interlude: Working from home

HomeA co-worker just reminded me that I promised to discuss the effects a home network could have on your Indigo Service, and how that could differ based on the transport you are using.

The first thing to realize about a home network is that your computer is likely behind a NAT. Most DSL or Cable routers will act in this role. The router is assigned an IP Address from your service provider via DHCP, and therefore your router is the only member of your network that is globally addressable. Which means that while you can connect to other machines on the internet, they cannot connect to you.

The practical upshot of this situation is that you cannot use the “dual” standard bindings (a.k.a. WsProfileDualHttpBinding and NetProfileDualTcpBinding) to access services located outside of your home network. Which won’t affect you unless you are using a duplex service contract. I can already hear you saying “but duplex contracts are incredibly useful!” Yes they are. And if you want to use them from behind a NAT you have 2 options:

  1. Use NetProfileTcpBinding. This will allow you to be the client for duplex communication with another Indigo endpoint.
  2. Host a router service on the Internet. This is a complicated solution that entails bridging the TCP connection from your home machine to a dual HTTP connection with the service. I call out HTTP specifically, because the only scenario where NetProfileTcpBinding will prove insufficient is if your service is a non-Indigo service and needs to be contacted through HTTP. I will try to comment further on this solution in a future post.

The second thing to note is that your network may not be configured with DNS. Which means that even though machines within your network are all addressable, you may need to identify them by their IP Address. To do this, simply use your machine’s IP Address where you would normally use a hostname (for a Service’s URL or a Proxy’s remote endpoint).

And now back to our (somewhat) regularly scheduled Channels/layering tutorial…

Channels 102b: Close vs. Abort (vs. Dispose)

A major source of confusion with Indigo Beta 1 is how to shut down our communication objects, so I’m going to drill further into this topic. First, I must note that this area is under review for Beta 2. We understand there are some inconsistencies between our usage of IDisposable and its usage by other parts of the .Net Framework. But the CTP bits are what you have to work with for now, so I will do my best to explain them 🙂

When I talked about the common state machine in Indigo, I neglected to point out that ICommunicationObject derives from IDisposable. This gives you a third option (in addition to our Close() and Abort() methods) for shutdown. It also allows you to use any ICommunicationObject with the C# using statement.

In the CTP bits, Dispose == Abort. Or put another way, Dispose != Close. This is a point worth re-iterating. Dispose will rudely abort your object.

So when you are done with an ICommunicationObject, what should you do?

First you should call Close(), which will flush any outstanding buffers, acknowledge outstanding data, and provide a graceful shutdown. This is similar to what happens when you call Stream.Close() or XmlWriter.Close(). Then you should make sure that Abort/Dispose is called if anything fails (i.e. throws an exception). This way any heavyweight network resources are released in an eager fashion.

Which leads to code like the following:

IOutputChannel channel = CreateOutputChannel();

using (channel)
{

channel.Open();
channel.Send(message);
channel.Close();

}

Channels 102: Communication object lifecycle

Before we dive into implementing our Channels, it’s important to first understand the lifecycle of communication objects in Indigo. These include our Channel Layer objects (IChannel, IChannelFactory, and IListenerFactory), as well as Typed Layer objects (ServiceHost, ServiceSite, EndpointListener).

All such objects derive from ICommunicationObject, and have a common state machine.

There are 5 states represented by the CommunicationState enum:

  • Created — This is the state of an ICommunicationObject when it is first instantiated. The object may be configured in this state (e.g. properties can be modified, events registered, etc). No I/O occurs in this state.
  • Opening — Objects transition to this state when ICommunicationObject.Open() is called. At this point properties are made immutable, and I/O may begin. This transition is only valid from the Created state.
  • Opened — Objects transition to this state when the open process completes. This transition is only valid from the Opening state. At this point the object is fully usable for transfer.
  • Closing — Objects transition to this state when ICommunicationObject.Close() is called for a graceful shutdown. This transition is only valid from the Opened state.
  • Closed — In the Closed state objects are no longer usable. In general, most configuration is still accessible for inspection purposes, but no communication can occur. This state is equivalent to being disposed.

There are events that fire for each state transition. ICommunicationObject also has an Abort() method that can be called at any time. This will cause the object to transition immediately from its current state into the Closed state. Abort() indicates that any unfinished work will be rudely terminated (Exceptions are likely in this case).

With this background knowledge in place, we can now go into detail about writing our factories.

Channels 101: Get with the MEP

Last week I gave an overview of Indigo layering. In the next few posts I’ll guide you through these layers from the bottom up. We’ll start by writing a custom transport. Then we’ll add layered channels on top of this transport, and progressively work our way up the stack to a typed Service and Proxy implementation.

A fundamental concept at the Channel Layer is that of a Message Exchange Pattern (often referred to as “MEP”). There are three basic MEPs supported at the Channel Layer:

  1. Datagram: In this MEP, messages are “fire and forget”. It’s like when I send a postcard to my Mom. Just because I put the postcard in a mailbox does not mean that she received it. It may have gone to the incorrect address, or thrown out in the post office if she was on a long vacation, or eaten by a dog. I need to get any confirmation of successful delivery out of band (with a followup telephone call for example). Datagram is a fundamental building block for Messaging, as you can build arbitrary protocols on top of it (including reliable protocols, secure protocols, etc).

Datagram Message Exchange

  1. Request-Response: This is the most common MEP on the web today. It consists of a single message being sent, followed by a single response to that message. RPC calls are request-response, as are browser GETs. If I had paid for Delivery Confirmation on the postcard I sent my Mom, then I would have been using this MEP (with the confirmation as my response). This pattern is also known as half-duplex.

Request-Response Message Exchange

  1. Duplex: Duplex is equivalent to bi-directional datagram communication. Duplex communication is what we are used to in daily conversation. When I’m talking to a co-worker, we are free to speak at any time. We can comment at any time, and are engaging in an unrestricted ordering of “messages” between us. Note that delivery is still unreliable, and data can still be lost (if one of us is daydreaming or otherwise distracted).

Duplex Message Exchange

Each of these MEPs also have a session-ful variant. A session is an object that provides correlation for the MEP. Session is very useful for classifying data. It turns out that we use the concept of session constantly in our daily lives. Take as an example the conversations I have with my boss. As per our above definitions, when we converse we engage in Duplex communication. During a daily triage meeting we’ll talk about current issues for Indigo, and over lunch we’ll discuss the 520 commute. Each of these topics can be thought of as a session that subdivides our communication. While in the real world sessions often bleed into one another, in Indigo we can keep your session state localized to a given Channel. This gives you a total of 6 MEPs (Datagram, Request-Response, Duplex, Datagram+Session, Request-Response+Session, or Duplex+Session).

Some transports support multiple MEPs. For example, SOAP-over-TCP supports Datagram, Datagram+Session, and Duplex+Session. SOAP-over HTTP supports Datagram and Request-Response.

For our example, we are going to build a transport based on the User Datagram Protocol, so the MEP we will support is Datagram. The next step is to create an IChannelFactory and IListenerFactory to implement our Channels.

Baking a Seven Layer Service Cake

Indigo is constructed with layering as a central design principle. This means that in essence Indigo is actually a collection of sub-frameworks, each of which can be supplemented by ISV code (or replaced completely for that matter). Each framework is defined by two things:

  1. Fundamental unit(s) of data
  2. Transformations that can be applied to said data

Which (as you might be thinking) is another way of saying an object is defined by its members and its methods. This is true, but it’s more difficult (but very important) to stay focused on consistent pivots when constructing clean frameworks.

MessagesIndigo is built around the concept of a Message object. The structure of a Message object loosely represents a SOAP envelope, and consists of two distinct parts: the message’s body and an extensible collection of headers. The body is application-defined data, and headers are added/processed by infrastructure (and can also be used by the application). Access to a header or the body is achieved through XmlReader, as it contains structured data. It is this structured primitive which is the core of Indigo’s power (and which departs from the classic networking primitive of a byte array). It allows for extensible, interoperable protocols to be built, and for data contracts to be established.

At a high level, Indigo consists of two larger frameworks: the Typed Layer, and the Channel Layer.
Indigo Architecture

The fundamental unit of data for the Channel Layer is Message. Channels are used to Send and Receive Messages. Channels come in two forms: Transport Channels and Layered Channels.

Transport Channels perform the actual send/receive of the Message to a network resource, including any necessary serialization.

Layered Channels perform a function based on the Message passed in, and then delegate further modification and transmission to their Inner Channel. So as you can see, even within a layer we have layers 🙂 Some examples of Layered Channels include Protocol Channels that use Message headers and infrastructure Messages to establish a higher-level protocol (such as WS-ReliableMessaging), and Reshaping Channels for changing the Message Exchange Pattern (such as converting an underlying [Send Channel, Receive Channel] pair into a Duplex Channel).

Message is a very flexible object, but to fully utilize it requires knowledge of Infosets and XML. What most developers instead will likely interface with is the Typed Layer. The fundamental unit of data for the Typed Layer is a CLR class. In the Typed Layer you create CLR objects that implement Services and Typed Proxies. The Typed Layer converts parameters and return values into Message objects and method calls into Channel calls. In this way, the Typed Layer builds on the functionality of the Channel Layer, and can transparently leverage any changes/improvements made to Channels.

Next I’ll provide an example to give you a flavor of this layering in action, stay tuned!

Bindings vs. Transports, live on ESPN

If you’re not familiar with the “ABCs of Indigo” (Address, Binding, and Contract), please click here for background on some basic Indigo terminology and concepts.

When users first start playing with Indigo, there are two common approaches to the system: those that want to trace a message exchange from the “top” (by building a Service), and those that want to start at the “bottom” (by tracing the Message as it enters the system and makes its way up to the typed ServiceMethod). Those in the latter camp generally start by hunting for transports, which put your messages on the proverbial wire.

A few notes about transports in Indigo:

  • Transports need to be considered in the greater context of a binding. A binding is simply an ordered list of binding elements. There are binding elements for reliability, security, transaction flow, and transports. When I talk about a transport (such as HTTP), you will likely be interfacing with it through its binding element (e.g. HttpTransportBindingElement).
  • Indigo includes 4 transports in the Community Tech Preview:
    1. HTTP (for cross machine interoperable messaging)
    2. TCP (for cross machine Indigo to Indigo messaging)
    3. Named Pipes (for on machine Indigo to Indigo messaging)
    4. MSMQ (for queued messaging)
  • Transports are responsible for encoding+transmitting messages (on Send/Request), and receiving+decoding messages (on Receive/ReceiveReply). Indigo is architected so that transports can delegate the task of translating between a Message and a byte array to an encoder.
  • Indigo includes 3 encoders: Text, Binary, and MTOM. Each encoder is associated with an implementation of XmlReader and XmlWriter, as well as a SOAP version. By default, HTTP uses the text encoder, and TCP/Named Pipes use the binary encoder. However, an Indigo transport can be used with any encoder (either built-in or custom).
  • The combinatoric possibilities involved in constructing a binding can be staggering, so Indigo includes a small number of predefined bindings that you can use for most common scenarios. The documentation gives a good high-level overview of what scenario each binding is intended for and the various tradeoffs inherent in using each one. Each predefined binding has a transport binding element associated with it, though it may only expose a subset of the properties available directly on the transport binding element itself.

While Indigo at its core is “transport agnostic” there are of course implications to choosing each transport. Up next, I’ll discuss what some of these implications are in a home networking scenario.