The Dot Net Hub

Creating a SOAP/REST – based WCF Service with Custom Binding/Behaviors

2012-01-13T22:44:00.000+03:00

Introduction

Ok, basically the title says it all. This post shows a WCF service with a method exposed both as a SOAP-based and REST-based endpoint. For the SOAP-based endpoint I found that there is no binding that satisfy my needs (yep, I am picky!) so I will create a custom binding – explaining what are custom bindings and what is the Channel layer in general.

Also In order to enable logging operations, I will create two custom behaviors – again explaining what are custom behaviors and what is the ServiceModel layer in general.

The code can be downloaded from here: http://www.devlifestyle.net/media/p/1177/download.aspx

So let’s start with the basics. I have an IIS 7 (you can also go with IIS 6 – no Net-based bindings here) hosted WCF service which exposes a single operation. My simple target is to expose this operation on SOAP-based and REST-based endpoints at the same time.

Lets add a twist: for the SOAP-based endpoint I want to use the basicHttpBinding but I want to have the calls authenticated via a username/password however I do not want any on-the-wire security (i.e. encryption) since all calls are inside my intranet and I do not want the overhead of SSL – so in short I want the same basic behavior of good old ASMX services, just using WCF! Now here is the bump: there is no way to do so out of the box in WCF.

Enter Custom Bindings and the Channel layer.

WCF Architecture…the Channel Layer

WCF is made up of two architectural layers: the Service Model and the Channel layers. Service Model is the higher-level abstraction that you deal with most of the time. Most of developers know it by the famous name of “proxy” at the client side (and less commonly by the name of “dispatcher” at the service side). In its most simple form, when acting as the proxy the Service Model converts our “objects” (data contracts) into the proper message format that the Channel layer understands and hands it down to the Channel layer (known commonly as “serialization”); while when acting as the dispatcher the Service Model gets the message from the low level Channel layer and puts it back into the “objectized” format that our code understands (known commonly as “deserialization”).

While we focus on extending the Service Model later when we create the inspectors, let’s focus now on the low-level Channel layer and creating the custom binding.

So, now we know that the Channel layer is the low-level devil that we all hate to get near to. But anytime you deal with WCF bindings, all what you are actually doing is controlling the creation of WCF channel stack! You are just doing it in the higher-level abstraction courtesy of predefined WCF bindings. But what happens when the out of the box bindings are not enough? Simple: you write down your own binding. In order to do so, you must understand some essentials about WCF bindings and the channel stack.

Bindings (simply) produce a set of ordered BindingElement objects at runtime depending on the type of binding you chose. These binding elements form the entire channel stack (thus the Channel layer). Memorizing all the available BindingElements is difficult unless writing custom bindings (and at times custom BindingElements) is your daily work. Luckily this is not my case! Unluckily though, everytime I have to deal with the channel stack I have to pay a visit for MSDN (http://msdn.microsoft.com/en-us/library/aa347793.aspx) to refresh my memory about the BindingElements and what each does.

So once you get yourself familiar with the type of BindingElements, all what is left for you is to create a custom binding that utilizes these elements in a predefined order…and that’s all what you need to know about the (scary) channel stack.

So what is a BindingElement exactly? Simply it’s a processing step. So simply a binding stacks a set of BindingElements on top of each other where a WCF message passes through each step of this stack and “something” happens to the message. For example, a message security binding element provides message encryption (and other stuff), encoding binding element sets message encoding (text, binary, etc…), a transport encoding sets the protocol (TCP, HTTP, HTTPs, etc…)…you got the point.

Custom Binding

So, back to our custom binding implementing. Recall what are my requirements? I just want to have username/password authentication over HTTP and without any SSL. Here is the part of the code that adds up the proper BindingElements:

public override BindingElementCollection CreateBindingElements()

{

var res = new BindingElementCollection();

XmlDictionaryReaderQuotas rqMax = XmlDictionaryReaderQuotas.Max;

TextMessageEncodingBindingElement textBE = new TextMessageEncodingBindingElement() { MessageVersion = MessageVersion.Soap12WSAddressing10, MaxReadPoolSize = int.MaxValue, MaxWritePoolSize = int.MaxValue };

rqMax.CopyTo(textBE.ReaderQuotas);

res.Add(textBE);

TransportSecurityBindingElement transportSec = SecurityBindingElement.CreateUserNameOverTransportBindingElement();

transportSec.LocalServiceSettings.MaxClockSkew = new TimeSpan(10, 0, 0); //allow clock difference between client and service

transportSec.LocalClientSettings.MaxClockSkew = new TimeSpan(10, 0, 0);

res.Add(transportSec);

res.Add(new AutoSecuredHttpTransportElement() { MaxReceivedMessageSize = int.MaxValue, MaxBufferPoolSize = int.MaxValue, MaxBufferSize = int.MaxValue });

return res;

}

Once the custom binding is done, I want to expose my SOAP-based endpoint using this new binding instead of the basicHttpBinding. That’s the easy part. Let’s see the service configuration:

First we create a binding extenstion element and supply and custom binding dll:

bindingExtensions>

extensions>

Then we create the binding element itself:

binding>

customHttpBinding>

bindings>

Finally, we define our SOAP endpoint using the new binding, as follows:

service>

services>

Needless to say that the binding must also be used at the client side…since now you know what the configuration elements mean you can check the client configuration yourself.

Custom Username/Password Validator

Before wrapping up, while we are in the “customization” mode, lets do one final thing: now that we have a custom binding that supports username/password authentication, lets a create a custom username/password validator and plug that into our service to isolate that authentication logic from the service logic. The code is rather simple:

Public Class UsernameValidator

Inherits UserNamePasswordValidator

Public Overrides Sub Validate(ByVal userName As String, ByVal password As String)

Try

If userName <> "mohamad" Or password <> "halabi" Then

Throw New Exception("Unknown Username or Password")

End If

Catch ex As IndexOutOfRangeException

Throw New FaultException("Unknown Username or Password")

End Try

End Sub

End Class

Needless to say, I hardcoded my username/password…you will authenticate against your favorite user store; be it SQL, AD, etc…

Again, lets see how to add the custom username/password validator to the WCF config file:

<userNameAuthentication userNamePasswordValidationMode="Custom" customUserNamePasswordValidatorType="CustomUsernameValidator.UsernameValidator,

CustomUsernameValidator"/>

serviceCredentials>

behavior>

serviceBehaviors>

The “userNameAuthentication” tag is supplied with the dll holding my custom validation logic.

Just in case you’re wondering about the other configurations, here’s a quick look:

--dataContractSerializer: if you want to parse large messages in WCF (http://thedotnethub.blogspot.com/2009/10/passing-large-messages-in-wcf.html) there are some attributes you want to set. One of those is maxItemsInObjectGraph which sets how many objects can be serialized and passed around.

--serviceMetadata: enables metadata publishing which grants access to the wsdl

--serviceDebug: enables exception details to be propagated to callers; handy in development

REST-enabling the service

So, let’s think about this: we have a WCF-SOAP service that is using an HTTP based binding. What SOAP-features are we exactly using here? Besides SOAP envelopes…nothing! We aren’t using message security, we aren’t using addressing, we aren’t using routing, we aren’t using transactions…in short we aren’t using any of the WSE standards. Frankly, SOAP is an overhead here! Why? try passing an array of – say 500 – objects from a SOAP service to a client. Now pass the same exact objects in REST style. You will find that the size of the REST request is at least 50% less than the size of the SOAP request. Now that’s some saving.

Moral of the story: unless you need SOAP (i.e. any of the WSE standards), REST is the way to go. Its reliance on HTTP verbs makes it perfect for interoperability (SOAP also supports interoperability), it supports caching, and makes it a perfect fit for client-side invoking via JavaScript, jQuery, or Ajax. Even if you need encryption, you can still enable SSL over REST-based services (but you cannot implement the fine-grained message level security that SOAP supports).

So, in our example I will enable the same operation to be invoked REST-style returning the result in JSON format. Why JSON? It’s the easiest to be parsed with client-side calls…although not shown here. Why JSON #2? Although it makes you sound cool to say “POX”, I just like Javascript more.

So, lets see our operation:

[OperationContract]

[WebInvoke(Method = "POST", UriTemplate = "/PostPerson", ResponseFormat = WebMessageFormat.Json)]

List<Person> PostPerson(Person person);

The OperationContract is for exposing the operation SOAP-style…what we seen before. The WebInvoke attribute is what enables REST magic. I will be using the “POST” verb; although I can also use the “PUT” verb but I will follow the analogy that POSTs are for inserts while PUTs are for updates. You also have DELETE and GET.

Simply, in the web.config we publish the same contract via a second binding, called webHttpBinding:

Now there’s one important point to highlight here: in this example, I am using the webHttpBinding with the REST starter kit dlls (http://msdn.microsoft.com/en-us/netframework/cc950529) to REST-enable WCF. The updated way to do this is using WCF Web API dlls (http://wcf.codeplex.com/wikipage?title=WCF%20HTTP). While I am lazy to update the code, the idea – the concept – behind REST and the method of REST-enabling WCF remains largely the same.

So how do we actually call the REST-enabled operation from the client code? Recall there’s no proxy here, just plain simple HTTP requests. So here is a simple helper method that works with generics to be called from multiple places with different in/out objects.

public static Response Post(string username, string password, RESTService service, string uri, Content content, System.Xml.UniqueId messageId)

{

Response response;

using (HttpClient client = new HttpClient(GetRESTURL(service)))

{

if (string.IsNullOrEmpty(username)) throw new Exception("You must pass username");

if (string.IsNullOrEmpty(password)) throw new Exception("You must pass password");

if (messageId == null) throw new Exception("You must pass messageid");

client.TransportSettings.ConnectionTimeout = new TimeSpan(0, 3, 0);

client.DefaultHeaders.Add("username", username);

client.DefaultHeaders.Add("password", password);

client.DefaultHeaders.Add("clientmachinename", Environment.MachineName);

client.DefaultHeaders.Add("messageid", messageId.ToString());

HttpContent httpContent = HttpContentExtensions.CreateJsonDataContract(content);

using (HttpResponseMessage httpResponse = client.Post(uri, httpContent))

{

if (httpResponse.StatusCode == HttpStatusCode.BadRequest)

{

DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(FaultContract));

FaultContract fault = (FaultContract)ser.ReadObject(httpResponse.Content.ReadAsStream());

throw new WebFaultException(fault, HttpStatusCode.BadRequest);

}

else if (httpResponse.StatusCode == HttpStatusCode.ServiceUnavailable)

throw new Exception("ServiceUnavailable");

else

{

DataContractJsonSerializer ser = new DataContractJsonSerializer(typeof(Response));

response = (Response)ser.ReadObject(httpResponse.Content.ReadAsStream());

return response;

}

WCF Architecture…the Service Layer

Enough easy stuff! Back to the architecture. So recall, that one of the targets of the example is to have logging feature to log every message in and out of the service. Sure we can do this inside the service itself. But this will be neither a good architecture nor cool enough for a post.

So, how do we create a module that catchs every single message into and out from our service, yet we want to do it as a reusable extension without touching our service?

Enter Custom Behaviors and the Service Layer.

Recall that the Service Layer is called “proxy” at the client side and “dispatcher” at the service side. Its job is simply to serialize and deserialize messages to and from the Channel Later respectively. Both the proxy and the dispatcher give you locations – called extensibility locations – that allow you to inject your own logic to inspect messages at various stages and possibly alter the default execution behavior. You can create this logic to inspect messages, operations, parameters, and you can even change serialization & deserialization behavior.

In our example, I am interested in a “message inspector” at the dispatcher side which simply intercepts messages coming in and returning back (requests & responses), and log them into a database. There is one catch here: I want to use the same inspector to log both SOAP and REST messages. SOAP messages are XML-based while REST messages are JSON based so there are some considerations to be taken.

At the proxy side, I also want to create a “client message inspector” to add some metadata to my SOAP message before it is handed on to the channel stack. (note: for variety of configurations, I will enable the client inspector to the SOAP endpoint only).

Message Inspector

Once a message is received, we need to differentiate if it is a SOAP message or a JSON message. Why? because the way to transform the stream into a string format for logging differs:

object IDispatchMessageInspector.AfterReceiveRequest(ref System.ServiceModel.Channels.Message request,

System.ServiceModel.IClientChannel channel,

System.ServiceModel.InstanceContext instanceContext)

{

HttpRequestMessageProperty httpReq = (HttpRequestMessageProperty)

request.Properties[HttpRequestMessageProperty.Name];

if (!httpReq.Headers["Content-Type"].Contains("soap"))

return RESTAfterReceiveRequest(ref request, channel, instanceContext);

else

return SOAPAfterReceiveRequest(ref request, channel, instanceContext);

}

Similarly, before sending the response back we again want to log it:

void IDispatchMessageInspector.BeforeSendReply(ref System.ServiceModel.Channels.Message reply, object correlationState)

{

if (reply.Headers.Action == null)

RESTBeforeSendReply(ref reply, correlationState);

else

SOAPBeforeSendReply(ref reply, correlationState);

}

There are a lot of details in this inspector, such as reading HTTP headers, implementing authentication for REST calls based on these headers, using Task Parallel Libray (TPL) to log data without delaying moving the message down to the channel stack and up to the dispatch operations, and compressing the message before logging it…these are all enhancements however and should lay themselves clear when you download the code.

Now lets see the configuration to add the message inspector:

behaviorExtensions>

extensions>

behavior>

behavior>

endpointBehaviors>

behaviors>

service>

services>

The inspector is added as a behavior extension…recall this means plugging it in the Service Layer. We then define two endpoint behaviors to add the message inspector to both SOAP and REST endpoints. And finally we update the endpoint definitions so that both endpoints now use the message inspector.

Client Message Inspector

Finally, I also said that I want to add a client inspector at the proxy side to add some metadata to the message. Basically, what I want is simply to log (I know, I have a thing for logging) the name of the machine from where the request is initiated:

object IClientMessageInspector.BeforeSendRequest(ref Message request, IClientChannel channel)

{

// Prepare the request message copy to be modified

MessageBuffer buffer = request.CreateBufferedCopy(int.MaxValue);

request = buffer.CreateMessage();

request.Headers.Add(MessageHeader.CreateHeader("MachineName", "http://machinename/CustomHeaders", System.Environment.MachineName));

return null;

}

And we then configure our client application to use this inspector for every SOAP message going out:

behaviorExtensions>

extensions>

behavior>

endpointBehaviors>

behaviors>

<endpoint address="http://localhost/WCFService/Service.svc/soap"

behaviorConfiguration="EndpointBehavior" binding="customHttpBinding"

bindingConfiguration="myCustomHttpBinding" contract="SOAPRef.IService"

name="CustomHttpBinding_IService" />

client>

Azure AppFabric, Claims-based access, WIF Recording

2011-07-05T15:57:00.000+03:00

Here is my recordings talking about Windows Azure AppFabric, Claims-based access, and Windows Identity Foundation(WIF)...this recording is a part of Devlifestyle June 2011 session held in PSU.
Note that this recording is in Arabic only.

Windows Azure Demo-Based Online Recording

2011-06-20T23:35:00.000+03:00

Here is an online recording for my Devlifestyle Online June session about Windows Azure...I present the architecture behind Windows Azure and then dive into demos showing the various services from the Azure Platform

Parallel Task with ASP.NET HttpContext

2011-06-13T15:44:00.000+03:00

When using .NET 4.0 Parallel Programming (Check my article) with ASP.NET, you will lose HttpContext once you jump in to another task.
In my case I wanted to get hold of the context from which I wanted to get the current page handler. So I came up with the below code.

HOWEVER, and as you can see this is BIG HOWEVER, although I tried the code and it did work I am not 100% sure about it because the idea of manually setting the context (as you will see in a moment) and messing up with the way threading works gives me the creeps! But anyway I thought I would share it:

Action<object> action = new Action<object>(dostuff);
Task.Factory.StartNew(action, obj);

where:

private void dostuff(object obj)
{
IHttpHandler han = ((HttpContext)obj).CurrentHandler;
//do stuff
}

or you can use delegates:

Task.Factory.StartNew(new Action<object>(
delegate(object objContext)
{
IHttpHandler han = ((HttpContext)objContext).CurrentHandler;
//do stuff
}
)
, obj);

or event better: lambda:

Task.Factory.StartNew(new Action<object>(
objContext =>
{
IHttpHandler han = ((HttpContext)objContext).CurrentHandler;
//do stuff
}
)
, obj);

Ok so it was a short post, and I thought I would spice it up with some code variations :)

Azure Storage Emulator: Process cannot access the file because it is being used by another process

2011-06-03T18:20:00.000+03:00

If you are trying to run the services of the azure storage emulator (Blob, Queue, Table) and you ever get this error:
The process cannot access the file because it is being used by another process

Then continue reading...
As you might imagine it happened to me, and I found that the blob service is the one complaining:

I knew something was wrong with the endpoint, from the error message i could tell that most probably the problem was that the endpoint address being used by some other process.
So from the command prompt i ran Network Statistics tool (Netstat) and got the following:

As you can see, i had ports 10001 and 10002 saying "cannot obtain ownership information" but anyway I had no problems with these ports as they are associated with the Queue and Table services respectively and both are started. However, it is clear that port 10000 is occupied by BitTorrent on my machine (yes you caught me!) so that is why the Blob service could not be started.
Once I stopped BitTorrent I could start the Blob service.

Note: you would have another option which would be changing the ports the emulator use for the storage services. You could do that by navigating to:
C:\Program Files\Windows Azure SDK\v1.4\bin\devstore
(replace 1.4 with your SDK version) and openning DSService.exe.config. From there you could change the configuration and make your services listen to another ports.

Utilizing WCF Power for Integrations - Devlifestyle May 2011 Session Demo

2011-05-05T08:23:00.000+03:00

So, after a (very?) long time without a post (absolutely hectic period), I am back with a WCF post for WCF lovers out there...here are the session materials for a session I did for Devlifestyle (actually the session is due in 5 hours as of this writing)
http://www.devlifestyle.net/media/p/1173/download.aspx

The demo shows various WCF configurations using basic, net, and MSMQ bindings. The demo also tackles WCF custom bindings and shows a custom http-based binding created as well as custom extensions (behaviors)....ENJOY!

Second Riyadh Community Summit

2010-12-21T15:48:00.000+03:00

I will speaking at the second Riyadh Community Summit, Thursday 23 December. Session will be about RESTful services tackling the popular SOAP vs. REST debate.
Hopefuly I will be able to post the video recordings of the event.
Full agenda here:

http://www.devlifestyle.net/rcs2/

SharePoint 2010: Passing data from Workflow to Task Form

2010-12-16T12:36:00.008+03:00

Implementing a state machine workflow project on a SharePoint 2010 Document Library (i will do a full post soon), I faced a situation where I wanted to pass data from the workflow instance to the ASP.NET task edit form.

Before I have many times done the reverse: passing data from the Task Edit forms to the workflow instance by defining custom content types and referencing their fields inside my workflow using the ExtendedProperties Hashtable.

So now I thought I could do the same to pass data into my task form: from inside the wf instance I used the ExtendedProperties hashtable to set a value to my field when creating the task as follows:
task.ExtendedProperties["myItemGUID"] = ".....";
Now in the task form i tried to access the same property using:
mytaskListItem.Properties["myItemGUID"] but it is always null.

I then "watched" over the Xml property of mytaskListItem and found that "myItemGUID" was actually populated inside as follows:
ows_ExtendedProperties = 'ows_myItemGUID=......'
Further if I use mytaskListItem.GetFormattedValue("ows_ExtendedProperties") i can get the same value "ows_myItemGUID=..." where parsing will be easier than having to parse the full Xml value.

Now of course this will make me move on for now, but this is hardly an optimum solution. Till now I do not know why I could not "strongly-type get" the value of the field the same way I can do when getting data from the wf instance...Needless to say, SharePoint is not exactly my thing (so far!)...workflows are :) so I could be missing some SharePoint related stuff...anyway will get back to this once I finish the business process...

WCF MSMQ Transactions Explained – Part1

2010-11-14T15:06:00.010+03:00

Introduction

Before starting, I advise you to read my detailed article about .NET transactions and WCF implementation for more in depth knowledge about how transactions work. However, you can still go through this post directly to get the required knowledge about WCF MSMQ transactions.
When working with MSMQ and WCF, you have two options for bindings: NetMsmqBinding and MsmqIntegrationBinding.

The NetMsmqBinding is intended to be used when both the client and service are WCF and communicate over the assigned endpoints.
MsmqIntegrationBinding on the other hand can be used with already-written native MSMQ applications that use MSMQ COM or the System.Messaging APIs.

I will be using the NetMsmqBinding.

Getting to the point…

I am going to show you the code directly explaining the concepts as we go.

The service contract of my service application is defined below:

[OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
public void GetData(int value)
{
System.Diagnostics.EventLog.WriteEntry("WCF", value.ToString());
Transaction.Current.Rollback();
}
TransactionScopeRequired indicates that my operation must be enlisted within a transaction and TransactionAutoComplete indicates that the transaction will be committed automatically once the operation finishes.

In the operation body I simply print something to the event viewer and interestingly manually roll back the transaction. Why did I do that? This will become clear later.

The relevant service configuration is where the magic happens. It is shown below:

<services>
<service name="Service" behaviorConfiguration="ServiceBehavior">
<endpoint address="net.msmq://localhost/private/MSMQService/Service.svc" binding="netMsmqBinding" contract="IService" bindingConfiguration="MSMQ"></endpoint>
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="ServiceBehavior">
<serviceMetadata httpGetEnabled="true" />
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netMsmqBinding>
<binding name="MSMQ" maxRetryCycles="1" receiveRetryCount="1" retryCycleDelay="00:00:20" deadLetterQueue="System" receiveErrorHandling="Move" exactlyOnce="true" durable="true" timeToLive="1.00:00:00">
<security mode="None"></security>
</binding>
</netMsmqBinding>
</bindings>
First things first; I am using the netMsmqBinding and have set the address to a local TRANSACTIONAL private queue with the same name as my service application (check this post on how to set up WCF-MSMQ application on IIS 7). In this example, both the client and the service are hosted on the same machine so only the private queue will be used.

Now on to the binding configuration:

exactlyOnce: when set to true means that transaction is enabled and the queue must be marked as “Transactional” on creation. This means that each message is guaranteed to be delivered one and only one time and a batch of messages are delivered in order.
durable: indicates that messages are never lost. Messages are persisted to the disk so if the MSMQ service is restarted messages are preserved.
receiveRetryCount: this setting indicates how many times the service will try to receive and process the message
retryCycleDelay: specifies the time between retry counts. For example if receiveRetryCount is set to 2 and retryCycleDelay is set to 20 seconds, then the service will try to receive and process the message total of 2 times within an interval of 20 seconds between each try.
maxRetryCycles: the number of cycles the retryCycleDelay and receiveRetryCount are run. For example if maxRetryCycles is set to 2, receiveRetryCount is set to 2, and retryCycleDelay is set to 20 seconds then the service will try to receive and process the message 2 times within and interval of 20 seconds and the whole cycle will be repeated 2 times.
timeToLive: Indicates the maximum amount of time a message can live in the queue waiting to be delivered to the service. This value supersedes all retry logic; if the TTL is less that the total time the cycle is configured to process then the cycle terminates once the TTL is reached . The default value is 1 day.

So what happens if the cycle fails? One of two things depending on the reason of failure:

If the failure is caused by the fact that the message cannot be delivered to the service, then the message will be placed in a transactional dead-letter queue. Setting deadLetterQueue specifies that the system dead letter queue will be used. A better idea would be to use a custom dead letter queue in order not to mix messages from different services in the same system dead letter queue. This is done by selecting “Custom” value for the deadLetterQueue setting and then specifying the name of this custom queue using setting customDeadLetterQueue.
If the message is delivered to the service but the transaction fails while processing, then the message will be considered “poisoned” and setting receiveErrorHandling specifies what to do with the message. “Move” means putting the message in a special automatically created queue called “poison”. Other values for the receiveErrorHandling are:

Fault: sends a fault to the listener that caused the ServiceHost to fault. The message must be removed from the application queue by some external mechanism before the application can continue to process messages from the queue
Drop: drops the poison message and the message is never delivered to the application
Reject: MSMQ will send a negative acknowledgement back to the sending queue that the application cannot receive the message. The message is placed in the sending queue dead-letter queue.

So all in all I have configured my service to try to receive and process the message 1 time, wait 20 seconds and try to receive again. This ends the 1 cycle I have configured. If after this the message still cannot be received or processed then I want to either store my message in the system dead letter queue or store it in a poison queue depending on the reason of failure (error in delivery or error in processing – respectively).

In my operation code now you will understand why I did manually roll back my transaction. I am simulating the required conditions to fail message processing and placing the message in the poison queue.

Now create a console client application and add a service reference to the service. Then use the below code to invoke the service:

using (TransactionScope sc = new TransactionScope(TransactionScopeOption.Required))
{
proxy.GetData(1);
sc.Complete();
}
I am enlisting the client in a transaction and I send the message to the private queue. It is important to note that the transactions enlisted at the client and the service are not related. The client transaction ends once the message is delivered to the queue; then the service transaction is in play trying to deliver the message from the queue to the service.

Run the example and examine the complete process:

First: The client sends a message to the private queue. DTC is used internally to accomplish this. You can see the stats of the DTC by examining the Local DTC Transaction Statistics from the Component Services on Windows 7. See below:

The client transaction ends here. If the transaction fails sending the message to the queue, the client will be notified and the transaction will roll back. This will be most helpful when the client is sending a group of messages to the queue and it’s important for all the messages to be either delivered as a batch or none is delivered.

Second, the service picks the message from the main queue. The below image shows when the message first arrives at the queue:

The WCF service operation starts executing and then the transaction is rolled back. The message is placed back in the retry queue as shown below:

After 20 seconds the message is delivered again to the WCF service; yet the transaction is rolled back again. This time the message is placed in the poison queue for good as shown below:

What happens for messages placed in the dead-letter queue or the poison queue? Custom application logic must be written to compensate these messages and do “something” about them.
Note that messages would have ended in the Transactional Dead-letter system queue had the message been failed to be delivered as opposed to fail during processing. The system queue is shown below:

What will happen in case of Public/Private queues?

Part 2 of this series will show the structure where the client and service are on separate machines (also MSMQ will be on a Windows Server Cluster…). But in a nutshell, in this case the following will happen:

The client WCF endpoint will be configured to send the message to the destination queue on the service machine (actually in case of MSMQ cluster it will send to a virtual IP, but this is a separate issue to be discussed in part 2), however, since the service is on another machine then the client application will silently deliver the message to an automatically created private outgoing queue. This queue will then try to deliver the message to the real destination queue.

The client will engage in one DTC transaction to deliver the message to the temp outgoing queue. The service will then be engaged in a separate DTC transaction to retrieve the message from the destination queue.

The same retry rules and poison/dead-letter message discussions mentioned above still apply.

But isn’t DTC heavy on Performance? Do I really need it?

Sure! And it depends.

Again if you follow the link (at the start) of my transactions article, you know that DTC implements a two-phase commit protocol and that definitely has a performance hit. However, whether you can live with or without transactions is governed by the scenario.

Let’s start with the obvious: in the scenario I presented above DTC is not required! The client needed the transactions “only” to commit sending messages to the queue. The service also needed the transaction “only” to commit receiving messages from the queue. So why did the DTC kick in? Simply because – as of this writing – it seems that the netMsmqBinding is configured to use DTC and there seems to be no way of changing that. But I do need transactions you might say, only not DTC; so what is the alternative? The answer would be MSMQ Internal Transactions. This is a light weight transaction implemented internally for MSMQ that does not need DTC. However, this seems to be not supported when using WCF netMsmqbinding; the only way you can use it is to use the System.Messaging APIs to send and receive from MSMQ.

To use the Internal Transaction you would use classes MessageQueue and MessageQueueTransaction from the System.Messaging API to send and receive messages within a transaction.

So why would I bother going for DTC (and netMsmqBinding) when I can go with internal transactions? Well because netMsmqBinding rocks…just kidding! Consider this scenario: when you receive the message at the service you want to use it to perform an update on a SQL server DB. If the update fails however, you want to place the message back in the queue so that you can fix the problem and retry again. Meaning that you want to enlist the receiving of the message and the DB update in a single atomic transaction: either all succeeds or the transaction rolls back and the message is back safe in the queue. In this case DTC is a must because the transaction is spanning two resource managers (again for more details see link at the start).

Note: I could not find any official say from MS that the netMsmqBinding does not support MSMQ internal transactions. I rather tested it myself and it did work.

Download Microsoft Open Door Azure Session materials

2010-11-08T21:54:00.004+03:00

I had a great time presenting about Windows Azure in Microsoft Open Door conference held in Riyadh and Jeddah.
Slides and Demos can be downloaded from Devlifestyle website:

http://www.devlifestyle.net/blogs/articles/archive/2010/11/08/ms-open-door-azure-session-materials.aspx

Installing Oracle Client on Windows Server 2008 R2 x64

2010-10-26T12:23:00.005+03:00

i had previously installed Oracle client 10g on win 2003 machine to connect to my Oracle db from my .NET app.

this time i wanted to do the same from my Win Server 2008 R2 x64 machine...
for 64 bit machines Oracle has released a 11g 64 bit client which you can get from here:
http://www.oracle.com/technetwork/database/enterprise-edition/downloads/112010-win64soft-094461.html
(win64_11gR2_client.zip).

Run the installation and select the first item (Oracle instance client):

Continue the installation...it will finish in no time.

However, after installation there was no tnsnames.ora and sqlnet.ora files...and there was no Network and Admin folders. In all previous installations these were the folders/files i deal with to set my settings and service names.

What i did next was wierd but worked! i manually created folder Network and inside it created folder Admin...the full path was:
C:\app\username\product\11.2.0\client_1\Network\Admin
And i created the tnsnames.ora and sqlnet.ora inside the above path.
That did the trick.

Before that...

If you are going to invoke Oracle client from a .NET application like me, you will get an error that says:
"Attempt to load Oracle client libraries threw BadImageFormatException. This problem will occur when running in 64 bit mode with the 32 bit Oracle client components installed."
if your application process is targetting 32-bit.

if you are using a console application, go to project properties --> advanced compilation options and change the target cpu to x64.
From IIS 7 make sure that the setting for the application pool accessing Oracle has "Enable 32 bit applications" set to false.

PS: i read that its an IIS 7 best practice to keep 32-bit processes (pools) working on x64 machines. so the above step might incur performance overhead. my next attempt will be to install Oracle client 32-bit on my x64 machine and see if that works...

Share your Opinions/Feedbacks with Microsoft!

2010-10-22T10:11:00.000+03:00

"Microsoft regularly conducts surveys for customers and partners to ask for your open and honest feedback (what we did right or wrong last 6 months). And we’re very much thankful for those who sharing the feedback..."

Click here!

Riyadh Online Community Summit Invitation

2010-10-06T23:50:00.002+03:00

Riyadh Online Community Summit will be the first mega online event featuring MVPs and community leads from many regions. For more details and registration please check the below link:

http://rocs1.eventbrite.com

Microsoft Open Doors

2010-10-06T02:07:00.002+03:00

Microsoft Open Doors (KSA) is around the corner. The event will take place in Riyadh and Jeddah and I will be talking about Windows Azure in both events. Fun stuf!

http://www.microsoft.com/saudi/about/opendoor/

What is ROCS?

2010-09-30T00:48:00.002+03:00

We are preparing for a mega ONLINE event in Riyadh...hence ROCS (Riyadh Online Community Summit). The summit will feature MVPs and Community Leads from many regions in the world and the discussions will cover a wide array of topics ranging from client to mobile programming, database development and administration, all the way to collaboration and systems integration.

The event will be on October 21 and 22 and you can watch it live from anywhere. The event details and participation link will be published soon.

Managing Workflow Services with Windows Server AppFabric

2010-09-06T22:32:00.002+03:00

Recently I started a series of posts tackling WF 4 Services development. Those posts used self hosting for all the examples. This post tackles the super cool AppFabric extensions that makes hosting WF Services in IIS a delightful experience.

Read full article HERE

WF 4 Persistence, Tracking, and Bookmarks: Practical Approach

2010-08-21T23:54:00.002+03:00

Shows practical examples about WF 4 persistence, tracking, and bookmarks.
See full article HERE

WF 4.0 WorkflowServiceHost Custom Tracking

2010-07-06T15:09:00.008+03:00

In previous two-part post (here and here) I showed how to add persistence to a workflow hosted via the WorkflowServiceHost. In this post I will do some modifications on the same example to add tracking support. Now since I am a good guy and would like to make your lives easier, I will repost the full code here instead of pointing out only the differences. Let’s roll…

Below is our simple workflow (Workflow1.xaml) (use Workflow Console Application template from VS 2010):

It receives a request from a client application, using a custom activity it retrieves the workflow instance id and returns it as a response to the client. Finally a delay shape delays execution for 30 seconds.

Below is the code for the custom activity:

public sealed class CodeActivity1 : CodeActivity
{
public OutArgument WFInstanceId { get; set; }

protected override void Execute(CodeActivityContext context)
{
context.SetValue(WFInstanceId, context.WorkflowInstanceId);

CustomTrackingRecord customRecord = new CustomTrackingRecord("CustomInfo")
{
Data =
{
{"Date", DateTime.Now.ToShortDateString()},
}
};

context.Track(customRecord);
}

}

The custom activity returns the workflow instance id and creates a new CustomTrackingRecord. CustomTrackingRecords are any custom information you want to be able to track. In this simple example I am tracking a custom variable which stores the current date…more on this in a moment.

Below is the code from the Program.cs file needed to set up the host and tracking:

class Program
{
static void Main(string[] args)
{

string baseAddress = "http://localhost:8089/TestWF";

using (WorkflowServiceHost host =
new WorkflowServiceHost(new Workflow1(), new Uri(baseAddress)))
{
host.Description.Behaviors.Add(new
ServiceMetadataBehavior() { HttpGetEnabled = true });

host.AddServiceEndpoint("IService",new BasicHttpBinding(), baseAddress);

TrackingProfile fileTrackingProfile = new TrackingProfile();
fileTrackingProfile.Queries.Add(new WorkflowInstanceQuery
{
States = { "*" }
});
fileTrackingProfile.Queries.Add(new ActivityStateQuery()
{
ActivityName = "*",
States = { "*" },
// You can use the following to specify specific stages:
// States = {
// ActivityStates.Executing,
// ActivityStates.Closed
//},
Variables =
{
{ "*" }
} // or you can enter specific variable names instead of “*”

});
fileTrackingProfile.Queries.Add(new CustomTrackingQuery()
{
ActivityName = "*",
Name = "*"
});

FileTrackingParticipant fileTrackingParticipant =
new FileTrackingParticipant();
fileTrackingParticipant.TrackingProfile = fileTrackingProfile;
host.WorkflowExtensions.Add(fileTrackingParticipant);

host.Description.Behaviors.Add(new WorkflowIdleBehavior()
{
TimeToPersist = TimeSpan.FromSeconds(5),
TimeToUnload = TimeSpan.FromSeconds(20)
});

host.Open();
Console.WriteLine("Car rental service listening at: " +
baseAddress);
Console.WriteLine("Press ENTER to exit");
Console.ReadLine();
}

}
}

public class FileTrackingParticipant : TrackingParticipant
{
string fileName;
protected override void Track(TrackingRecord record,
TimeSpan timeout)
{
fileName = @"c:\tracking\" + record.InstanceId + ".tracking";
using (StreamWriter sw = File.AppendText(fileName))
{
WorkflowInstanceRecord workflowInstanceRecord = record as WorkflowInstanceRecord;
if (workflowInstanceRecord != null)
{
sw.WriteLine("------WorkflowInstanceRecord------");
sw.WriteLine("Workflow InstanceID: {0} Workflow instance state: {1}",
record.InstanceId, workflowInstanceRecord.State);
sw.WriteLine("\n");
}

ActivityStateRecord activityStateRecord = record as ActivityStateRecord;
if (activityStateRecord != null)
{
IDictionary variables = activityStateRecord.Variables;
StringBuilder vars = new StringBuilder();

if (variables.Count > 0)
{
vars.AppendLine("\n\tVariables:");
foreach (KeyValuePair variable in variables)
{
vars.AppendLine(String.Format(
"\t\tName: {0} Value: {1}", variable.Key, variable.Value));
}
}

sw.WriteLine("------ActivityStateRecord------");
sw.WriteLine("Activity DisplayName: {0} :ActivityInstanceState: {1} {2}",
activityStateRecord.Activity.Name, activityStateRecord.State,
((variables.Count > 0) ? vars.ToString() : String.Empty));
sw.WriteLine("\n");
}

CustomTrackingRecord customTrackingRecord = record as CustomTrackingRecord;
if ((customTrackingRecord != null) && (customTrackingRecord.Data.Count > 0))
{
sw.WriteLine("------CustomTrackingRecord------");
sw.WriteLine("\n\tUser Data:");
foreach (string data in customTrackingRecord.Data.Keys)
{
sw.WriteLine(" \t\t {0} : {1}", data, customTrackingRecord.Data[data]);
}
sw.WriteLine("\n");
}
}
}
}

So let me explain what’s going on: in a nutshell, when you deal with WF 4.0 tracking you have to understand three concepts:

Tracking records: these are the information records emitted by your workflow. There are four derived classes that correspond to the four types of tracking records:

WorkflowInstanceQuery: events about workflow instance state. For example started, suspended, unloaded, etc...
ActivityStateQuery: events about activities inside the WF
CustomTrackingQuery: any custom information you want to track
BookmarkResumptionQuery: bookmark name you want to track whenever this bookmark is resumed (I won’t cover bookmarks in this post to keep the concept clear, but I will have another dedicated post for this topic)

Tracking profiles: act like filters over tracking records in order to track only required information
Tracking participants: the medium where tracking information will be written to. WF 4.0 comes with one participant which writes to ETW (event tracking for windows). Custom participants can be easily developed to write tracking data into SQL Server for example, or like I did in this example into the File System

In my code, I create an instance of class TrackingProfile and added three types of records: WorkflowInstanceQuery, ActivityStateQuery, and CustomTrackingQuery.

For the WorkflowInstanceQuery I asked the profile to track all workflow states using the “*” operator. I could also limit the number of states that I want to track (check commented code). Recall that in my workflow I have states of persist and unload (review behavior configuration and delay timespan) so you will see these states in he tracked data. For the ActivityStateQuery I asked the profile to track all states for all activities as well as any variables declared for these activities. Similarly for the CustomTrackingQuery I asked the profile to track any custom data I defined (recall that I did define custom data in my custom activity).

Now that I have defined the tracking records as well as the tracking profile; final step is to define the tracking participant. I could have used the out of the box EtwTrackingParticipant class which writes to ETW, but to make things more interesting, I have created a custom tracking participant class (FileTrackingParticipant) which derives from TrackingParticipant. Check the code for that class which simply gets the information for each of the tracked records and writes it into a file.

Back to the main program, I associate my tracking profile (fileTrackingProfile) with my tracking participant (fileTrackingParticipant) and finally add the participant to the set of the host WorkflowExtensions.

To make sure you will have a running program, below is the trivial client program code:

ServiceReference1.ServiceClient proxy = new ServiceReference1.ServiceClient();

Guid workflowId = (Guid)proxy.Operation1();

Console.WriteLine("client done");
Console.WriteLine("enter key");
Console.ReadLine();

Run the service so that it is listening to requests on “http://localhost:8089/TestWF“, now run the client console and wait until the service console window prints “Workflow Ended”. Now examine the file where tracking is written and you will see workflow events, activity events, variables, and custom information all tracked…see the below image (reformatted for presentation):

WCF NetMSMQBinding on IIS 7

2010-07-03T14:14:00.009+03:00

I had a previous post (click here) about how to build a WCF queued messaging scenario. that example used self-hosted services. you can refer back to that post for all the details. in this post i will quickly show you how to get a WCF service with MSMQ binding hosted in IIS 7 up and running.

Create a simple WCF web site, and add a simple operation as follows:

[ServiceContract]
public interface IService
{

[OperationContract(IsOneWay=true)]
void GetData(int value);

}

below is the service configuration:

<system.serviceModel>

<services>
<service name="Service" behaviorConfiguration="ServiceBehavior">
<host>
<baseAddresses>
<add baseAddress="http://localhost/TestWCFMSMQ" />
</baseAddresses>
</host>

<endpoint address=
"net.msmq://localhost/private/TestWCFMSMQ/Service.svc" binding=
"netMsmqBinding"
contract="IService"
bindingConfiguration="NoMsmqSecurity"/>
<endpoint address="mex"
binding="mexHttpBinding"
contract="IMetadataExchange"/>
</service>
</services>

<bindings>
<netMsmqBinding>
<binding name="NoMsmqSecurity">
<security mode="None" />
</binding>

</netMsmqBinding>
</bindings>

<behaviors>
<serviceBehaviors>
<behavior name="ServiceBehavior">

<serviceMetadata httpGetEnabled="true"/>
<serviceDebug includeExceptionDetailInFaults="true"/>
</behavior>
</serviceBehaviors>
</behaviors>
<serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
</system.serviceModel>

This is a WCF services hosted on IIS, listening to a MSMQ endpoint through the net.msmq binding. also I have exposed the mex http endpoint to enable adding a service reference from the client application.

next we need to enable the net.msmq binding on IIS 7 as its not enabled by default. on the web site hosting your service and on the service itself, go to configuration and add "net.msmq" to the list of bindings. by default it will only have "http".

now lets turn our focus into MSMQ itself. turn on the MSMQ feature from the control panel. once you do you will be able to access it through right clicking on My Computer and clicking Manage. from there create a private queue called "TestWCFMSMQ/Service.svc" if you followed the same naming i used. else use "[yourvirtualdirectoryname]/[myservicefile].svc". i will explain later why this naming convention is important.
next, give the application pool identity account sufficient priveleges on the queue created. this can simply be done by right click and properties on the queue itself.

final step is to make sure the windows service "Net.Msmq Listener Adapter" is running. this is the service that checks for new messages in queues and notifies the WCF service about them. now for some reason that at the time of this post i do not yet know, this service failed to notice the messages in the queue unless the queue name is in the format i created above...will post an update once i know what's the catch here...

oh,one more thing: when creating the private queue make sure to make it Transactional.

now create a simple console application and add a service reference to the WCF service using the mex http endpoint. use the below code to invoke the service operation:

ServiceReference1.ServiceClient proxy = new ServiceReference1.ServiceClient();

using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required))
{
proxy.GetData(12);
scope.Complete();
}

Console.WriteLine("end");
Console.Read();

to see queuing in action. from IIS stop the WCF service. now run the client which drops a message in the private queue. open the queue and notice the message. now from IIS run the WCF application, wait for a couple of seconds and the message will be delivered to the service and disappear from the queue.

WF 4.0 WorkflowServiceHost Persistence – Part 2

2010-07-03T10:20:00.004+03:00

In part 1 I configured the Persistence store for a WF 4.0 workflow services hosted by WorkflowServiceHost. I showed how the workflow is persisted just before sending a response to a client application and observed the persisted state in the database.

In part 2 I will add a host behavior to persist the workflow based on an Idle time as opposed to before sending the response from the workflow itself, moreover I will also add a host behavior to unload the workflow after a specified time interval thus showing the difference between “just” persisting a workflow and unloading it from memory.

Let’s start by unchecking (disabling) property PersistBeforeSend of the SendReplyToReceive shape. Next add the following code to the Program.cs file of the workflow application just before the “host.Open()” statement:

host.Description.Behaviors.Add(new WorkflowIdleBehavior()
{
TimeToPersist = TimeSpan.FromSeconds(5),
TimeToUnload = TimeSpan.FromSeconds(20)
});

This code adds a behavior to persist the WF instance after 5 seconds of inactivity and to unload it completely after 20 seconds of inactivity. What’s the difference? Persistence – as explained in Part 1 – persists the workflow state in the DB but keeps the workflow instance running. Unloading however, does two things: it also persists the WF state but also unloads it from memory; typically for a long running process.

Build the WF and run it. It’s ready to listen to requests on its endpoint.

Now run the client. Once the service sends the response back to the client (recall – no persistence on send here), it enters a delay shape for 1 minute. Meanwhile we have configured to persist the WF after 5 seconds. So wait 5 seconds and open table “InstancesTable”; just like in Part 1 you will see the WF state persisted:

However, now we have also configured the WF to unload after 20 seconds. So after an additional 15 seconds, this time open table “RunnableInstancesTable” and you will see one record corresponding to the same workflow. Why the “RunnableInstancesTable” table? Because this time the WF is not only persisted, but also is unloaded meaning that it is ready to be loaded again and continue execution:

After the delay shape finishes, the WF instance will be loaded again into memory and finishes execution. Both records will disappear from the database.

Finally, let’s add a control endpoint to our WF. Control endpoints allow sending commands to a WF instance from a client. Add the below code in the host application:

WorkflowControlEndpoint controlEndpoint =
new WorkflowControlEndpoint(
new BasicHttpBinding(),
new EndpointAddress(new Uri(baseAddress) + "/wce")
);
host.AddServiceEndpoint(controlEndpoint);

This code adds a special type of endpoint called the control endpoint with the following address: “http://localhost:8089/TestWF/wce”
Run the service and update the service reference at the client application. Add the below code at the client application just after invoking the service operation:

System.Threading.Thread.Sleep(new TimeSpan(0, 0, 30));
WorkflowControlEndpoint ep = new WorkflowControlEndpoint(new BasicHttpBinding(), new EndpointAddress("http://localhost:8089/TestWF/wce"));
WorkflowControlClient client = new WorkflowControlClient(ep);
client.Terminate(workflowId);

This code creates a workflow control client and uses the workflow instance id to send a command to terminate the WF instance. To test this I have caused the thread to sleep for 30 seconds. This way I am sure that at this point the WF instance has been unloaded (review the WF configuration). Now when I send the Terminate command, notice how the WF instance does not continue execution and is terminated. To be sure, see how the database persistence records are deleted after just 30 seconds as opposed to the 1 minute as explained previously.

WF 4.0 WorkflowServiceHost Persistence – Part 1

2010-07-03T10:00:00.009+03:00

First step is setting up the new persistence databases. Go to “C:\Windows\Microsoft.NET\Framework\v4.0.30319\SQL\en” and there you will find two files:
- SqlWorkflowInstanceStoreSchema.sql
- SqlWorkflowInstanceStoreLogic.sql

In SQL Server (Express or above), create a database with a convenient name (my database is called “SqlWorkflowInstanceStore”) and run the above scripts to create the required objects.

The persistence model in WF 4 is different than that of WF 3.5. the new model is now called InstanceStore instead of PersistenceProvider. For WF 3.5, the PersistenceProvider databases are still available in the same location:
- SqlPersistenceProviderLogic.sql
- SqlPersistenceProviderSchema.sql

In WF 3.5 there was service for managing WFs such as waking up workflow instances… the SQL files for this service is also still available in the same location:
- SqlPersistenceService_Logic.sql
- SqlPersistenceService_Schema.sql

For tracking in 4.0 there is no tracking participant that writes to a SQL. The Tracking_Schema.sql applies to 3.5 WFs. Out of the box with WF 4.0 there is a tracking participant that writes to ETW (Event Tracing for Windows). Again, those files for WF 3.5 are:
- Tracking_Schema.sql
- Tracking_Logic.sql

Building the WF

Create a new Console Workflow Application, and use the default Workflow1.xaml to build the following simple workflow:

The Receive and SendReplyToReceive shape receives an empty request and returns the workflow instance id to the caller. The CodeAcitivty1 is a custom activity that retrieves the workflow instance id. Below is the code for this activity:

public sealed class CodeActivity1 : CodeActivity
{

protected override Guid Execute(CodeActivityContext context)
{
return context.WorkflowInstanceId;
}
}

The important catch is to set the property PersistBeforeSend on the SendReplyToReceive shape. This will cause the WF to persist just before sending the response to the caller. This means that now the WF will have a point to go back to in case the WF is suspended or unloaded…more on that later.

The Delay activity is used to delay the WF for 1 minute…we will see why is that required after moments.

Now go to the Program.cs file and let’s write the code required to host the WF and set up persistence.

Use the following code and then I will explain it:

class Program
{
static void Main(string[] args)
{
//WorkflowInvoker.Invoke(new Workflow1());

string baseAddress = "http://localhost:8089/TestWF";

using (WorkflowServiceHost host =
new WorkflowServiceHost(new Workflow1(), new Uri(baseAddress)))
{
host.Description.Behaviors.Add(new
ServiceMetadataBehavior() { HttpGetEnabled = true });

host.AddServiceEndpoint("IService",new BasicHttpBinding(), baseAddress);

SqlWorkflowInstanceStore instanceStore = new SqlWorkflowInstanceStore(@"Data Source=.\SQLServer2008;Initial Catalog=SqlWorkflowInstanceStore;Integrated Security=True");
host.DurableInstancingOptions.InstanceStore = instanceStore;

host.Open();
Console.WriteLine("Car rental service listening at: " +
baseAddress);
Console.WriteLine("Press ENTER to exit");
Console.ReadLine();
}

}
}

By default, the WorkflowInvoker is used to invoke the WF. I changed this into the WorkflowServiceHost, the reason is that the WorkflowInvoker cannot be configured with Persistence. WorkflowServiceHost is used to host WF Services. The other method of calling (hosting) workflows is the WorkflowApplication which – like WorkflowServiceHost – can be used to host long running asynch workflows with extensions (Persistence, Tracking) however only for workflows that are non services.

After creating my WCF endpoint and adding it to the host, I use the SqlWorkflowInstaceStore class to configure my persistence store, which is the same store I prepared when I first started.

Creating the Client

Run the WF so that the host is running and the WCF endpoint is listening for requests on “http://localhost:8089/TestWF”
Now create a client (console) application and add a service reference to the above URL. Write the following code to invoke the service operation:

Run the client and observe the scenario: once the operation is invoked (and the workflow instance id is returned), open the table “InstancesTable” and you will see a record for a persisted WF. Why is that? Recall that we set a property on the SendReply shape to persist just before sending the reply. For this the WF has persisted state in the database but is still running as we did not suspend or unload the WF. See images below:

For how much time will this record appear in the database? For 1 minute because we have a delay shape in the WF to delay completion for 1 minute after which the WF execution will complete and the record will disappear from the database.

In part 2 I will add support (behavior) to persist and unload the WF as well as add control endpoint to execute remote operations on the WF.

Web Service First Call Delay - SSL Handshake

2010-06-14T12:06:00.003+03:00

i faced this issue recently and hope it helps someone...so here we go:

a service provider gave me a XML Webservice to consume. this service was SSL secured and they gave us the certificate to install. However, they did not give us the CA...lets move on.

i created a .net console app to test the web service and of course i had to write the code to ignore the certificate warning since again i did not have the CA. now do not judge me just yet :) instantly i mailed the web service owner asking for the CA; but it took some time...

so meanwhile, my console app worked just fine. now i went to my main ASP.NET application and used the same code to invoke the web service...and there was a big problem: the first call took forever (more like 30 seconds really), then the subsequent calls were fast until my application is restarted!
now since the console application worked just fine (first call was fast) but the problem was only in the ASP.NET application i moved my attention immediately to the serialzation issue.

i read alot about how the serializer dll might take time to load the first time and thus the first call was slow. Now although this does not really explain why the problem occured only when calling the web service from my IIS hosted web site; however, i still used the SGen tool to pre-generate the serializer assembly in the form of [myproxyassemblyname].XmlSerializers.dll and put it inside my bin folder but without any result...still the first call was too slow.

Note: instead of using the SGen tool you can use the compile options from VS and turn on the generate serialzer assembly option which will create the serialzer dll for you...

now out of no where, the service provider sent my the CA. i installed it in the trusted root certificate authorities folder, removed the warning-ignoring code and boom the first call was fast and the problem was solved!
thinking about it, the SSL handshake on the first call was taking a lot of time to find a problem in the CA and generating a warning.

i have to admit though, it was a by chance. because till this moment i do not know why the console app did not complain about the SSL handshake (or to be more accurate, the SSL handshake did not take much time) while when from IIS the SSL handshake took about 30 seconds...the first logical explanation - although do not quote me on this - is that the IIS worker process was much slower than the .NET managed assembly (console app) in dealing with the SSL handshake issues...

anyway, i can live with this for now :)

Windows Azure Videos

2010-06-05T12:17:00.004+03:00

...so since i am in the mood of posting videos :) here are the videos for a chalk-talk session i gave with my buddy Mohamed Mosallem in Microsoft GDC 09 about Windows Azure...its a late post but better late than never :)
Part1:

Part2:

Part3:

Part4:

Part5:

Part6:

Part7:

Devlifestyle May Event - Session Recordings

2010-06-04T12:38:00.004+03:00

As part of Devlifestyle May 2010 Event, we talked about Windows 7 Sensor and Location Platform and SQL Server 2008 R2 Data Tier Applications. Below are the vidoes:

Windows 7 Sensor and Location Platform: Part1, Part2, Part3, Part4

Data tier applications, a new feature for SQL Server 2008 R2 and VS2010: Part1, Part2, Part3, Part4, Part5

Building and Managing WCF and WF 4 Services with Windows Server AppFabric

2010-06-03T12:15:00.005+03:00

Here are my session recordings talking about Building and Managing WCF and WF 4 Services with Windows Server AppFabric, as part of Devlifestyle Event...
Part1:

Part2:

Part3:

Part4:

Part5:

Part6:

Part7:

Part8:

Devlifestyle Event on May 27

2010-05-18T09:58:00.000+03:00

Our next community event is just around the corner. we will be discussing a lot of hot topics: Win Server AppFabric, Win 7 Location and Sensor Platform, Windows Azure, SQL 2008 R2, Silverlight 4.

View here
Cannot wait to start!

Clean AppFabric Databases

2010-05-18T00:05:00.000+03:00

When you are developing and testing with AppFabric, you will quickly notice how much the tracked data size grows. as such you will need at times to empty the monitoring and persistence databases and start fresh.

for this there is a great post by Ron Jacobs about how to do just that.

however, here are some minor issues that i faced to make this work. it might help you also.

the powershell script to use is the following:
-------------------------------------------------------
import-module applicationserver
$ConfirmPreference = "None"

Remove-ASPersistenceSqlDatabase -Force -Server ".\SQL2008" -Database "AppFabricPersistence"

Initialize-ASPersistenceSqlDatabase -Admins $env:computername\AS_Administrators -Readers $env:computername\AS_Observers -Users "BUILTIN\IIS_IUSRS" -Database "AppFabricPersistence" -Server ".\SQL2008"

Clear-ASMonitoringSqlDatabase -Database "AppFabricMonitoring" -Server ".\SQL2008"
---------------------------------------------------

where AppFabricPersistence and AppFabricMonitoring are the database names i used when configuring AppFabric. while SQL2008 is my SQL named instance where AppFabric is configured.

Save this text into a ".ps1" file and take care to remove line breaks. meaning that you will have a total of four lines. if you don't you will get script execution errors when later you run the script.

now when you run powershell, make sure to do so in admin mode. in addition run the one called Windows Powershell and not Windows Powershell (x86). if you run the X86 verison you will get an error that module "applicationserver" cannot be loaded when you try to run the script. note that my setup was on Windows 7 64-bit.

we are still not done, again now if you try to run the script in powershell by typing [filename].ps1 you will get an error ".ps1 is not recognized as the name of a cmdlet, function, script file, or operable program...".
to solve this type ".\[filename].ps1"

Finally, when you run the script you might get an error that "the execution of scripts is disabled on this system". you need to enable script execution for powershell. for this run command "set-executionpolicy unrestricted" for allowing all scripts. another key values might be "Restricted", "AllSigned", and "RemoteSigned".

again i point that my environment was win 7 64-bit, so in another environment there could be some differences...but anyway...

this should be enough then to run the script and watch your AppFabric dashboard clean again!

My MVP Interview

2010-04-16T01:19:00.000+03:00

As being a new MVP, I was interviewed by MicrosoftFeed. Here is the link:
http://microsoftfeed.com/2010/meet-mohamad-halabi-aspasp-net-mvp-from-saudi-arabia/

My TechEd 2010 Sessions

2010-04-09T23:04:00.000+03:00

I was a speaker at Tech Ed 2010 in Dubai.
Below are links for my two breakout sessions (i had 3 other labs but are not recorded :( )

BizTalk Dev

Windows Azure Dev

Introducing .NET 4.0 Parallel Programming

2010-04-07T20:01:00.000+03:00

Download sln - 858.6 KB

“With great power comes great responsibility”…As the hardware world is booming and its processing power increases, the responsibility lies with developers to utilize such power. Ok, so the comparison is lame but I am a fan of Spider Man movie and was always looking for a way to use its slogan…guess should keep looking :)

Seriously though, multi-core (and multi-processer) machines are dominant these days. Soon enough as reported by some lead vendors, you will not be able to find machines with less than 8 cores. As such, the traditional way of writing serial programs will not be able to make use of such power by default. These programs should be written in a way which utilizes the power of multi-cores.

Do we have to? Some might say. Wouldn’t running my program on a multi-core machine automatically improve performance; after all there are more threads? The answer is no! Running a serial program on a multi-core machine has nearly zero performance enhancements. With the exception of IO operations, your program will still use one thread of a single core at a time. Thread switching occurs – which incurs overhead by the way – but at all times only one thread will be active (again with the exception of IO operations which can happen on different threads).

Multi-threading programming is the way which allows developers to take advantage of multiple threads concurrently. And while it has been available for many years, multi-threading programming has often been avoided due to its complexity. Developers had to put most their effort in writing clean multi-threaded code rather than focusing on the business problem.

So what does Multithreading have to do with multi-cores? Cannot we write Multi-threaded code if we have a single core? Yes we can. But again as long as we have a single core, one thread at a time will work since all threads do belong for that same single core…logical enough. When your Multi-threaded program runs on a multi-core machine, threads belonging to different cores will be able to run together and hence your program will take advantage of the hardware power.

One final note before going on: Multithreading programming does not differ if you have multi-core machine or multi-processer machine. This is a hardware separation; underneath your code will be working with threads all the same.

Parallel vs. Concurrent

Well both concepts are related but not the same: In its simplest definition, Concurrency is about executing multiple un-related tasks in a concurrent mode. What these un-related tasks share are system resources, but they do not share a common problem to solve; meaning, they are logically independent. Now think of these tasks as threads. And since these threads share system resources, then you face the globally common problems of concurrent programming such as deadlocks and data races. This makes concurrent programming extremely difficult and debugging complicated.

So what is Parallelism? Parallelism is taking a certain task and dividing it into a set of related tasks to be executed concurrently. Sweet! So again thinking of these tasks as threads, Parallel Programming still has the same problems of concurrent programming (deadlocks, data races, etc...) and introduces new challenges most notably sharing and partitioning data across these related threads. This makes Parallel Programming even more difficult and debugging even more complicated :(

However, it is not all bad news. .NET 4.0 parallel programming is a great step onwards. The new API solves a lot of problems (but not all) of Parallel Programming and greatly eases up parallel debugging...

Setting the Context of .NET Parallel Programming

The previous sections introduced Multi-threading programming and its need. This section introduces the context of Parallel Programming in the .NET Framework in a gradual manner:

So we have Multi-core machines and we need to think of a way to utilize their power…So is the need to think of Parallel Programming
So what is Parallel Programming all about? It’s about how to partition a single piece of work into multiple concurrent units
What partitioning options do we have? Option1 would be static partitioning. Static Partitioning is “hard-coding” the number of units and threads you want to use. For example saying that you want to divide a large chunk of work into “n” units each running on a single thread over a single core. The problem with this approach is that it is not scalable. While this solution would be fine on a machine with “n” cores, it quickly will become un-scalable for machines with “n+m” cores.
So the natural evolution of solving the partitioning problem would be Dynamic Partitioning. This means that you divide your work into a single unit per available core. While this would solve the scalability problem, it introduces another one: Load-imbalance. In real problems it is really unlikely that all work units will take the same amount of time to complete. This means that some cores will finish their work while other cores will be still busy without being able to utilize the now-free cores.
Moving up the chain of solutions we now have one more option: Over partition work into the smallest feasible units which will make use of the most feasible number of available threads. This will now solve the load-imbalance issue because as work is divided into the smallest possible units where each unit will run over an available thread. However, again now we have a problem: Thread overhead. Creating and killing threads is an expensive procedure.
So this brings us to the final partitioning option: The .NET Thread Pool. The .NET thread pool maintains a scheduler and a thread queue. Instead of killing a thread once it finishes its work, this thread is pushed into the queue and it is now available to be picked up by the next unit of work scheduled for execution. The thread pool is exposed through the System.Threading.ThreadPool namespace and it has been available since before .NET v 4.0.

Wait a minute, so what is exactly new in the .NET 4.0 Parallel Programming model if the above discussion applies to pre-v 4.0? The answer will come by inspecting the shortcomings of the thread pool as discussed. The thread pool is a fire-and-forget programming model. Once you created a thread by code, you basically have no way to perform any operations on it. There is no support for continuing, composing, monitoring data flow, performing work cancellation and work waiting.

So what is the solution? It is the .NET 4.0 Task-based Programming. In v 4.0 of the framework you can program over Tasks as opposed to the lower-level threads. Now you can divide your work over Tasks instead of threads. This will give you much more power and control over your parallel work as we will see later. Of course you can still work with the thread pool directly if you want to work with the deeper-detail of threads.

.NET 4.0 Parallel Programming

The Parallel Programming model of the .NET 4.0 is composed of the following:

The Task Parallel Library (TPL): this is the base of the Task-based programming discussed in the previous section. It consists of:

Task class: the unit of work you will code against instead of the previous thread model
Parallel class: a static class that exposes a task-based version of some parallel-nature problems. More specifically it contains the following methods:
- For
- Foreach
- Invoke

Parallel LINQ (PLINQ): built on top of the TPL and exposes the familiar LINQ as Parallel extensions.

About the Examples

All examples discussed are downloadable. To illustrate main ideas, some code samples will be shown in the below sections.

All examples are built using VS 2010 RC1; nothing else is required. In order to feel the value of Parallel Programming you will need a multi-core (or multi-processer) machine; the results shown are that of running the examples on a dual-core machine.

Example1: Tasks

The first example shows the new Tasks programming model. As explained previously, Tasks – as opposed to the Thread Pool – grants more control over thread programming. In fact you now deal with Tasks and not threads.

static void Main(string[] args)
        {
            Task t = Task.Factory.StartNew(() =>
            {
                Console.WriteLine("I am the first task");
            });

            var t2 = t.ContinueWith(delegate
            {
                //simulate compute intensive
                Thread.Sleep(5000);
                return "Tasks Example";
            });

            //block1
            //string result = t2.Result;
            //Console.WriteLine("result of second task is: " + result);
            //end block1

            //block2
            //t2.ContinueWith(delegate
            //    {
            //        Console.WriteLine("Here i am");
            //    });
            //Console.WriteLine("Waiting my task");
            //end block2

            Console.ReadLine();

        }

In the above code we first create a new Task using the Task class and we pass to the lambda expression the code to run inside that Task (in a separate thread).

Another feature is the ability to “continue” on a running Task. What this means is that once the Task is completed (in our example task “t”) start running another task “t2” with the code to run supplied in an anonymous method (through a delegate).

Now uncomment block1. Here we see another feature which is to get a result back from a task. Notice that we have assigned a value for task “t2” called “Tasks Example” through the “return” statement. Now in block1 we can retrieve that value. Now this kicks in a question: what will happen if the statement “t2.Result” is executed before “t2” finishes execution (recall that “t2” is running on a separate thread other than the thread of the main program)? Force this behavior (that is why I am using Thread.Sleep) and you will see that in this case the main thread will wait at “t2.Result” until “t2” finishes execution and its “return” statement is executed. Neat! A power which was not possible before.

Now uncomment block2. Set two break points at the first and second “Console.WriteLine” statements. Run the program and notice that line “Waiting my task” is printed before “Here i am”. This indicates that the main program (thread) was waiting task “t2” running on another thread to finish. Once “t2” finishes (delayed by using Thread.Sleep) its statement is printed.

Example2: Parallel

This example introduces the Parallel class. Specifically it shows how to use the new For method to perform what usually was done serially in a parallel manner.

static void Main(string[] args)
        {
            Stopwatch watch;
            watch = new Stopwatch();
            watch.Start();

            //serial implementation
            for (int i = 0; i < 10; i++)
            {
                Thread.Sleep(1000);
                //Do stuff
            }
            watch.Stop();
            Console.WriteLine("Serial Time: " + watch.Elapsed.Seconds.ToString());

            //parallel implementation
            watch = new Stopwatch();
            watch.Start();
            System.Threading.Tasks.Parallel.For(0, 10, i =>
            {
                Thread.Sleep(1000);
                //Do stuff with i
            }
            );
            watch.Stop();
            Console.WriteLine("Parallel Time: " + watch.Elapsed.Seconds.ToString());

            Console.ReadLine();

        }

The above code shows the difference in execution time of performing a unit of work (simulated by a Sleep operation) serially versus in parallel. In my dual-core machine the time is cut by half. Now this is huge improvement! Imagine all the work you usually do inside For loops (or Foreach loops) and think of the time enhancement you now can get by utilizing the new Parallel.For and Parallel.Foreach methods.

One important note: sometimes it’s just not that easy though. When parallelizing your work sometimes you have to take care of special problems like synchronization and dead-locks; you still have to handle these and I will discuss them in the next examples.

For the sake of variation, let’s see an overload of the Parallel.For method: In order to support breaking out of a parallel loop, there is an overload of the Parallel.For method which takes ParallelLoopState as a parameter in the Action delegate:

Parallel.For(0,1000,(int i, ParallelLoopState loopState) =>
{
    If(i==500)
       loopState.Break();
       //or loopState.Stop();
      return; // to ensure that this iteration also returns
     //do stuff
});

The difference between the Break and Stop methods is that Break insures that all earlier iterations that started in the loop are executed before exiting the loop.

The Stop method makes no such guarantees; it basically says that this loop is done and should exit as soon as possible.

Both methods will stop future iterations from running.

Example3: PLINQ

PLINQ is a LINQ Provider, so you still use the familiar LINQ model. In the underlying model however, PLINQ uses multiple threads to evaluate queries.

static void Main(string[] args)
        {
            Stopwatch watch;

            //for loop
            watch = new Stopwatch();
            watch.Start();
            bool[] results = new bool[arr.Length];
            for (int i = 0; i < arr.Length; i++)
            {
                results[i] = IsPrime(arr[i]);
            }
            watch.Stop();
            Console.WriteLine("For Loop took: " + watch.Elapsed.Seconds);

            //LINQ to Objects
            watch = new Stopwatch();
            watch.Start();
            bool[] results1 = arr.Select(x => IsPrime(x))
                .ToArray();
            watch.Stop();
            Console.WriteLine("LINQ took: " + watch.Elapsed.Seconds);

            //PLINQ
            watch = new Stopwatch();
            watch.Start();
            bool[] results2 = arr.AsParallel().Select(x => IsPrime(x))
                .ToArray();
            watch.Stop();
            Console.WriteLine("PLINQ took: " + watch.Elapsed.Seconds);

            Console.ReadLine();

        }

The above code shows how to perform a deliberately time-expensive IsPrime function using LINQ (and For) vs. PLINQ. The AsParallel extension allows your code to run using PLINQ instead of LINQ. The step of using AsParallel wraps the data source with a ParallelQuery wrapper and causes the remaining extension methods in the query to bind to PLINQ rather than to LINQ to Objects.

Run the example and will notice that while the time for execution using serial For loop and traditional LINQ is nearly the same (actually opposite to many beliefs, LINQ has a performance overhead over For loop…but that’s another story), the execution time using PLINQ is again nearly cut in half.

Example4: PLINQ Extensions

PLINQ has many extensions that affect how work partitioning is done. In this example we will discuss one of these extensions. Examine the below code:

static void Main(string[] args)
        {
            Stopwatch watch = new Stopwatch();
            watch.Start();
            int[] src = Enumerable.Range(0, 200).ToArray();
            var query = src.AsParallel()
                         .Select(x => ExpensiveFunc(x));

            foreach (var x in query)
            {
                Console.WriteLine(x);
            }
            watch.Stop();
            Console.WriteLine("Elapsed: " + watch.Elapsed.Seconds.ToString());
                    
            Console.ReadLine();
        }

        private static int ExpensiveFunc(int x)
        {
            Thread.Sleep(1);
            return x;
        }

This code is similar to the previous example. It shows how to execute a function using PLINQ’s AsParallel extension. Now run the program. What you will see is something that you might not have expected. Notice that values 0 through 199 are printed however they are not printed in the correct order.

Think about it: we are using an enumerator from 0 to 200. However since we are using PLINQ and not LINQ; the partitioning itself is not serial. The work is divided into random parallel units; for example from 0 till 10 is assigned to thread A while from 11 till 30 is assigned to thread B and so on. Once a unit of work is done its output is immediately flushed back to the final result. That’s why the output is not arranged. The below image gives an indication about this:

Now edit the code and use the AsOrdered extension as follows:

var query = src.AsParallel().AsOrdered()
                         .Select(x => ExpensiveFunc(x));

Run the example again and this time notice that the output is arranged from 0 till 199. This time the AsOrdered extension forces the use of a buffer where all work units outputs are gathered and arranged before being flushed back to the final output. The below image shows the process:

Example5: Data Races

In Example2 I mentioned that Data Races can occur when doing multi-threading programming. This example shows a Data Race scenario and how to solve it.

static void Main(string[] args)
        {
            Stopwatch watch = new Stopwatch();
            watch.Start();
            Parallel.For(0, 100000, i =>
            {

                Thread.Sleep(1);

                counter++;
            });
            watch.Stop();
            Console.WriteLine("Seconds Elapsed: " + watch.Elapsed.Seconds);
            Console.WriteLine(counter.ToString());
            Console.ReadKey();
        }

The above code again uses the Parallel.For method to execute a simple Sleep statement. A counter gets incremented each time the loop body gets executed.

What number do you expect to see if you run the program on a multi-core machine? 100000 is the logical result. Now run the program and you will notice that the number is less than that (if you do get 100000, you are just being lucky so try again:)). So what happened here? Why did we get this wrong when it seemed to be so obvious? The answer is data race occurred.

Let’s explain this. The trick here is in understanding how statement counter++ gets executed. When at Intermediate Language (IL) level, counter++ is not a single statement anymore. Actually it gets executed in 4 IL steps by the JIT (Just In-time Compiler):

Get the current value of counter
Get the value of 1
Add the two numbers together
Save the result back to counter

So what does this have to do with Parallel Programming and using Parallel.For? When executed over multiple cores as we do now, the body of the loop gets executed in parallel over multiple threads. The following image explains what happens in this case:

So as a result, one step increment was overlooked due to the fact that two threads were granted access to the variable counter at the same time. The above scenario occurs many times and you will get the final value wrong as we did when we ran the program. This is classical case of Data Race.

The solution of such case is to lock the variable so that once one thread is accessing that variable; some other thread has to wait until the lock is released before getting access to the variable. Granted this introduces time delay but it is a cost we have to pay for solving data races.

There are many ways we could use to lock a variable and a full discussion is outside the scope of this article, but I will use the best option in my case which is the Interlocked.Increment method as shown below (the “lock” statement is another method which is used in another example. The comments show how to use it instead):

static void Main(string[] args)
        {
            //Object thisLock = new Object(); //if not using Interlocked

            Stopwatch watch = new Stopwatch();
            watch.Start();
            Parallel.For(0, 100000, i =>
            {

                Thread.Sleep(1);

                #region L
                Interlocked.Increment(ref counter);
                //lock (thisLock) //lock is another method and is done using two lines
                //    counter++;
                #endregion
              
                //counter++;
            });
            watch.Stop();
            Console.WriteLine("Seconds Elapsed: " + watch.Elapsed.Seconds);
            Console.WriteLine(counter.ToString());
            Console.ReadKey();

        }

Example6: Debugging

Note: this example is built by Microsoft.

static void Main(string[] args)
        {
            var primes =
                from n in Enumerable.Range(1, 10000000)
                .AsParallel()
                .AsOrdered()
                .WithMergeOptions(ParallelMergeOptions.NotBuffered)
                where IsPrime(n)
                select n;
            foreach (var prime in primes)
                Console.Write(prime + ", ");

        }

        public static bool IsPrime(int numberToTest)
        {
            if (numberToTest == 2) return true;
            if (numberToTest < 2  (numberToTest & 1) == 0) return false;
            int upperBound = (int)Math.Sqrt(numberToTest);
            for (int i = 3; i < upperBound; i += 2)
            {
                if ((numberToTest % i) == 0) return false;
            }
            // It’s prime!
            return true;
        }

The above code uses PLINQ to print all prime numbers of a given range. There is nothing special about the parallel code itself, except that the IsPrime function has a logical error deliberately thrown inside it: on running the program you will notice that numbers 9, 15, and 25 are reported as prime numbers which is wrong. Now say you want to debug the IsPrime function and set a breakpoint inside it. Run the program and you will quickly notice the problem here: because multiple threads are executing, the debugging behavior is unclear as focus will switch from one thread to another as threads take turn hitting the break point.

To solve this issue Microsoft gave us the Parallel Tasks window (Debug Menu -> Windows -> Parallel Tasks):

Through this window you can see all running threads in debug mode. You can select to freeze all threads except the one that you want to actually debug and once done you can run other threads.

Another useful window is the Parallel Stacks window (Debug Menu  Windows  Parallel Stacks):

Through this window you can see a graphical representation of the running threads of your program and how they originated.

Example7: Deadlocks

Note: this example is built by Microsoft.

Example5 explained Data Races and how to deal with them through locks. However, you should take care that locking can bite you in the ugly form of Deadlocks. Simply put deadlock is a case where thread TA is locking resource R1 and thread TB is locking resource R2; now TA needs R2 to continue while TB needs R1 to continue. Both threads are stuck and cannot continue thus causing a deadlock.

Let’s see an example of such deadlock.

static void Main(string[] args)
        {
            int transfersCompleted = 0;
            WatchDeadlock.BreakIfRepeats(() => transfersCompleted, 500);
            BankAccount a = new BankAccount { Balance = 1000 };
            BankAccount b = new BankAccount { Balance = 1000 };
            while (true)
            {
                Parallel.Invoke(
                () => Transfer(a, b, 100),
                () => Transfer(b, a, 100));
                transfersCompleted += 2;
            }

        }

        class BankAccount { public int Balance; }
        static void Transfer(BankAccount one, BankAccount two, int amount)
        {
            lock (one)
            {
                lock (two)
                {
                    one.Balance -= amount;
                    two.Balance += amount;
                }
            }
        }

The above code uses Parallel.Invoke to concurrently perform money transfer between bank accounts “a” to “b” and vice versa. Inside the Transfer method, Locks are applied on both objects (using “lock” statement) with the intention of making it thread-safe for concurrent execution. This method of locking will cause a deadlock scenario.

The “WatchDeadlock” class is used to force break using Debugger.Break if it notices that no new transfers have been completed after a certain amount of time.

Run the program and see how now we have deadlock between our hands.

Let’s see how VS 2010 debugging support helps us identify such deadlocks. With the program running, fire the Parallel Tasks window. The window will clearly show you a case of deadlock. Not only that, it will specifically tell you what are the two threads causing the lock and how each thread is waiting a resource (in this case a Bank object) owned by the other thread:

So how do you avoid deadlocks? The answer is: simply do not do them. When doing parallel programming and locks, always program in a way that deadlocks will not occur!

Takeouts

It’s obvious that the new Parallel Programming in .NET 4.0 solves a lot of the multi-threading complexities. It grants you more control over threads in the form of tasks, it gives you many method overloads for performing parallel operations like For, Foreach, and Invoke, it gives you the power of PLINQ, it does take care of partitioning for you, and it introduces new Thread-Safe collections in .NET 4.0 such as ConcurrentDictionary, ConcurrentQueue, and ConcurrentStack. In addition, improved debugging support makes it easier to handle parallel related problems. However, you still have to take care of multi-threading related concepts such as Data Races and Deadlocks.

The better news is that for Microsoft, Parallel Programming is still an ongoing project so we can expect more features and improved support in the next releases.

MVP is here!

2010-04-02T00:41:00.000+03:00

It's been a while since I've blogged due to some mad things at work and personal life; but this is really worthy :)
I have been awarded with the 2010 Microsoft® MVP Award...that's it for now until I activate the award and access the MVP site for the first time; this will be fun :)

A Common Question: Concurrency vs Parallelism

2009-12-23T10:13:00.001+03:00

While presenting a .NET 4.0 session in GDC, i came across Parallel Computing. I allocated about 15 minutes to this topic since the session was a lap around .NET 4.0 features...
One of the questions I received and seemed to cause much of the confusion was the traditional one: What is the difference between Concurrency and Parallelism.

Well both concepts are related but not the same: In its simplest definition, Concurrency is about executing multiple un-related tasks in a concurrent mode. What these un-related tasks share are system resources, but they do not share a common problem to solve; meaning, they are logically independent. Now think of these tasks as threads. And since these threads share system resources, then you face the globally common problems of concurrent programming such as deadlocks and data races. This makes concurrent programming extremely difficult and debugging even more tendous.

So what is Parallelim? Parallelism is taking a certain task and dividing it into a set of realted tasks to be executed concurrently. Sweet! So again thinking of these tasks as threads, Parallel Programming still has the same problems of concurrent programming (deadlocks, data races, etc..) and introduces new challenges most notibly sharing and partitioning data across these related threads. This makes Parallel Programming even more difficult and debugging even more tendous :(

However, it is not all bad news. .NET 4.0 parallel programming is a great step onwards. The new API solves a lot of problems (but not all) of Parallel Programming and greatly eases up Parallel Debugging...

I will post more about this great feature, but for now I hope the idea is clarified.

GDC 09 Materials Available for download

2009-12-23T10:09:00.000+03:00

As I mentioned before, GDC 09 is over. Audiences requested the materials, and we love our audiences :)
so go to www.devlifestyle.net to download the presentations and demos
also there are a lot of vidoes uploaded there

GDC 09 is over now!

2009-12-20T05:32:00.000+03:00

The Gulf Developers Conference is over now, it was a great experience. I met some really nice people who also happen to be very experts, and this could not hurt :)

I gave 5 sessions during the GDC:

1. Cloud Computing Key Note; that was a last addition. I showed a simple demo about creating and deploying an ASP.NET web role on windows Azure

2. Cloud Computing Chalk-talk. It was a detailed discussion about cloud computing in general and i showed detailed demos about Windows Azure, SQL Azure, .NET Services, and Live Services. This was an amazing session with people eager to discuss and share their ideas. Time passed by very quickly.

3. A Lap Around .NET 4.0. This was a very packed session. A lot of features to be explained in 75 minutes. I showed 1-Click web deployment, Ajax 4.0, Parallel Computing (my personal favorite), Entity Framework, Data Services (REST is sooo cool!), CLR Type Equivelance, and Dynamic Data. My head was spinning towards the end!

4. Building SOA Applications with WCF and WF 4.0. Focused mainly on WCF advantages and why to use it. Then I created an example of a SOA app based on WF 4.0 Services (that's also covering WCF) and Data Services.

5. A comparison of Integration Technologies Chalk-talk. The main focus was BizTalk Server due to the audiences requests, but i also discussed WF-Services and the new Windows Server AppFabric. This was a cool session as somewhere during the middle of the session, 6 experts were suddenly added as guests to the session and I had to manage and give each one a role to perform...wondering why? MBC tv channel came to shoot and they wanted to create the ultimate Chalk-talk environment :)

Anyway, most of the materials are posted on DevLifestyle.

I will post about all these technologies in the near future...so stay tuned

Azure Service Bus Lab: ContractFilter Mismatch Error

2009-12-08T15:41:00.000+03:00

If you are following the Windows Azure (November CTP) SDK, and you are implementing the .NET Service Bus (AppFabric) Hands on lab called "Introduction to the .NET Service Bus", then there is an error that you need to correct.

In task 4 titled "Task 4 – Creating the WCF Client Project ", the lab says that you create an interface IEchoContract and apply to it the following attribute:

[ServiceContract(Name = "EchoContract", Namespace = "http://samples.microsoft.com/ServiceModel/Relay/")]

Well the Name is wrong. if you have followed the lab manual precisely, then the Name should be "IEchoContract" instead.

If you run the sample without correcting the issue, you will be faced with the following exception:

The message with Action 'http://samples.microsoft.com/ServiceModel/Relay/' cannot be processed at the receiver, due to a ContractFilter mismatch at the EndpointDispatcher. This may be because of either a contract mismatch (mismatched Actions between sender and receiver) or a binding/security mismatch between the sender and the receiver. Check that sender and receiver have the same contract and the same binding (including security requirements, e.g. Message, Transport, None)."

This problem can also appear if the namespace is different between the client and the service, so make sure to use the same namespace

SQL Azure Object Browser Support in SQL Server 2008

2009-12-01T17:10:00.000+03:00

If you are playing with the Azure training kit (November update) and you come across the lab about setting up SQL Azure, then you need to notice one thing:

the lab shows you how to connect to the cloud database using SQL Server 2008 R2 and thus you will get the Object Browser support.

However, the object browser is not supported in SQL Server 2008 pre-R2. So if you follow the steps without having R2 you will end up with an error message saying:
"Invalid Object name sys.configurations".
The reason is that the Management Studio cannot load the Object Browser from SQL Azure simply because it is not supported yet (again for pre-2008 R2 releases).

So the only way to connect to your SQL Azure database is through the "New Query" link in the Management Studio. This will give you full connectivity but without the Object Browser, so you will have to rely on your DDL skills to manage your db objects.

Activating Azure Storage Service for Non SQL Express

2009-12-01T12:19:00.001+03:00

I recently installed the November release of the Azure SDK.

I created a sample Web Role application and tried to run it, but I got the below error:

"Windows Azure Tools: Failed to initialize the Development Storage service. Unable to start Development Storage. Failed to start Development Storage: the SQL Server instance ‘localhost\SQLExpress’ could not be found. Please configure the SQL Server instance for Development Storage using the ‘DSInit’ utility in the Windows Azure SDK."

Under default configurations, the SDK will point your storage service into SQL Server express instance.
Well in my case I did not have Express, instead I had SQL Server 2008 Dev Edition.

So to resolve this, go to the "Windows Azure SDK Command Prompt" and run the following command:
DSInit /sqlInstance:. /forceCreate

This will create your storage database under the default instance. Of course you will have to supply the named instance in case you have one. For example:

DSInit /sqlInstance:MyNamedInstance

PDC 2009 Videos Released

2009-11-21T13:48:00.000+03:00

The Professional Developers Conference (PDC) 2009 was recently held in LA between 17 and 19 November.
PDC is the biggest Microsoft developer event where usually new technology releases are discussed.
Now the videos for the sessions are released, and as usual some really great stuff to learn.
You can download the videos here:

http://microsoftpdc.com/Videos

ASP.NET Ajax 4.0 takes the page on a diet

2009-11-14T11:45:00.000+03:00

I was playing around with the VS 2010 Beta 2 training kit and came across the new ASP.NET Ajax 4.0. A lot of interesting stuff there, but a killer feature is how Ajax now gives you the ability to minimize the request and response size.

See prior to 4.0, Ajax most famous control was the UpdatePanel. Granted, that is a great control and used heavily; it still lacked the true heart and soul of pure client-side Ajax. UpdatePanel kept both data processing and HTML rendering at the server. This caused that each page post was actually a full post with a full page life cycle and the viewstate rendered was still fairly large.

Now in 4.0, Ajax improves on the idea of client-side templates. That is moving the responsibility of HTML generation to the client and thus giving the server only the mission of data processing. This results in smaller request size and smaller viewstate in the response.

If you already have the 2010 beta 2 training kit installed, you will find the following solution:

[installation path]\Demos\AspNetAjax10in1\Source\AjaxApplication.sln

This is a really great source to start learning about Ajax 4.0, but I want to show you how the page size is reduced:

The page “1_ServerSideAjax.aspx” is a traditional Ajax page. It uses the UpdatePanel to partially load a set of data based on an item selection of a dropdown list. Run this page and change the item selected as follows:

Great, now view the source of the page and take note of the view state:

That is a lot of viewstate!

Now similarly run page “2_ClientSideAjax.aspx” and select the same item. View the source and take a note of the viewstate:

See how much the viewstate is reduced! This is amazing. The difference is that in Ajax 4.0 you can define client templates and use new client controls like the DataView control (not to be confused with the ADO.NET DataView). Now the request to the server is issued to a web service which returns result in the Javascript Object Notation (JSON) format. The result data is then de-serialized and bound to the client control. This idea of keeping the rendering responsibility on the client while getting only the data from the server, greatly reduces both request and response viewstate size.

Let’s have a look at how this is implemented. Open the “2_ClientSideAjax.aspx” page and notice the following line:

movieView = $create(Sys.UI.DataView, {}, {}, {}, $get("movieBody"));

The $create function of the Ajax Sys library created a new client control; this control is of type Sys.UI.DataView. Also the $get function acts as a shortcut for the getElementById function to get the movieBody template which is defined in the page as follows:

<tbody id="movieBody" class="sys-template">
<tr>
<td>{{ Title }}</td>
<td>{{ Director }}</td>
<td>{{ DateReleased.localeFormat("D") }}</td>
</tr>
</tbody>

“Title”, “Director”, and “DateReleased” are binding fields which will be returned by the JSON response of the web service. The below line calls the web service:

AjaxApplication.Services.MovieService.GetMovies(categoryId, showMoviesComplete);

If you are like me and would want to examine things behind the scenes, open your favorite network monitoring tool – such as Fiddler and examine the traffic. You will see for example that the request and response for the page using the UpdatePanel:

Now examine the request and response for the page using Ajax 4.0:

Notice any difference!!

CLR4 No-PIA

2009-11-10T07:03:00.000+03:00

One of the .NET 4.0 (actually CLR4) enhancements is a feature called “No-PIA” which greatly enhances COM interoperability. This fits in a larger picture in CLR4 which is Type Equivalence.
First let’s review what PIA is…

Primary Interop Assemblies (PIAs)

MSDN best described PIA as follows:

“Like any other managed assembly, an interop assembly is a collection of types that are deployed, versioned, and configured as a single unit. However, unlike other managed assemblies, an interop assembly contains type definitions (not implementation) of types that have already been defined in COM. These type definitions allow managed applications to bind to the COM types at compile time and provide information to the common language runtime about how the types should be marshaled at run time.
While any number of interop assemblies may exist that describe a given COM type, only one interop assembly is labeled the PIA. The PIA contains the official description of the types as defined by the publisher of those types. The PIA may contain certain customizations that make the types easier to use from managed code. The PIA is always signed by the publisher of the original COM type.
The Tlbimp (Type Library Importer) tool provided in the Microsoft® .NET Framework SDK is capable of creating an interop assembly from an existing type library.”

Importance of PIA

Again best explained via MSDN:

“Not using a PIA will lead to type incompatibilities for customers. For example, Company A uses the COM Widget object provided by the Acme Company within its managed application. Company A decides to produce another interop assembly with definitions of Acme's Widget library rather than use the PIA provided by the Acme Company.
Company B obtains the PIA for the Widget Library directly from the Acme Company and uses it within its managed application. Both applications work fine on their own until a developer from one of Company A and Company B's common customers calls to find out why he keeps getting a type mismatch whenever he tries to pass a Widget from Company A to Company B.
The reason for the type mismatch is because the Widget type used by Company A is completely different from the Widget type used by Company B. Even though they both have the same name, each interop assembly was signed by a different publisher, thereby making the types incompatible. The simple solution is for both companies to use the PIA provided by the Acme Company.”

Umm…so why the hell are you bragging about no-PIA?!

Actually no-PIA does not mean that PIAs are not there anymore. It simply means that you do not have to carry them with you anymore! Ok so let’s elaborate…

Say you have a .NET 3.x application which “talks” to Microsoft Excel. When deploying your application on client machine A you have to include the PIA of Excel in your deployment. Why? Simply because Microsoft Office does not require the installation of .NET framework (it is a COM application right?), so it is very likely that by the time Office was installed on machine A, the Excel PIA was not installed with it due to the fact that .NET framework is missing. PIAs are managed assemblies that live in the GAC so they need .NET framework.

So now when you want to deploy your application, you will want to include the PIA with it.

What is so bad about deploying PIAs with your application you might say? It’s size. Office PIAs for example are multi MBs of size which you will have to deploy when your application is probably few KBs of size. This extra size is an overhead especially when you know you are using a very small number of interfaces available in that PIA and you do not need to deploy and load the whole thing.

No-PIA to the rescue

Now in .NET 4.0 and CLR4 (or CLR3 if you keep counting from the previous version CLR2), you can instruct your assembly to include inside it only the needed types of the PIA and thus the PIA won’t have to be deployed along with your application.

Let’s see an example.

Below is a simple VS 2010/C# 4.0 (currently beta2) console application code which simply creates an Excel document.

class Program
{
static void Main(string[] args)
{
var x1 = new Excel.Application();

x1.Workbooks.Add();
x1.Visible = true;
x1.get_Range("A1").Value2 = "Name";
x1.get_Range("B1").Value2 = "Gender";
x1.get_Range("C1").Value2 = "Age";

x1.get_Range("A2").Value2 = "Mohd";
x1.get_Range("B2").Value2 = "Male";
x1.get_Range("C2").Value2 = "27";

x1.get_Range("A3").Value2 = "Sara";
x1.get_Range("B3").Value2 = "Female";
x1.get_Range("C3").Value2 = "25";
x1.get_Range("A3").Select();

((Excel.Range)x1.Columns[1]).AutoFit();
((Excel.Range)x1.Columns[2]).AutoFit();
((Excel.Range)x1.Columns[3]).AutoFit();
}
}

The below screen shot shows this:

The Excel PIA is added as a reference for the project. It is through this PIA that the managed code interacts with Excel. The PIA contains all the managed equivalent of interfaces, structs, enums, delegates, etc…defined in the COM with the required marshalling data…see the below image:

Now if you use .NET reflector, you will find that our managed assembly (ExcelAddIn.exe) actually has the Excel PIA as a reference; which is predictable of course. So in this case when we deploy the application we have to deploy the PIA with it. See the below screen shot:

However, what you can do is open the VS2010 properties box of the referenced PIA and change the property “Embed Interop Types” to true. This will instruct the compiler to embed the portions (types) of the PIA that your application use inside the application’s assembly itself. This way the PIA is not needed anymore and your application is using the required types implicitly. Check the below image:

As we will see in more details shortly, CLR4 will make sure that the types embedded inside the application’s assembly are equivalent with the types defined in the PIA; this is known as CLR4 “Type Equivalence”.

Ok, build your application. Refresh .NET reflector and now you can see that the PIA is no longer a reference. Instead the types used by the application are now embedded as namespaces inside the assembly itself. This way the application is not depending on the PIA anymore; hence no-PIA! See the below screen shot:

CLR4 Type Equivalence

You have seen how VS 2010 enables you to change a property and the compiler will embed the needed types inside the assembly. However, let’s dive into a little more details.

Locate the Excel PIA physical path on the machine and add it to the .NET reflector. Now you have the applications assembly (with no-PIA) and the PIA itself both loaded inside the reflector. Below is the view:

Now expand the Microsoft.Office.Interop.Excel namespace inside ExcelAddIn.exe and disassemble interface _Application as shown below:

Notice the new 4.0 keyword: TypeIdentifier. This tells the CLR that this type will participate in type equivalency. But how is this type equivalency actually happens? It is through the Guid. This Guid will be matched up by the CLR with the PIA type equivalent Guid such that when the application passes the interop type to the native code (Excel), this certifies that the application is using the same version of the type expected by Excel.

To witness this, expand the dll Microsoft.Office.Interop.Excel and again disassemble interface _Application. You will notice that it has the same Guid of the type embedded inside the assembly. This is how the CLR did the type equivalence matching:

Of course it should be clear by now, but still worth mentioning: notice how the PIA has tons of types defined but only the ones needed by the application have been imported to the assembly.

VS 2010 1-Click Publishing

2009-11-06T17:51:00.000+03:00

Web applications deployment has always been a tricky task for developers. Developers used to have many ways to perform deployment ranging from X-Copy to FTP to using installers. However, all deployment options have common problems; these are:

The solution has to be separately packaged. The database is just another package.

After generating the packages, these should go “somehow” to the destination server and the deployment will happen on that server.

On the destination server, IIS deployment for the web app and database deployment are two different tasks.

On the destination server, configuration files have to be edited to reflect on the new environment such as database connection strings.

These problems have just surfaced the difficulties usually associated with deploying large APS.NET applications.

Well in VS 2010 (currently in beta2), there is a huge feature: this is 1-Click Deployment (or publishing). It basically allows the developer to deploy the application along with the database and configuration changes directly from his development box to the destination server with one click!

The technology behind 1-Click Deployment is MSDeploy. VS 2010 uses MSDeploy implicitly to implement the deployment.

In the source server (development box), VS 2010 uses MSDeploy to create a zipped package of your project (containing web app, database with data, and deployment configurations). This package is then sent to the destination server which will have MSDeploy configured to listen to deployment requests and will perform the actual deployment of you web app and your database! Note that MSDeploy is installed with VS 2010 by default.

Well in this article I will show you two walkthroughs on how this is implemented; during which I will try to highlight how stuff is happening…

Deploying on a Hosting Company

When deploying web apps, you usually have two options; you either deploy on a web hosting company or on a server which you own. First let’s discuss option 1.
ORCSWeb (https://www.orcsweb.com/signup/?planid=58) allows you to create free testing accounts to deploy VS 2010 Beta 2 web projects. I have created an account and during the registration process I have selected SQL Server 2008 as my database; because then they will allow you to deploy your app along with the database…

After completing registration, I received an email containing the information I need to deploy. Lets us review the information:

The information that you will need to enter into the Visual Studio "Publish Web" wizard is:
Publish Method: MSDeploy Publish
Service URL: https://SHARED18.orcsweb.com:8172/msdeploy.axd
Site/Application: ITD1.orcsweb (you can optionally append a subfolder name)
Allow Untrusted Certificate needs to be checked.
User Name: itd1
Password: xxxxxx

You can view your site here: http://206.72.116.231/

Additionally, you can deploy a database by going to
the properties of the project and then the Deploy-SQL tab.

Your database connection string is:
Password=xxxxx;User ID=ITD1.orcsweb;Initial Catalog=ITD1.orcsweb;
Data Source=sql08-2.orcsweb.com

The new thing to you might be the “Service URL”. This is the IIS handler which will perform the actual deployment of the application on the hosting server. See in the destination server assigned to me by ORCSWeb, MSDeploy is running and configured to listen to requests on the specified handler to perform the actual deployment. This detail is hidden from you in this case and is done by the destination server admin (ORCSWeb in this case). In the next walkthrough I will show you how to set up your own destination server to listen to requests from VS 2010…

MSDeploy also comes with a SQL handler to perform database deployment. This will also be called implicitly on the destination server.
Going back to the information, I have been supplied by a username/password combination to submit while deploying and with a SQL connection string for my database deployment.

So now let’s see my web app I want to deploy. In VS 2010 I have created the simplest of applications: I used the default web app template and added a second page which reads a list of “Departments” from a SQL Server 2008 table.
The below screen shots shows my simple app:

So now it is time to deploy! Let’s review the three problems that I want to solve for deployment:

I actually want to deploy the application without manual packaging and without leaving my dev box

I want to deploy the database along with my app in a single click!

I do not want to go to the destination server and change any configuration (like database connection string)

To solve the configuration issue, VS 2010 introduces XML Document Transforms (XDT) for automatic configuration changes. You can learn about the complete syntax of XDT from various online resources; however, for the sake of our discussion what you need to know is that VS 2010 allows you to create multiple versions of your web.config and associate each one with a configuration build (for example your application will have the normal web.config file, but it will also have web.config.debug and web.config.release for each of the debug and release configuration builds…you can have more builds as you like; for example web.config.staging…).

Now let’s say that you are deploying your release version; XDT allows you to set a certain syntax in these config files such as while VS 2010 is performing the packaging of your project, the entries you specify in web.config.release replace the entries in web.config and a new final web.config file is created containing the updated configuration…so for example your web.config will hold the connection string for your local database. In the web.config.release you specify the connection string for the production database, and through XDT you instruct VS 2010 to dynamically replace the connection string in the web.config file with the connection string in the web.config.release when generating the production version of web.config which will be in the final package…

Ok XDT wanted some explanation so now let’s start the actual packaging and deployment.

If you expand the web.config node, you will find that by default there will be web.config.debug and web.config.release. As explained, these correspond to the default build configurations (Debug and Release) respectively. In our case we are going to be deploying the release version, so go ahead and edit the web.config.release file to read as follows:

Here we are using XDT (xdt:Transform=”Replace”) to instruct VS 2010 to replace the development-time connection string available in web.config with the release (or production) time connection string available in the web.config.release. This is the connection string supplied by the hosting company. The final web.config file in the final package will contain the updated connection string.

Now let’s set up VS to deploy our SQL database also…this solves the concern of deploying the database along with the application in a single click. Open the properties window of the project. You will find a new tab called Deploy SQL. In this tab set up the connection string of the development database which you want to deploy (you can make VS 2010 import this from the web.config file by clicking “Import from Web.config”) and then provide the destination URL where the database will be deployed. This will be the connection string provided to you by the hosting company. Instruct VS 2010 to include data while deploying and make it get schema and data…the below screen shot shows the configuration I used to deploy my application:

Ok so now you are ready to do the actual deployment. First make sure that the “Release” build configuration for your project is selected as shown below.

Now right-click the project node and select the new “Publish…” option. A new window will open where you will enter the publish settings:

Publish Profile: you are allowed to create a profile for your settings so that you are not required to type them again and again for each deployment. For example I have created two profiles; one to deploy to the hosting company – which I am using now, and another one to deploy to another server which I will show you next.

Publish Method: Select MSDeploy Publish to perform the publishing based on MSDeploy

Service URL: recall that this is the URL provided to us by the hosting company which contains the IIS handler

Site/Application: the name of the IIS web site to deploy to – also provided by the hosting company. And the name of the application (virtual directory) under which to deploy.

Allow Untrusted Certificate: select this since we are deploying on a demo server and the certificate won’t probably be from a trusted CA (like VeriSign)

Username/password: the account provided by the hosting company which will have permission to call MSDeploy and perform the deployment.

Ok, so now we are ready. Click publish and your web app and the database (with the data) will be deployed to the hosting company.

Behind the scenes VS 2010 (and MSDeploy) created a packaged zip file of your application along with a manifest xml file and used these two files to publish your application to the destination server. The package file can be found by default on “obj\Release\Package\”; although you can change this in the “Package/Publish” tab of the properties window of your project.

Actually, if by the time you are reading this article the testing account is still valid and the application is still deployed then you can check the application on the following URL:

http://206.72.116.231/oneclickwebapp/Default.aspx

Deploying on another Server

Now let’s discuss the case when you own your production server and want to deploy on it. In this case the settings on the development box will be almost the same. However, now you have the responsibility of configuring MSDeploy on the production server to listen for deployment requests.

In my case, I will deploy the same application I showed you on the same machine on IIS 7 on win server 2008. So the first and most important mission I have is to configure MSDeploy on my machine (acting as server) to listen for deployment requests.

First make sure IIS 7 is installed on the server.

Now you have to download the MSDeploy package. You can get it from here:

http://www.microsoft.com/downloadS/details.aspx?familyid=32A781A2-4961-49FC-B34D-170BFA78414F&displaylang=en

Run this package on the server and install the following components:

The above components install MSDeploy along with its management UI and the IIS 7 handler for managing deployment requests. The management service will be the actual service listening for requests. The Remote Agent Service is another MSDeploy option used mainly for command-prompt deployment but is not required for VS 2010 1-Click Publishing.

Now make sure that services “Web Deployment Agent Service” and “Web Management Service” are started and have a Startup type of “Automatic”.

The final step is to enable remote connections for IIS (intermgr.exe --> Management Service --> Enable Remote Connections).

Now you have the server ready to handle VS 2010 publish requests.
Back to our project in VS 2010. Edit web.config.release to have the connection string of the server instead of the hosting company (in my case since it is the same server, I will have the same connection string as the one in the web.config file).
Now on the Deploy SQL tab, also edit the connection string to point to the production server (again same server for me…) and select to move data and schema. Now you are ready to publish, select the “Publish…” tab and enter the below settings:

Notice that I have created a second profile for publishing. The settings are similar to the ones I used for hosting company’s profile; just note that in the Service URL I just supplied the name of the server to deploy to. This is the server I have configured MSDeploy service on; in this case my same dev machine…

Summary

With all the great features coming in VS 2010 and .NET 4.0, 1-Click Publishing is one of the most important in my opinion. The ease of use and simplicity really stands out. The fact that you now can deploy your database and your configuration changes along with your application in one zipped file is great! Add to this that you can do this without even leaving your development machine is really cool…

VS 2010 Beta 2 Released - No More VSTS

2009-10-26T18:23:00.000+03:00

Beta 2 of VS 2010 is released. This is supposed to be the final release before the official launching in March 2010.

Visual Studio Team System (VSTS) does not exist anymore as a terminology in VS 2010. In the 2008 version, different Team Editions where available:
- VSTS Team Suite
- VSTS Developer Edition
- VSTS Test Edition
- VSTS Architecture Edition
- VSTS Database Edition
In addition to that you had the VS 2010 Professional Edition which contained all the basic VS features without any Team features.

Well, that was a lot of naming and caused a lot of confusion when it came to licensing.
MS has made a decision to ship 3 editions starting VS 2010 and no more is in use the term VSTS. These editions are:
- VS Ultimate
- VS Premium
- VS Professional
All previous VSTS 2008 team editions (plus more) features are still there but under the 3 above versions.
A full feature comparison can be found here

Passing Large Messages in WCF: maxItemsInObjectGraph, maxReceivedMessageSize, and maxBufferSize

2009-10-24T09:12:00.000+03:00

in WCF it is known that you can pass messages up to multi GBs between a client and a service. however, there are certain configurations that you need to tweak.

in one of my projects i had a SQL table with around 100,000 records. i had a service which simply reads all the records and pass them to a client.
i decided to use the netTcpBinding for binary encoding which will enhance performance..of course i had .NET-based apps on both ends.

i started out with the default configurations and i immediatley got an error that the max message quota limit was exceeded. so i set both
- maxReceivedMessageSize = 2147483647
- maxBufferSize = 2147483647

however, it was interesting that with the above confiruation, although i have set up the allowed message and buffered size to more than 2 GB; i could not pass more than around 8000 records with message size of 3 MB! i was buffled for a second because i have granted the communication channel all the size limit it needs...

i then turned on message logging and tracing, and every time i tried to pass more than 8000 records (again 3 MB) i got the following error:
"the socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue"

searching around i found on MSDN a property of DataContractSerializer called maxItemsInObjectGraph. after i set this property for a high value, then i could pass the full 100,000 records; that's around 50 MB.

see, in my example i obviously was using the DataContractSerializer to serialize data on the wire, but more interesting i was filling out each of my SQL records into a seperate .NET object. so at then end i was trying to pass 100,000 objects from the service to the client. the maxItemsInObjectGraph, sets the max allowed objects to be passed around so setting this property to a high value solved the problem.

the relevant part of the service configuration was:

<servicebehaviors>
<behavior name="ServiceBehavior">
<datacontractserializer maxitemsinobjectgraph="2000000">
</behavior>

the relevant part of the client configuration was:

<behaviors>
<endpointbehaviors>
<behavior name="ClientBehavior">
<datacontractserializer maxitemsinobjectgraph="10000000">
</behavior>
</endpointbehaviors>
</behaviors>

Introducing WF4.0: Building Distributed Apps with WF 4.0 and WF 4.0 Services

2009-07-04T07:50:00.000+03:00

In 2006, Microsoft released .NET Framework 3.0, which basically consisted of extensions to .NET Framework 2.0. WCF was the biggest hit back then and gained the most attention. That even continued with the 3.5 release of the framework.

With .NET Framework 4.0, Microsoft made WF the major focus, and with the many enhancements they have in place, it is evident the importance that WF 4.0 will have for building .NET-based applications.

Read Full Article Here

.NET 4.0 Code Contracts

2009-06-06T14:58:00.000+03:00

One of the great features of .NET 4.0 is Code Contracts. Code Contracts aid in the unit testing process of code and make catching bugs before deploying the code much easier.

This post shows a sneak preview of Code Contracts in .NET Beta1 and VS 2010 Beta1. Note that by the time of the current writing, you need to install an Add-On to visual studio 2010 in order to work with code contracts.

To do so, go to the following URL and download and install Code Contracts for .NET VSTS edition.
http://msdn.microsoft.com/en-us/devlabs/dd491992.aspx#2

once done, you will notice a new tab appearing in the Project Properties window. This tab is shown below:

Now to test how Code Contracts work, consider the below code

class Ctrct
{
public int Divide(int x, int y, ref int z)
{
Contract.Requires(y > 0);
Contract.Ensures(z > 1);

z = x / y;

return z;

}
}

This class has a method that simply divides two numbers. First notice the line
Contract.Requires(y > 0);
This line indicates that y should be greater than zero otherwise the code will fail. This check will happen before the execution of the method.

The second line is
Contract.Ensures(z > 1);
This line indicates that by the time the method ends, z should be greater than 1 else the code will fail.

Now test with the above code passing first y=0 and you will find that the method fails at the entry level. Now even more interesting, pass y=1 and x=1; in this case the code will execute and z will be calculated to 1. in this case after the return statement is executed, the method will still fail. that is because the post condition of z being greater than one is not met. execution won't be delivered to the caller even though the return statement is executed!

C# 4.0 Optional and Named Parameters

2009-06-01T18:16:00.000+03:00

C# 4.0 now supports the optional and named prameters. this feature was already there in VB.NET so it is a C# language enhancement.
consider the below class:

class Optional
{
public void DoOperation(string name = "hello", int age = 4)
{
Console.Write("Name {0}, Age {1}", name, age.ToString());
}
}

you can notice that now we can specify default values for parameters "name" and "age". so now when using the below code:

Optional optional = new Optional();
optional.DoOperation();

the default parameter values are passed to the method. and when we call the method as follows:

optional.DoOperation("mohamad");

then "name" will have its new value set, while "age" parameter will have its default value.

now the question is what if we want to call the method passing only the "age" parameter while giving the "name" parameter its default value? we cannot simpy do the following:

optional.DoOperation(2);

because this will give a compilation error since the compiler "understands" parameters by position and this means we are supplying an integer value to a string parameter. The solution is using Named Parameters. We can do the following:

optional.DoOperation(age:2);

by specifying the Parameter name we can ommit the first parameter and pass the second one only.

as you might have noticed, this feature will for sure result in much less method overloading that we used to do just to change the number of parameters of each overload!

C# 4.0 Dynamic

2009-06-01T16:46:00.000+03:00

one of the core functionalities of .NET 4.0 is its support for Dynamic languages. this is done primarly through the Dynamic Language Runtime (DLR) on top of the CLR. this post shows some of the usages of dynamic in C# 4.0.

consider the following class:
public class Sample
{
public static int Add(int x, int y)
{
return x + y;
}

public static float Add(float x, float y)
{
return x + y;
}

public static double Add(double x, double y)
{
return x + y;
}
}

this shows a very familier pattern where we use overloading to support multiple versions of the Add method. Now normally in order to invoke the many overloads of the Add method, you would write something like the below code:

int x = 3;
int y = 4;
int z = Sample.Add(x, y);

float x1 = 3f;
float y1 = 4f;
float z1 = Sample.Add(x1, y1);

long x2 = 3L;
long y2 = 4L;
long z2 = Sample.Add(x2, y2);

the problem with this pattern is that we had to define parameters with appropriate types for each method overload. Now in C# 4.0 we can write much more elegant code as follows:

dynamic x3 = 1.2;
dynamic y3 = 1.55;
dynamic z3 = Sample.Add(x3, y3);

using the dynamic keyword, it is left to the runtime to determine the overload of method we would like to call; in this case the decimal-typed overload. for example changing the values of x3 and y3 to integers would call the integer-typed overloads as follows:

dynamic x3 = 1;
dynamic y3 = 1;
dynamic z3 = Sample.Add(x3, y3);

Now lets take another - even more impressive - example. Consider the below class

public class Wrapper
{
public void DoAction1()
{
//any stuff
}
}

Normally, we do the following to invoke method DoAction1:

Wrapper wrapper = new Wrapper();
wrapper.DoAction1();

ok so now what if you do not know about the method at design time? the direct answer would be reflection. true, but that's not pretty at all. Using dynamic you can use the following code:

//instead of reflection
dynamic dynWrapper = new Wrapper();
dynamic val = dynWrapper.DoAction2();

by initializing the class to a variable of type dynamic, we can now call methods that will be verified at runtime only. for instance the above code will compile even though DoAction2 does not exist! of course this code will fail at runtime but the idea is that at design time you might not know about the methods of class dynWrapper but you know that at runtime the method will be available...

That's just a first look at C# 4.0 dynamic keyword.

Cool C# Code

2009-06-01T06:50:00.000+03:00

i was reviewing a piece of code of a colleague, and i came across a very familiar pattern of code. basically he had a list of employees where he wanted to "loop" over and exclude any employees with age over 30. so his first attempt was the most common one: he used a foreach loop to remove all employees with age > 30. the code looked like the following

foreach (Employee empl in lst)
{
if (empl.age > 30)
lst.Remove(empl);
}

now clearly this will compile but blow up at runtime since it is trying to modify a collection while enumerating it! the second attempt was to create a new list and add all employees with age > 30 to it...that would have worked but it is completely nonesense to do the days.

the first thing that came into my mind is Predicates. so istead what we can do is to define a method as a predicate as follows:

private static bool isAboveAge(Employee empl)
{
return empl.age > 30 ? true : false;
}

then simple call the FindAll method of the list; as follows:

List newLst = lst.FindAll(isAboveAge);

that's it; now it works in a much better way.

However, why stop there? the only problem with the above code is that i had to create a method (isAboveAge) just for it to being called by FindAll. so to solve this we can use an anonymous method as follows:

List newLst = lst.FindAll(
delegate(Employee empl)
{
return empl.age > 30 ? true : false;
}
);

very cool! but we can take this one more stop...lambda expressions. using lambda we can now - and finally - shrink our code to this:

List newLst = lst.FindAll(empl => empl.age > 30 ? true : false);

Nice!

BizTalk Business Features

2009-04-08T11:18:00.000+03:00

BizTalk Server is an integration platform for developers. By nature Biztalk solutions are very technichal and its nearly impossible to show the value for business users.

For that there are special BizTalk capabilities for business users to step in. These are:
- Business Activity Monitoring (BAM)
- Business Activity Services (BAS)
- Business Rules Engine (BRE)

BAM is defenitely the most important as it gives business users a detailed view on Key Performance Indicators (KPIs) of the solution. Standard reporting is possible but argubly the most interesting part is the out of the box integration with SSAS cubes.

BAS is also interesting in that it allows business users to identify trading partners and agreements with these partners to "customize" business processes according to these agreements.

BRE is somewhat sitting in the middle of developers and business users. BRE is a place for developers to identify dynamic rules instead of "hard coding" those. On the other hand, and through a special feature called Vocabulary, business users can access these rules at runtime and change the values of these rules. Processes will pick up the new values without any downtime...

Keep tuned for articles and posts for every one of these features. After all its business users who own the money and their satisfaction is a priority for us poor developers :)

Truly Understanding .NET Transactions and WCF Implementation

2009-04-08T11:15:00.000+03:00

Strictly speaking, implementing transactions in WCF is easy: you use attributes to control the service and operations behaviors and you are done!

just about like everything else in programming; you need to understand the architecture, the underground of whatever seems to be an easy task. Only then you can architect (design) solutions as opposed to only coding them…

WCF transactions are built from ground up around .NET transactions. This is hardly surprising anyway since WCF is a .NET technology; however, this sums up as follows: you need to understand how .NET does transactions in order to utilize them in WCF.

Read Full Article...

WCF Duplex Reentrant Services

2009-03-25T17:38:00.000+03:00

This article illustrates working with Duplex mode in WCF with concurrency set to Reentrant. In general, Duplex is one of the three message exchange patterns (MEPs) which are:

One-way

Request-Response (synchronous and asynchronous)

Duplex

On the other hand, concurrency in WCF services deals with how many concurrent threads can access these services at one time...

Read full article here

ODBA Visio 2007 Hack

2009-03-24T09:05:00.000+03:00

the orchestration designer for business analyst (ODBA) works with Visio 2003 SP1. if you try to install it when having Visio 2007 you will get an error. however, there is a cool hack (see link at the end) to apply to allow installing the ODBA with Visio 2003.

If you want to use BizTalk ODBA with Visio 2007 instead of having to install Visio 2003, follow these steps:

Export this registry key:[HKLM\SOFTWARE\Microsoft\Office\12.0\Visio]

Edit the resulting .reg file in notepad and replace all instances of 12.0\Visio with 11.0\Visio

Reimport the file into the registry and the install for BizTalk ODBA should work.

Note: Do not change any other values in the file. Essentially, you will be copying the Visio key under the 12.0 key to the 11.0 key without making any value or data changes.

Click here for the original hack post

WCF Queued Messaging

2009-03-16T18:18:00.000+03:00

One common requirement for all MEPs is message durability; that is the ability to preserve the message under all conditions and make sure that a message will arrive where it is supposed to.

Microsoft Message Queuing service is a proven method to provide a queue for messages to reside inside until recipients are ready to pick them up. WCF offers implicit MSMQ integration through the NetMsmqBinding which is the topic of this article...

Read my full article

Developing with Microsoft Business Rule Engine

2009-03-13T15:49:00.000+03:00

Mainly this article is for two set of readers: first are those who are new to Microsoft Business Rule Engine and are looking for a direct hands-on approach rather than going though the theory and architecture behind BRE (links for that at the Additional Resources section).

Second are those who are familiar (but not experts) with the topic in general and are looking to understand in practical manner how to program with BRE and leverage some of its powerful offerings...

Read my article

.NET Delegates and Events

2009-03-13T15:44:00.000+03:00

Many developers tend to use events and delegates "as is" without truly understanding the internal stuff - which, trust me, once understood, can help you do many cool stuff. When I first started .NET coding, I used to just double click the "Button" or "DataGrid" control, go to the code-behind, and write my code inside the event handler. Well, then a few years ago, I decided enough is enough, and I dug deep to really understand .NET events and delegates: This article is an attempt to share with you what I came up with...

Read my full article

ASP.NET Viewstate and Page Life Cycle

2009-03-13T15:41:00.000+03:00

Viewstate and page life cycle are amazingly interesting topics to dig into. once you understand these you have a different perspective about ASP.NET development. My published article has won the article of the month competition at code project.
Enjoy!

Read my article

ASP.NET Request Architecture

2009-03-13T15:37:00.000+03:00

ASP.NET request architecture is a must know topic for any ASP.NET advanced developer or architect. my article below aims at explaining the topic inside out.

Read my article

SSO Security Configuration Error

2009-03-09T06:23:00.000+03:00

I was configuring BTS 2006 R2 on a machine. After I ran the configuration wizard I got an error saying that “login failed for user on SQL server. Cannot create database SSO”.

The logical resolution was to go to SQL Server and give the specified user the appropriate permissions; which I did. I re-ran the configuration again and again I got the same error.

Puzzled for a couple of minutes, I checked the event viewer and there it was: I found the exact error reason:

“Cannot create file 'C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\DATA\SSODB.mdf' because it already exists. Change the file path or the file name, and retry the operation.”

It turned out that I had a previous installation of the SSO database on the same machine, and it was not deleted automatically (for some reason which I cannot remember frankly). So I manually deleted the “mdf” and “ldf” files, re-ran the configuration, and this time it worked fine.

Moral of the story: some error messages can be misleading. Always check the event viewer when working with BizTalk Server because many times the real reason of a problem will only be found there.

Azure Invitation Codes Activation

2009-03-09T06:12:00.000+03:00

Invitation codes for Microsoft’s Azure services platform currently come in three flavors:
- Code for .NET Services and SQL Data Services
- Code for Live Services API and Live Framework CTP
- Code for Windows Azure (hosted services, deployments, and storage accounts)

Once you get your tokens you need to activate them (associate these with your Windows Live ID).

For the .NET Services and SQL Data Services, follow the following URL “http://go.microsoft.com/fwlink/?LinkID=129428”. Sign in using your token as shown below:

For the Windows Azure and Live Services tokens, follow the following URL “http://lx.azure.microsoft.com”. Redeem your token by associating it with your Live ID. Once done you can click on the “Account” tab to see the services that you get as a result of redeeming the two tokens. The following image shows what are these services (again note that the .NET Services and SQL Data Services are not included here as they have their own portal).