SOAP AND REST WEB SERVICES IN MICROSOFT DYNAMICS CRM

SOAP AND REST WEB SERVICES IN MICROSOFT DYNAMICS CRM 

REST

„REST is a software architecture style consisting of guidelines and best practices for creating scalable web services. REST is a coordinated set of constraints applied to the design of components in a distributed hypermedia system that can lead to a more performant and maintainable architecture.

REST has gained widespread acceptance across the Web as a simpler alternative to SOAP and WSDL-based Web services. RESTful systems typically, but not always, communicate over the Hypertext Transfer Protocol with the same HTTP verbs (GET, POST, PUT, DELETE, etc.) used by web browsers to retrieve web pages and send data to remote servers.” (from Wikipedia)

 SOAP

“SOAP, originally an acronym for Simple Object Access protocol, is a protocol specification for exchanging structured information in the implementation of web services in computer networks. It uses XML Information Set for its message format, and relies on other application layer protocols, most notably Hypertext Transfer Protocol (HTTP) or Simple Mail Transfer Protocol (SMTP), for message negotiation and transmission.” (from Wikipedia)

“Should I REST or Should I SOAP?”

I have spent hours fretting over the choice between SOAP and REST finding the answer.  Some Web services support one and some support the other. Microsoft Dynamics CRM supports both. Our decision depends on which web service best meets your NEEDS, rather than which PROTOCOL to USE!

SOAP is a heavyweight player: it provides the following advantages when we compare to REST:

Security:

SOAP supports SSL – just like REST – but also support WS-Security (Web Services Security is an extension to SOAP to apply security to Web Services. WS-Security supports identity through intermediaries, not just point to point. It implements data integrity and data privacy.

Transactions:

Need ACID (Atomicity, Consistency, Isolation, Durability) Transactions over a service? You need SOAP. However REST supports transactions, it is not as comprenesive and is not ACID compliant! REST has its own weakness: it is limited by HTTP itself: HTTP is not able to provide two -phase commit across distributed transactional resources, but SOAP can.

Reliability:

SOAP supports WS-ReliableMessaging which is a protocol that allows SOAP messages to be reliably delivered between distributed applications in the presence of software component, system, or network failures.

Let me show you some reasons why REST is almost the “right” answer:

REST uses standard HTTP: it means it is much simpler in some cases:

Creating clients and APIs is easier than SOAP.

Performance:

REST permits many different data formats: SOAP only permits XML. REST supports JSON which is a better fit for data because it parses much FASTER!. REST allows BETTER support for most of browser client.

REST has better performance and it is scalable: REST can be cached, SOAP can’t be.

Conclusion:

Even though REST seems to be in favor now I think there are some cases when SOAP makes sense when you choose it: if I was writing an smart phone application to interface with a service which provides confidental data I would definitely need to use SOAP. All features (WS-Security, WS-AtomicTransaction, WS-ReliableMessaging) are required for this kind of service.

On the next article I will compare REST and SOAP in Dynamics CRM developers’ perspective.

==========================================================

One of my first choices while starting my Silverlight development in Microsoft Dynamics CRM 2011 was to used SOAP or REST. What surprised me was the lack of a clear answer in the Microsoft SDK. All I could see was a fuzzy answer, something along the lines of  “Well REST is potentially OK for this… but SOAP can do that as well…”, and “SOAP is good for this…, but REST can actually achieve that as well…”. AHHHHHHHHH!!!!!!!! So I’ve given both a go and finally came to a conclusion.

This is only my opinion so please chip in with comments. Also I’m thinking more in terms of CRM integration and maintainability than actual REST and SOAP architecture… I believe that the best option is SOAP! Why? well lets look at what Microsoft say about REST and SOAP.

Here is their recommendation table .

The following table describes the appropriate web service to use depending on the task you need to perform.

Task	Web Service
Create, Retrieve, Update and Delete records.	REST Endpoint
Associate and Disassociate records	REST Endpoint
Assign Records	SOAP Endpoint
Retrieve Metadata	SOAP Endpoint
Execute Messages	SOAP Endpoint

Here is what they have to say about REST and SOAP

The REST endpoint provides a ‘RESTful’ web service using OData to provide a programming environment that is familiar to many developers. It is the recommended web service to use for tasks that involve creating, retrieving, updating and deleting records. However, in this release of Microsoft Dynamics CRM the capabilities of this Web service are limited to these actions. Future versions of Microsoft Dynamics CRM will enhance the capabilities of the REST endpoint.

The SOAP endpoint provides access to all the messages defined in the Organization service. However, only the types defined within the WSDL will be returned. There is no strong type support. While the SOAP endpoint is also capable of performing create, retrieve, update and delete operations, the REST endpoint provides a better developer experience. In this release of Microsoft Dynamics CRM the SOAP endpoint provides an alternative way to perform operations that the REST endpoint is not yet capable of.

So what this means is you should use REST if you want to perform CRUD operations or want your code to look prettier through LINQ utilisation. Lets look at that from a maintainability point of view. Lets say we have developed a lovely CRUD based Silverlight application and it works perfectly. However our client has now requested some new functionality that can only be achieved through an Execute Request.

Well now we have to implement the SOAP connection too, to perform the request!  All of a sudden my visual studio solution has grown a couple of classes and involves two services (SOAP and REST). Already thinking messy? I’d prefer to use just one service. What’s that… I can!!! Ah yes because you can perform CRUD operations with SOAP as well. So I might as well of used SOAP in the first place.

SOAP 1 – REST 0

Now apparently the other thing REST has going for it is that it provides a better developer experience. I assume this is because LINQ can  interpret it. Well that is great, however my client isn’t happy that its taking me longer than it should do to implement his new functionality. For some reason he doesn’t seem to understand why using LINQ is so great, but why I am having to now implement a different service until Microsoft increase the REST functionality.

SOAP 2 – REST 0

Microsoft also say that a downside to SOAP is that there is no strong type support. What they are referring to is early and late binding. In my opinion this is a blessing in disguise! Strong typing may look nice in code and allow easier data extraction, but it has one major downfall for me. When developing I try to keep code as generic as possible so that it can be reused.  You can achieve this through late binding better than early binding. Thought I haven’t looked in to it potentially you could achieve early binding in SOAP by using the crm svcutility tool to generate a strongly typed object class.

SOAP 3 – REST 0

Did I mention that you can achieve more through SOAP?

SOAP 4 – REST 0

So while we may argue away from CRM, REST may be better because its lightweight, provides human readable results, and is easy to integrate with. Or SOAP is better because its easier to consume, or is rigid and contains type checking. Or one is better than the offer because its faster or more secure etc… At the end of the day for me with regard to CRM SOAP wins 4 – 0
===========================================================

In the last two columns, I've described the basics of REST and talked about exposing and consuming Web feeds. In this column, I'll answer a number of questions that often come up when I make presentations or conduct training sessions on using REST to build service-based applications.

Which is better, REST or SOAP?

This is one of the most common questions I get about REST, and it is probably the least fair. Both REST and SOAP are often termed "Web services," and one is often used in place of the other, but they are totally different approaches. REST is an architectural style for building client-server applications. SOAP is a protocol specification for exchanging data between two endpoints.

Comparing REST with the remote procedure call (RPC) style of building client-server applications would be more accurate. RPC is a style (rather than a protocol, which is what SOAP is) of building client-server applications in which a proxy (generally generated from metadata) is used in the client's address space to communicate with the server and the proxy's interface mimics the server's interface. Although SOAP doesn't require the RPC style, most modern SOAP toolkits are geared toward (at least they default to) using RPC.

In contrast to RPC, REST lacks the metadata-generated proxy (see the next question for more information), which means that the client is less coupled to the service. Also, because REST relies on the semantics of HTTP, requests for data (GET requests) can be cached. RPC systems generally have no such infrastructure (and even when performing RPC using SOAP over HTTP, SOAP responses can't be cached because SOAP uses the HTTP POST verb, which is considered unsafe). SOAP intentionally eschews HTTP, specifically to allow SOAP to work over other protocols, so it's actually a little disingenuous to call SOAP-based services Web services.

My perspective is that both REST and SOAP can be used to implement similar functionality, but in general SOAP should be used when a particular feature of SOAP is needed, and the advantages of REST make it generally the best option otherwise.

What about security? Isn't SOAP more secure than REST?

This question touches one of my pet peeves because the answer is clearly no. It is just as easy to make a RESTful service secure as it is to make a SOAP-based service secure. In the majority of cases involving either REST or SOAP, the security system is the same: some form of HTTP-based authentication plus Secure Sockets Layer (SSL). Although technically the technology for secure conversations over HTTP is now called Transport Layer Security (TLS), SSL is still the name most commonly used.

What is true is that a SOAP-based service, because of the extra protocols specified in the various WS-* specifications, does support end-to-end message security. This means that if you pass SOAP messages from endpoint to endpoint to endpoint, over the same or different protocols, the message is secure. If your application needs this particular feature, SOAP plus WS-* is definitely the way to go. REST probably wouldn't be an option here because of its dependence on HTTP, and inherently you'd be designing a multiprotocol application. I believe that the fact that SOAP with WS-* enables end-to-end message-level security is the source of the misconception that SOAP-based services are more secure than RESTful services.

Another area in which the WS-* folks have spent a lot of time and effort recently is federated security. The simple idea behind federated identity is to create trust between two companies, where authenticated users from one company can be trusted and considered authenticated by another company without the second company having to maintain the authentication information (username and password, typically). The various WS-* specifications have implementations from all the major vendors, and Microsoft is integrating the ideas into Active Directory through Active Directory Federation Services (ADFS).

In the realm of federated security, the WS-* arena certainly has more standards than the RESTful arena (and this will probably always continue to be the case), but there are efforts to support federated security in the world of REST. OpenID is one such effort. The .NET Service Bus (part of Microsoft Azure) also contains a federated identity service, which works just as well with HTTP (and therefore REST) as it does with SOAP-based services.

What about transactions?

Here is another area in which SOAP and WS-* have explicit support for an "advanced" feature and REST has none. WS-Atomic Transactions supports distributed, two-phase commit transactional semantics over SOAP-based services. REST has no support for distributed transactions.

Generally speaking, if you want something like transactions in a RESTful system, you create a new resource. (Creating a new resource whenever you run into a problem with a RESTful system generally solves most problems.) You can have a resource called Transaction. When your client needs to do something transactional (such as transferring money between two bank accounts), the client creates a Transaction resource that specifies all the correct resources affected (in my example, the two bank accounts) by doing a POST to the Transaction factory URI. The client can then perform updates by sending a PUT to the transaction URI and close the transaction by sending a DELETE to the URI.

This, of course, requires some amount of hand-coding and explicit control over your system, whereas the WS-Atomic Transactions system is more automatic because (in the case of Windows Communication Foundation) it is tied to your runtime's plumbing.

If your system absolutely needs atomic transactional semantics across diverse systems, WS-Atomic Transactions is probably the way to go. Using distributed transactions in this way may or may not be smart because it increases the coupling between the two systems and creates potential problems if you aren't controlling the code on both ends. But the most important thing is to use the right tool for the right job (once you've figure out what the right job is).

In defense of REST, I think it is fair to say that given today's distributed, service-oriented architectures, coupling two endpoints so tightly using a distributed transaction may not be the best design. On the other hand, some situations call for this type of functionality, and if you need it, use SOAP and WS-Atomic Transactions.

What about interoperability? Isn't SOAP supposed to be about interoperability? Isn't SOAP more interoperable than REST?

If you define interoperability as the technical ability to communicate between two divergent endpoints, I assert that REST wins the interoperability battle hands down.

Since one of the driving points behind creating the SOAP specification was to create an interoperable way to communicate between different platforms and different languages, many people are surprised by this assertion. But a funny thing happened on the way to widespread interoperability: the WS-* specifications (and vendors' implementations of said specifications) made SOAP services less interoperable rather than more interoperable.

The problem in the SOAP and WS-* arena is the large number of different standards (and versions of each of those standards) to choose from. And when a particular vendor chooses to implement a particular standard, that vendor often provides an implementation that is just slightly different from another vendor's (or all others). This leads to problems whenever you have to cross vendor boundaries (languages and operating system).

Of course, even to use SOAP you need a SOAP toolkit on your platform, which most (but not all) platforms have today. And then you have to deal with myriad WS-* specifications and figure out which to use (or not to use) and how that affects interoperability. To be honest, it's kind of a mess out there.

In terms of platforms, REST has the advantage because all you need to use REST is an HTTP stack (either on the client or the server). Since almost every platform and device has that today, I would argue that REST has the widest interoperability. Given that mobile devices, household devices, POS devices, DVD players, and TVs all have Internet connectivity, there are more and more platforms for which having a full SOAP toolkit is impossible or unlikely. And even if you do have a SOAP toolkit for a particular platform, the chance of it working with another platform's implementation is not 100%.

But what about metadata? So what if REST is so interoperable—there's no WSDL with REST, and without WSDL, I can't generate a client-side proxy to call a service. REST is hard to use.

It's true that in the world of REST, there is no direct support for generating a client from server-side-generated metadata, as there is in the world of SOAP with Web Service Description Language (WSDL). A couple of efforts are being made to get such support into REST, one being a parallel specification, known as WADL (Web Application Description Language). The other is a push to use WSDL 2.0 to describe RESTful endpoints. I often say that REST is simple, but simple doesn't always mean easy. SOAP is easy (because of WSDL), but easy doesn't always mean simple.

Yes, using WSDL makes generating a proxy for a SOAP-based service easier than writing the code to call a RESTful service. But once you generate that proxy, you still have to learn the API. Nothing in the WSDL tells you which method to call first or second or whether you need to call the methods in any particular order at all. These are all things you need to figure out after you generate the proxy and are prototyping the code to use the service.

Building a client against a RESTful service means you are learning the service and how it works as you build the client. Once you have finished, you have a complete understanding of the service, its resources, and the interaction you can have with those resources. To me, this is a big benefit. Since RESTful services follow the constraints of REST (at least they are supposed to), there is a convention that you can easily follow as you determine the different parts of the service.

Also, out in the wilds of developer-land, most services are wrapped in something often called a "service agent," which is another layer of indirection to protect clients from changes in the service layer. This may be needed in either REST or SOAP.

Another point is that metadata-generated proxies are part of what SOAP was meant to get away from in the RPC era, namely local-remote transparency. The concept of having an API on the client that matches the API on the server was considered to be a bad idea, but that's exactly what happens in most SOAP-based services. Having a metadata-generated proxy in REST also reduces the chances of taking advantage of hyperlinking. Using hypertext as the engine of application state (HATEOAS) is one of the constraints of REST, and using it requires a more loosely coupled client API.

The last point I'll make is that as support for REST becomes more ubiquitous, building clients will get easier and easier. If you look at the Windows Communication Foundation (WCF) REST starter kit, it includes facilities that head in this direction. The new HttpClient API makes using HTTP much easier than using the .NET WebRequest/WebResponse API. Also, there is a new Paste as XML Serializable tool, which allows you to copy a piece of XML (say from the documentation of a RESTful endpoint) and generate a .NET type that can represent that XML instance in your application. This is similar to what the WCF tools do automatically for the whole service with WSDL. Over time, these tools will become much more sophisticated, further simplifying the client experience in WCF when using RESTful services.

What if I want to use a transport other than HTTP?

The common (somewhat sarcastic) answer from the REST community here is, "Go ahead, there isn't anything stopping you." Realistically, however, REST is currently tied to HTTP, if only because most developers and teams of developers do not have the time for the engineering effort necessary to get the semantics of REST to work over, say, TCP/IP.

The common answer is technically correct, because nothing is stopping you from implementing the concepts of REST over other protocols, but until vendors add support for this, I find it a dubious proposition for most.

After all that information, aren't you telling me that REST is good for Internet-facing applications, and SOAP for enterprise applications?

If you've read the rest of this column, you can probably imagine that I think this statement is generalized and false. Often I hear this sentiment after discussing the lack of explicit distributed transaction support in REST versus the explicit support in WS-Atomic Transactions. My retort is generally something like "Well, ASP.NET doesn't have support for distributed transactions, but does that mean ASP.NET isn't useful for enterprises?"

My point is that not every technology solves every problem, and there are plenty of technologies that don't support the typical features people think of when they think of enterprises but that are incredibly helpful for enterprises nonetheless.

In fact, when I think of enterprise applications, I often think of speed and scalability—scalability being one of the main differences between REST and SOAP. SOAP services are much harder to scale than RESTful services, which is, of course, one of the reasons that REST is often chosen as the architecture for services that are exposed via the Internet (like Facebook, MySpace, Twitter, and so on).

Inside enterprises, applications also often need to scale as well. Using REST means that you can take advantage of HTTP caching and other features, like Conditional GET, that aid in scaling services. Many of these techniques can't be used with SOAP because SOAP uses POST only over HTTP.

Bottom Line

I hope that after you read this column, you'll think that the answer to "Which is better, REST or SOAP?" is "It depends." Both the REST architectural style and SOAP and the WS-* protocols have advantages and disadvantages when it comes to building services. Those of us in the RESTafarian camp (yes, I must give full disclosure here: I am definitely in that camp) believe that for most service situations, REST provides more benefits than SOAP or WS-*. On the other hand, SOAP and WS-* have some features that are easy (and possible) to implement using REST. When you need those specific features, you definitely want to use runtimes and toolkits that can provide those features. Although this column wasn't specifically about WCF, one nice feature of adopting WCF is that it supports both REST and SOAP/WS-*. Moving back and forth between the two worlds becomes easier if you have one programming and runtime model to learn.