SOAP faults or results object?

I was having a discussion about this with a co-worker and we couldn't come to an agreement, so I wanted to get your thoughts. I have my own opinions on this, but I won't spoil it for you.

When should I be returning a SOAP fault and when should I be returning a result object that has error information? Assume this is for a generic web service that can be consumed by various systems (.NET, Java, whatever). The result object would have an isError flag, an errorType (similar to specific exception type), and a message.

Some points to consider:

1. Is a data validation error a fault?
2. Should there be a combination of faults (for very exceptional cases) and the results object (for "expected" errors)?
3. How would you group SOAP faults (critical [null reference] vs validation [zip code incorrect])?
4. Fail-fast vs having to remember to check for error
5. Best practices, patterns, standards, etc.

Links to articles are valid. Even though it sounds like I want your opinion, please stick to facts (x is better because of y and z...)

Most SOAP clients will convert faults into a runtime exception (if that is something the client language supports). With that in mind, I think you could rephrase the question as "When do I want to throw an exception instead of returning an error value"? I'm sure you can find lots of opinions about that API design in general and that topic in particular.

That said, returning an error is usually not helpful to the client:

1. The client needs to manually enumerate and handle your error codes vs. allowing the stub code to generate and throw an exception of the appropriate type. Using error codes prevents the client from using object-oriented techniques like handling exceptions by superclass.

2. If you don't make your error codes part of the WSDL; the client will have no documentation on what they are or when they occur. Typed faults are part of the WSDL and therefore (to some limited extent) self-documenting.

3. Fault messages can contain fault-specific context that the client can make use of for error reporting and recovery. For example, throwing an input validation fault containing the actual invalid input element and a reason. If you return a result with an error code and an opaque string, the client has little choice but to pass your error message on to the user, which breaks internationalization, UI consistency, etc.

To answer your specific questions:

1. A validation error is a fault. Imagine if you invoke the web service from an AJAX client with limited error handling ability; you want the service to return a 5xx HTTP code, not a 400 success code with some unexpected response.

2. No. APIs should provide consistent error reporting interfaces. WSDL design is API design. Forcing the client to implement two distinct error handlers does not make the client's life easier.

3. Fault design should mirror your request/response model and display information appropriate to the abstraction of the service. Don't design a NullReference fault; design a XYZServiceRuntimeFault. If clients frequently provide invalid requests, design a InvalidRequestFault, with appropriate subclasses so that clients can quickly figure out where the invalid data is.

A results object should only contain results. If your result object is providing a list of errors that have occurred on another system then that is an example of when you can have and "isError" flag; otherwise you can't because a result is either valid or not.

You should always be using a SOAPFault when an error occurs. Validation is an error, and it's the devils own trap to think of validation as being less severe than an inability to open a database. Both cases have the same impact - the operation cannot be completed as requested.

So you should use result objects for results and SOAP Faults for anything that prevents a valid result object; including but not limited to errors, validation failures, warnings, bus faults, etc..

In the days before exceptions there was no choice and as a result many APIs became inconsistent and most APIs differed on how to return an error. It was (and still is) horrible, confusing and often slows down development because you have to lookup how each API entry returns an error, and often how to decode or find out more about the error.

1. To handle validation with SOAPFaults / Exceptions is more logical when you think about it, and once you've thought about it is usually easier. You do need to design the validation fault class so that it contains sufficient information to identify the offending elements in a manner not necessarily requiring the original request. This way you can start to handle validation errors more generically.

2. If the results object contains errors they can only be within the domain of the results; for example Product out of stock because someone in the wharehouse can't count is within the domain of inventory control.

3. It is not wise to make the distinction between a critical error and a validation error, this to my mind isn't a valid comparison because any assignation of severity level is very subjective. For example in a system providing information about chemicals to a firefighter, critical probably means that the truck on fire is carrying UN 1298 & UN 1436 and not a null reference when attempting to load the warning graphic.

Design the faults in to allow them to be identified concisely and handled accordingly. Ensure that they convey sufficient information. Abritrary categorisation is something that is unncessary when you've got sufficient information because the Fault will allow itself to be indentified.

4. SOAPFaults turned into Exceptions are the surest way of having fail-fast.

5. Best practices, references etc.

• Managing Exceptions in a SOA world: Ramesh Ranganathan

• When Exceptions Are the Rule : Achieving reliable and traceable service-oriented architectures

• Best practices on SOAP Faults

• Use SOAP Faults to deliver the appropriate level of detail to the developer at development time, and to the customer while the Web service is in production.

• Web services use SOAP faults to report fault cases back to clients. The faults can be generated from the SOAP framework in a case of invalid SOAP messages, invalid security tokens or they can be generated from the service business logic itself

• If you send a message that was not successful for some reason, you may get back a response containing a SOAP fault element, which gives you status information, error information, or both. There can be only one SOAP fault element in a message, and it must be an entry in the SOAP body. Furthermore, if there is a SOAP fault element in the SOAP body, there can be no other elements in the SOAP body. This means that when you add a SOAP fault element, you have effectively completed the construction of the SOAP body.

• SOAP fault messages are the mechanism by which SOAP applications report errors “upstream,” to nodes earlier in the message path. It's the mission of this section to provide a full and detailed explanation of SOAP faults so that you can handle them appropriately in your own Web services.

I think the short answer is use a soap fault, unless you know the client will be equipped to handle an error returned as the result. I was also thinking the analogy between exceptions and error results, as mentioned in the other answers.

There are grey areas that could both be reasonably treated as exceptions and as result errors depending upon the needs of the client. You might then provide a parameter to the service that alters how these types of error is returned. The default is to use a SOAP fault, but if the client sets the parameter to get error results, then it is indicating that it is willing to handle this as part of the result. For me, validation errors are in this grey area. For most clients they should be considered faults, but if the client wants to use the data to markup the UI with validation errors, then returning the validation errors as part of the result may be more useful. 

The rule I follow in service design is:

• business level response (even business exceptions) into response objects
• technical/integration level faults into Soap Fault

The service consumer can rely that all kind of business response comes in response objects and presents it to service (business) users. The Soap Faults are used only when business response cannot be delivered.

The Soap Faults should trigger a support warning/action via monitoring.