ASP.NET Custom 404 Returning 200 OK Instead of 404 Not Found

After trying to setup my site for Google Webmaster Tools I found that my Custom ASP.NET 404 page was not returning the 404 status code. It displayed the correct custom page and told the browser that everything is OK. This is consider a soft 404 or false 404. Google doesn't like this. So I found many articles on the issue but the solution I want didn't seem to work.

The solution I want to work is adding the following two lines to the code behind Page_Load method of the custom 404 page.

Response.Status = "404 Not Found";
Response.StatusCode = 404;

This doesn't work. The page still returns 200 OK. I found however that if I hard code the following code into the design code it will work properly.

<asp:Content ID="ContentMain" ContentPlaceHolderID="ContentPlaceHolderMaster" runat="server">

<%
	Response.Status = "404 Not Found";
	Response.StatusCode = 404;
%>

 ... Much more code ...

</asp:content>

The page is using a master page. And I am configuring custom error pages in my web.config. I would really rather use the code behind option but I can't seem to make it work without putting a the hack inline code in the design / layout.

Solution:

The problem, it turned out, was the use of the master page. I got it to work by setting the status code later in the pages lifecycle, obviously the rendering of the master page was resetting it, so I overrode the render method and set it after the render was complete.

protected override void Render(HtmlTextWriter writer)
{
    base.Render(writer);
    Response.StatusCode = 404;
}

More work could be done to find out exactly when the master page is setting the status, but I'll leave that to you.

---

Original Post:

I was able to get a test web app to work fine, well it at least displayed the custom error page and returned a 404 status code. I can't tell you what is wrong with your app, but I can tell you what I did:

1. Edited the web.config for custom errors:

> > >

2. Added a 404.aspx page and set the status code to 404.

> public partial class _04 : System.Web.UI.Page > { > protected void Page_Load(object sender, EventArgs e) > { > Response.StatusCode = 404; > } > }

Thats about it, if I go to any page extension that is processed by Asp.Net and does not exist, my fiddler log clearly shows a 404, here is the header:

> HTTP/1.1 404 Not Found > Server: Microsoft-IIS/5.1 > Date: Sun, 07 Dec 2008 06:04:13 GMT > X-Powered-By: ASP.NET > X-AspNet-Version: 2.0.50727 > Cache-Control: private > Content-Type: text/html; charset=utf-8 > Content-Length: 533

Now if I go to a page that is not processed by Asp.Net, like a htm file, the custom page does not show and the 404 that is configured by IIS is displayed.

Here is a post that goes into some more details that may be of use to you and your problem, my test does do a redirect to the new page so the url of the requested file is pretty much lost (except its in the query string).

https://stackoverflow.com/questions/152307/google-404-and-net-custom-error-pages

Header Spy Response:

HTTP/1.1 404 Not Found
Date: Sun, 07 Dec 2008 06:21:20 GMT

I had a similar problem I want to show a custom page as a 404 (which is ASPX) and it worked fine on localhost but as soon as a remote visitor connected they would get the generic IIS 404.

The solution to this was to add

Response.TrySkipIisCustomErrors = true;

Before changing the Response.StatusCode.

Found via Rick Strahl http://www.west-wind.com/weblog/posts/745738.aspx

Try calling Response.End() to skip rendering...

Response.Status = "404 Not Found";
Response.StatusCode = 404;
Response.End();
return;

The IIS 7 solution is to just add this to your web.config file:

<system.webServer>
  <httpErrors existingResponse="Replace">
    <remove statusCode="500" subStatusCode="-1" />
    <remove statusCode="404" subStatusCode="-1" />
    <error statusCode="404" prefixLanguageFilePath="" path="404.htm" responseMode="File" />
    <error statusCode="500" prefixLanguageFilePath="" path="500.htm" responseMode="File" />
  </httpErrors>
</system.webServer>

http://forums.asp.net/t/1563128.aspx/1

After much testing and troubleshooting it appears that certain hosting providers can interfere with the return code. I was able to get around this by applying a "hack" in the content.

<%
// This code is required for host that do special 404 handling...
Response.Status = "404 Not Found";
Response.StatusCode = 404;
%>

This will allow the page to return the correct return code no matter what.

I was able to get around this issue by using the following setup in asp.net webforms using .NET 3.5.

The pattern I've implemented bypasses .NET's custom redirect solution in the web.config as I've written my own to handle all scenarios with the correct HTTP status code in the header.

First, the web.config's customErrors section looks like this:

<customErrors mode="RemoteOnly" defaultRedirect="~/error.htm" />

This setup ensures that CustomErrors mode is set to on, a setting we'll need later, and provides an all-else-fails option for the defaultRedirect of error.htm. This will come in handy when I don't have a handler for the specific error, or there's something along the lines of a broken database connection.

Second, here's the global asax Error event:

protected void Application_Error(object sender, EventArgs e)
	{
	   HandleError();
	}

	private void HandleError()
	{
		var exception = Server.GetLastError();
		if (exception == null) return;

		var baseException = exception.GetBaseException();

		bool errorHandled = _applicationErrorHandler.HandleError(baseException);
		if (!errorHandled) return;


		var lastError = Server.GetLastError();
	if (null != lastError && HttpContext.Current.IsCustomErrorEnabled)
	{
		Elmah.ErrorSignal.FromCurrentContext().Raise(lastError.GetBaseException());
		Server.ClearError();
	}
	}

This code is passing off the responsibility of handling the error to another class. If the error isn't handled and CustomErrors is turned on, that means we've got a case where we're on production and somehow an error hasn't been handled. We'll clear it here in order to prevent the user from seeing it, but log it in Elmah so we know what's going on.

The applicationErrorHandler class looks like this:

public bool HandleError(Exception exception)
		{
			if (exception == null) return false;

			var baseException = exception.GetBaseException();

			Elmah.ErrorSignal.FromCurrentContext().Raise(baseException);
			
			if (!HttpContext.Current.IsCustomErrorEnabled) return false;
			
			try
			{
				
				var behavior = _responseBehaviorFactory.GetBehavior(exception);
				if (behavior != null)
				{
					behavior.ExecuteRedirect();
					return true;
				}
			}
			catch (Exception ex)
			{
				Elmah.ErrorSignal.FromCurrentContext().Raise(ex);
			}
			return false;
		}

This class essentially uses the command pattern to locate the appropriate error handler for the type of error that's issued. It's important to use Exception.GetBaseException() at this level, because almost every error will be wrapped in a higher-level exception. For example, doing "throw new System.Exception()" from any aspx page will result in an HttpUnhandledException being received at this level, not a System.Exception.

The "factory" code is simple and looks like this:

public ResponseBehaviorFactory()
	{
		_behaviors = new Dictionary<Type, Func<IResponseBehavior>>
		             	{
		             		{typeof(StoreException), () => new Found302StoreResponseBehavior()},
		             		{typeof(HttpUnhandledException), () => new HttpExceptionResponseBehavior()},
							{typeof(HttpException), () => new HttpExceptionResponseBehavior()},
		             		{typeof(Exception), () => new Found302DefaultResponseBehavior()}
		             	};
	}

	public IResponseBehavior GetBehavior(Exception exception)
	{                                                                               
		if (exception == null) throw new ArgumentNullException("exception");

		Func<IResponseBehavior> behavior;
		bool tryGetValue = _behaviors.TryGetValue(exception.GetType(), out behavior);

		//default value here:
		if (!tryGetValue)
			_behaviors.TryGetValue(typeof(Exception), out behavior);

		if (behavior == null)
			Elmah.ErrorSignal.FromCurrentContext().Raise(
				new Exception(
					"Danger! No Behavior defined for this Exception, therefore the user might have received a yellow screen of death!",
					exception));
		return behavior();
	}

In the end, I've got an extensible error handling scheme setup. In each one of the "behaviors" that is defined, I have a custom implementation for the type of error. For example, a Http exception will be inspected for the status code and handled appropriately. A 404 status code will require a Server.Transfer instead of a Request.Redirect, along with the appropriate status code written in the header.

Hope this helps.

You can use the below code:

 Response.TrySkipIisCustomErrors = True
 Response.Status = "404 Not Found"
 Response.AddHeader("Location", "{your-path-to-your-404-page}")