How to properly invalidate an HTML5 Cache Manifest for online/offline web apps?

I'm currently using a Cache Manifest (as described here). This effectively makes the necessary resources to run the application available when the user is offline.

Unfortunately, it works a little too well.

After the cache manifest is loaded, Firefox 3.5+ caches all of the resources explicitly referenced in the cache manifest. However, if a file on the server is updated and the user tries force-refreshing the page while online (including the cache-manifest itself), Firefox will absolutely refuse to fetch anything. The application remains completely frozen at the last point it was cached. Questions:

1. I want Firefox to effectively only rely on the cached resources when the network connection fails. I've tried using the FALLBACK block, but to no avail. Is this even possible?
2. If #1 is not possible, is it possible for the user to force-refresh a page and bypass this cache (ctrl-F5 doesn't do it and neither does clearing the browser's cache, shockingly) short of clearing their private data? Alternatively, does the cache-manifest mechanism support expiry headers and is its behavior with respect to this documented anywhere?

I think I've got this figured out: if there's an error in one's cache-manifest (say, a referenced file does not exist), then Firefox completely will stop processing anything applicationCache related. Meaning, it won't update anything in your cache, including your cached cache-manifest.

To uncover that this was the issue, I borrowed some code from Mozilla and dropped this into a new (non-cached) HTML file in my application. The final message logged stated that there might be a problem in my cache-manifest, and sure enough there was (a missing file).

// Convenience array of status values
var cacheStatusValues = [];
cacheStatusValues[0] = 'uncached';
cacheStatusValues1 = 'idle';
cacheStatusValues[2] = 'checking';
cacheStatusValues[3] = 'downloading';
cacheStatusValues[4] = 'updateready';
cacheStatusValues[5] = 'obsolete';




// Listeners for all possible events
var cache = window.applicationCache;
cache.addEventListener('cached', logEvent, false);
cache.addEventListener('checking', logEvent, false);
cache.addEventListener('downloading', logEvent, false);
cache.addEventListener('error', logEvent, false);
cache.addEventListener('noupdate', logEvent, false);
cache.addEventListener('obsolete', logEvent, false);
cache.addEventListener('progress', logEvent, false);
cache.addEventListener('updateready', logEvent, false);




// Log every event to the console
function logEvent(e) {
var online, status, type, message;
online = (isOnline()) ? 'yes' : 'no';
status = cacheStatusValues[cache.status];
type = e.type;
message = 'online: ' + online;
message+= ', event: ' + type;
message+= ', status: ' + status;
if (type == 'error' && navigator.onLine) {
message+= ' There was an unknown error, check your Cache Manifest.';
}
log('
'+message);
}




function log(s) {
alert(s);
}




function isOnline() {
return navigator.onLine;
}




if (!$('html').attr('manifest')) {
log('No Cache Manifest listed on the  tag.')
}




// Swap in newly download files when update is ready
cache.addEventListener('updateready', function(e){
// Don't perform "swap" if this is the first cache
if (cacheStatusValues[cache.status] != 'idle') {
cache.swapCache();
log('Swapped/updated the Cache Manifest.');
}
}
, false);




// These two functions check for updates to the manifest file
function checkForUpdates(){
cache.update();
}
function autoCheckForUpdates(){
setInterval(function(){cache.update()}, 10000);
}




return {
isOnline: isOnline,
checkForUpdates: checkForUpdates,
autoCheckForUpdates: autoCheckForUpdates
}
return {
isOnline: isOnline,
checkForUpdates: checkForUpdates,
autoCheckForUpdates: autoCheckForUpdates
}

This was certainly helpful, but I should definitely request a feature from Mozilla that prints out malformed cache-manifests at least to the Error Console. It shouldn't require custom code to attach to these events to diagnose an issue as trivial as a renamed file.

I've used code from HTML5 Rocks: Update the cache:

window.addEventListener('load', function(e) {
  if (window.applicationCache) {
	window.applicationCache.addEventListener('updateready', function(e) {
		if (window.applicationCache.status == window.applicationCache.UPDATEREADY) {
		  // Browser downloaded a new app cache.
		  // Swap it in and reload the page to get the new hotness.
		  window.applicationCache.swapCache();
		  if (confirm('A new version of this site is available. Load it?')) {
			window.location.reload();
		  }
		} else {
		  // Manifest didn't changed. Nothing new to server.
		}
	}, false);
  }
}, false);

I had the same problem: once Firefox saved the offline files, it would not reload them ever. Chrome worked as expected, it checked the manifest file for changes and reloaded everything if the manifest file changed. Firefox didn't even download the manifest file from the server, so it could not notice changes.

After investigation, I found out that Firefox was caching the cache manifest file (old fashioned cache, not the offline cache). Setting the cache header of the manifest file to Cache-Control: no-cache, private solved the problem.

Disclaimer: my experience with manifests and cache is all Safari and FF may handle some things differently.

1. You are quite right. If there are any files listed on the manifest that can't be found, no caching will occur.

2. Even if you are online, the browser will check only the manifest file. While waiting for the manifest file, it will continue to load the site from the cache - that way it doesn't delay rendering - but it means that you don't see any changes on the first load.

3. The next time the site is loaded, if the manifest changed on the previous load, the new files will be loaded.

IT IS ALWAYS NECESSARY TO RELOAD TWICE to see any changes. In fact, I have sometimes had to reload 3 times to see the update. No idea why.

When debugging, I generate my manifest file on the fly with php, so there is no chance of a typo in a filename. I also generate the version number randomly each time to force an update but still have an offline webapp for testing.

Once complete, the php file can just echo the saved manifest data with a constant version number and the cache will always be used.

Just some things I have learned while playing with manifest and cache recently. It works great, but can be confusing.

There is no expiry. To uncache, you have to change the manifest file to have nothing in it and do a reload. On Safari, clearing the user cache does clear out all cached files.

I made an Firefox add-on which invalidates the Cache Manifest and clears HTML5 Local Storage.

http://sites.google.com/site/keigoattic/home/webrelated#TOC-Firefox-HTML5-Offline-Cache-and-Loc

You can also invalidate the cache manifest by typing the code below in the Error Console:

// invalidates the cache manifest
var mani = "http://.../mysite.manifest"; // manifest URL
Components.classes["@mozilla.org/network/application-cache-service;1"].getService(Components.interfaces.nsIApplicationCacheService).getActiveCache(mani).discard();

Or, by typing the code below in the address bar will manually enforece the cache to be updated:

javascript:applicationCache.update()

Hmm, I just called update() on the cache, after making an edit change to the manifest file, and received the full sequence of check/downloading/ready, did one reload, and a text change I had made in one of my js files, that shows up in the initial page of my app, promptly appeared.

Seems I only need one reload.