Discovery Metadata is Just Data

Another comment on Eran's discovery mechanisms list (which is itself great). I'm gradually reaching the conclusion that he's right that content negotiation isn't the best idea, but for the wrong reasons. The question is, are the alternatives any better?

Quoth Eran:

HTTP Content Negotiation - using the 'Accept' request header, the consumer informs the server it is interested in the metadata and not the resource itself, to which the server responds with the metadata document or its location. In XRDS, the consumer sends an HTTP GET (or HEAD) request to the resource URL with an 'Accept' header and content-type 'application/xrds+xml'. This informs the server of the consumer's discovery interest, which in turn may reply with the discovery document itself, redirect to it, or return its location via the 'X-XRDS-Location' (or 'Link') response header.

[-] Resource Declaration - does not address as it focuses on the consumer declaring its intentions.
[+] Direct Metadata Access - provides a simple method for directly requesting the metadata document.
[-] Web Compliant - while some argue that the metadata can be considered another representation of the resource, it is very much external to it. Using the 'Accept' header to request a separate resource (as opposed to a different representation of the same resource) violates the HTTP protocol. It also prevents using the discovery content-type as a valid (self-standing) web resource having its own metadata.
[-] Scale Agnostic - requires access to HTTP request and response headers, as well as the registration of multiple handlers for the same resource URL based on the 'Accept' header. In addition, improper use or implementation of the 'Vary' header in conjunction with the 'Accept' header will cause proxies to serve the metadata document instead of the resource itself - a great concern to large providers with frequently visited front-pages.
[-] Extendable - limited to a single content-type for metadata, and does not allow any existing schemas (with well known content-type).

Minimum roundtrips to retrieve metadata: 1

All of the points above are addressable with minor tweaks, turning this into a usable BigCo-scale solution. Specifically, I'd argue that it's perfectly web compliant to regard a resource's 'metadata' as a variant representation of the resource itself. As an example, consider an image resource that can be requested in several variants: image/gif, image/jpeg, or application/image-meta+xml. The last format gives you the EXIF metadata about the image but in a more convenient XML format. Format A gives you image bits; format B gives you images bits and metadata; formact C gives you just the metadata. But what's data and what's metadata just depends on your point of view.

The other argument against this is that buggy proxy caches may cache the wrong representation of, say, http://yahoo.com. This is something of a strawman in that this would need to be a cache between an RP and an OP. In any case, returning an uncacheable redirect (303?) to a metadata resource would avoid problems in practice.

All of this said, configuring something like http://yahoo.com/ (or, ahem, http://google.com/) to do content negotiation to enable discovery is a tough sell. Whatever technology is used to serve (and cache...) that page needs to be reconfigured to do the Right Thing with regard to content negotiation, with a big downside if something goes wrong and, so far, a small upside if things go well. Not a great sales pitch.

So I think Eran is right in that this isn't a great solution, but not because of web design purity; because of practical deployment issues. If there's a good alternative we should look at it and weigh the pros and cons, which is what I plan to do in the next post.