I started with the 404 Search from Weblog Tools Collection, which simply redirects 404 pages to search results. That's not technically correct behaviour, so I borrowed the first few lines of my 404.php from their example:

<?php
$search_term = substr($_SERVER['REQUEST_URI'],1);
$search_term = urldecode(stripslashes($search_term));
$find = array ("'.html'", "'.+/'", "'[-/_]'") ;
$replace = " " ;
$search_term = trim(preg_replace ( $find , $replace , $search_term ));
$search_term_q = preg_replace('/ /', '%20', $search_term);
get_header() ?>

Then the rest of the 404 template, which you can check out here, follows normally from the sandbox file, until after the search field (which is important for usability). I don't use this in my index or single page templates, because 404's shouldn't land there (if they do, it's a problem with wp_query() ). After the search field, we jump into a custom loop:

<?php /* run the url as a query */
query_posts('s='. $search_term_q );
?>
<?php if ( have_posts() ) :
/* check to see if there are posts, before telling people something that isn't true */ ?>
<div class="content-column left"><h3>You might have been looking for these posts</h3>
<ul>
<?php /* start the loop */
while ( have_posts() ) : the_post(); ?>
<li><strong><a href="<?php the_permalink() ?>" title="<?php printf(__('Permalink to %s', 'sandbox'), wp_specialchars(get_the_title(), 1)) ?>" rel="bookmark"><?php the_title() ?></a></strong><br />
<abbr title="<?php the_time('Y-m-d\TH:i:sO'); ?>"><?php unset($previousday); printf(__('%1$s', 'sandbox'), the_date('', '', '', false), get_the_time()) ?></abbr><br />
<em>
<?php printf(__('Posted in %s', 'sandbox'), get_the_category_list(', ')) ?>
. &nbsp;
<?php comments_popup_link(__('Comments (0)', 'sandbox'), __('Comments (1)', 'sandbox'), __('Comments (%)', 'sandbox')) ?>
</em>
</li>
<?php /* end the loop */
endwhile; ?>
</ul></div>
<?php /* close the list before closing the ifelse */
else : endif; ?>

You can read the comments, but basically, we take the URL and use query_posts() to run it as a search, and return the results as a list, if there are any. So far, it works for posts that I've referenced elsewhere, but it's not returning the actual post itself. That's probably because I use category slugs in my URL's. and BTW, the fancy code highlighting in this post is courtesy of a fancy new plugin.

UPDATE: Some additional code was used from Arun's nearly perfect 404.

Seeing the search on wordpress.org, i found an idea. Yahoo!'s search API is wide open, and they have plenty of Example Code. (important for me, since i'm no expert) it still needs a hell of a lot of tweaking, but it should be relatively easy to integrate into a theme, since theme authors can control what the results page looks like.

Unfortunately, this sort of thing should be implemented as a plugin. which means going the route of falbum, or UTW, or whathaveyou, and distributing a basic template, and giving instructions on how to integrate it.