Change static root path

[jsha]

Right now, docs.rs passes --static-root-path=/ to rustdoc, and static files are served directly from the docs.rs root, e.g. https://docs.rs/storage-20220709-1.64.0-nightly-6dba4ed21.js.

It would be aesthetically pleasing if these static files had their own directory. For instance, we could pass --static-root-path=/rustdoc.static/, and simultaneously change the storage path at which we upload these files to be /rustdoc.static/. Then static files would be served from URLs like `https://docs.rs/rustdoc.static/storage-20221225-1.68.0-nightly-abc1234567.js.

This does not require any coordinated changes from rustdoc. We could do it today. It happens to harmonize with (but not depend on) rust-lang/rust#101702, which envisions a similar change rustdoc's default output, where static files are put in their own directory.

It does depend on a tweak to how we serve static files, specifically SharedResourceHandler:

docs.rs/src/web/rustdoc.rs

Lines 726 to 749 in 802fd8e

	pub struct SharedResourceHandler;
	impl Handler for SharedResourceHandler {
	fn handle(&self, req: &mut Request) -> IronResult<Response> {
	let path = req.url.path();
	let filename = path.last().unwrap(); // unwrap is fine: vector is non-empty
	if let Some(extension) = Path::new(filename).extension() {
	if ["js", "css", "woff", "woff2", "svg", "png"]
	.iter()
	.any(|s| *s == extension)
	{
	let storage = extension!(req, Storage);
	let config = extension!(req, Config);
	if let Ok(file) = File::from_path(storage, filename, config) {
	return Ok(file.serve());
	}
	}
	}
	// Just always return a 404 here - the main handler will then try the other handlers
	Err(Nope::ResourceNotFound.into())
	}
	}

SharedResourceHandler considers only the last component of a URL's path when serving ["js", "css", "woff", "woff2", "svg", "png"]. We could add a small bit of extra logic: If a URL's path begins with /rustdoc.static, consider the full path.

[syphar]

we should definitely change the static path root. This the namespace overlaps with crate-names we typically try to use a /-/ prefix for our internal paths where possible.

IMO the storage location doesn't necessarily have to be the same / have the same name.

I agree that rust-lang/rust#101702 doesn't affect the static root path or the static file location.

In general I would have loved to have split the storage for static files by nightly release so we don't have everything in one folder, but that would break the shared caching of these since the prefix would have to include the nightly version.
Same as having a static file archive by nightly, which would ease the download but make caching / serving these harder.

[syphar]

@jsha I was trying to prepare a branch doing the change, and a question about rust-lang/rust#101702 came to me:

Will the static files be moved to a subfolder as you described?

To me it looks like our current build process would then also copy them to a subfolder onto our S3 bucket, which also means SharedResourcesHandler wouldn't pick them up since it only looks at the filename, ignoring any folder.

So from a pure docs.rs perspective the easier approach would be not to move the static assets in rustdoc.

We can still serve them through providing another static-root-path

[jsha]

Oh great! I had started on a branch (https://github.com/jsha/docs.rs/pull/new/static-new-path) but if you can finish it out that's great. The branch I have still needs a new route handler.

To me it looks like our current build process would then also copy them to a subfolder onto our S3 bucket, which also means SharedResourcesHandler wouldn't pick them up since it only looks at the filename, ignoring any folder.

Yes. Here's my idea: the /-/rustdoc.static/ files wouldn't rely on SharedResourcesHandler. Instead they would rely on a more traditional route handler for that path that serves from S3 in the "normal" way (including paths).

So from a pure docs.rs perspective the easier approach would be not to move the static assets in rustdoc.

Here's my reasoning on putting them in a subfolder by default in rustdoc: Some of the past bugs in static asset handling were related to confusion about the doc root vs the static asset root, because they are the same by default. Making them different by default makes it much easier to understand what's going on and makes bug less likely.

This is indeed awkward for the transition moment. Here's what I propose: If the docs.rs builder sees a target/doc/static.files/ folder, it copies those contents as the static root; otherwise it copies target/doc/ as the static root. Then once rust-lang/rust#101702 is released we can remove the logic and always copy target/doc/static.files/.

[jsha]

I updated https://github.com/jsha/docs.rs/pull/new/static-new-path and it now has a route-based handler. I had to comment out a different route to make it work. Clearly something is going on with priority and specificness of routes, but I couldn't figure out what or how to fix it so far. Putting my handler above or below the other handler doesn't seem to change things.

    routes.static_resource("/-/rustdoc.static/*", super::rustdoc::SharedResourceHandler2);
//    routes.rustdoc_page(
//        "/:crate/:version/:target",
//        super::rustdoc::rustdoc_redirector_handler,
//    );

[syphar]

Having slept over it, and read your comment, having a subdirectory from rustdoc for static assets is probably cleaner than the current situation.

IMO add_essential_files then only should look at this static-file folder and not copy everything as it does right now.

So when we are sure about the change in rustdoc, we should prepare docs.rs for it.

[jsha]

Great! Meanwhile I figured out the routing issue on my branch-in-progress (needed to add a route for /-/rustdoc.static/:single, because it appears Iron considers that more specific?).

[Nemo157]

Yeah, irons router has always been broken, it was eventually fixed in the underlying library http-rs/route-recognizer#20 but iron isn't updated.