invocation specific rustdoc static files are cached too long on `/latest/`

[syphar]

I want to clarify / wrap up a caching issue I'm seeing. We discovered it in the discussion in #1590 (partially fixed by rust-lang/rust#92742).

For rustdoc we serve three kinds of static file types:

• unversioned-shared-resources like fonts. Will be served from static-root-path with its filename, not hashed. Cached forever, can never change.
• toolchain-shared-resources are static files that might only change when the toolchain (nightly) changes. Will be served from static-root-path but includes the toolchain version in the filename (via --resource-suffix). Will be cached forever, when these change, the file name and its references in the HTML changes.

Now we come to invocation-specific, for example sitebar-items.js and search-index.js:

• in the past we only served these under a URL that includes create/version.
• so under the assumption that a certain release with a certain nightly will always get the same output for these, just adding the same night resource suffix is good enough.

Now with /latest/ the URL for these invocation-specific items don't contain the version any more.

so, if there are two releases of a crate with the same nightly version, the cache will still serve the invocation-specific files for both releases

possible solutions?

• I'm not sure about the best one. Our first thought was adding a timestamp or the release to the resource-suffix. The problem is that this would also change the suffix for toolchain shared resources. Which would be a performance issue, and also we would have to change static file serving for these.

Another approach could actually be doing the first step towards #1552 by creating an invalidation-request for /crate_name/* after each release.

I'm open for other ideas.

[syphar]

cc @Nemo157 @jsha follow-up on #1590

[jsha]

It seems like there should be a separate suffix for invocation-specific items - in fact, I thought that was the point of invocation-specific, though I hadn't looked at the code too closely. That would be a change in rustdoc - would that solve the problem for docs.rs?

Another possibility would be to make a prefix for invocation-specific items. That prefix could point to the explicitly versioned path (../../0.1.0/<crate_name>/search-index-XXXX.js).

[Nemo157]

Another possibility would be to make a prefix for invocation-specific items. That prefix could point to the explicitly versioned path (../../0.1.0/<crate_name>/search-index-XXXX.js).

That would have to be a dynamic prefix, the sidebar.js files get put into every subdirectory, so it'd have to prepend the correct number of ../ to get back to the doc-root before adding the ../../<version>/<crate>.

[syphar]

Also one additional thing I remembered: when we go the invalidation-way, we have to forbid caching in the browser for these.

Does @GuillaumeGomez has an opinion on this? This feels more like a design-question for rustdoc.

Should there be an additional suffix for invocation-specific files? Do sidebar-items.js and search-index.js even count as invocation specific?

[syphar]

we could also go another way:
don't cache these files on /lastest/, not in the CDN and not in the browser.

[GuillaumeGomez]

I don't really have an opinion on this. You have needs and if we can match your needs on rustdoc side without making breaking changes, I think it's fine to do it (depending on the size of the change too).

[jsha]

If we land rust-lang/rust#101702, the toolchain specific files won't get the resource suffix anymore (which is great, better cacheability!). That means docs.rs is free to add crate-specific info to the --resource-suffix: in particular the crate version number. That would fix this issue. Instead of requesting e.g. https://docs.rs/crate_foo/latest/crate_foo/some_module/sidebar-items-20220517-1.63.0-nightly-4c5f6e627.js, the HTML would request https://docs.rs/crate_foo/latest/crate_foo/some_module/sidebar-items-20220517-1.63.0-nightly-4c5f6e627-1.0.0.js (where 1.0.0 is the version of crate_foo). That could be infinitely cached, since version 1.0.1 of the crate would load a different JS URL.

Edit: Also I should mention I just filed #1851 which proposes a different, more heavyweight intervention: adding an additional flag. What I'm proposing in this comment would be a very lightweight fix (I think); In #1851 I contemplate a more comprehensive but more complicated fix.