Make streaming API / websocket information more visible

[andypiper]

Per comments in this issue, the websocket API is currently only really findable via Google search - needs to be surfaced more obviously.

[ThisIsMissEm]

It's currently under API Methods -> Timelines -> Streaming, but it's not actually part of the REST API, so lifting it up to up near REST APIs (perhaps just "APIs") might make more sense.

[mjankowski]

Is this still desired?

Is the idea here to just bump up the link to streaming docs to a top level sidebar link rather than within the rest api?

Presumably we also want to preserve and/or redirect the urls.

Should the page content stay all as one, or be split into the portions which are indeed part of "the rest api" and the parts which are narrowly used by ws/streaming?

[trwnh]

i don't think we distinguish between REST and non-REST (whatever that would look like), and we generally shouldn't. for example, we include OAuth and OEmbed routes in the same "API methods" tree, and we do the same for health checks.

the confusion seems to have been about nesting the streaming methods indented underneath timelines, which was done because streaming is conceptually related to timelines. i think the better solution is to make the theme support search, and make the sidebar collapsible so that it's not so intimidating.

tangentially, it seems that some new methods were added that could be conceptually grouped together, but currently aren't:

• health could be grouped under instance
• profile could be grouped under accounts
• grouped notifications could be merged into notifications

[mjankowski]

i don't think we distinguish between REST and non-REST (whatever that would look like)

For streaming specifically, there's one endpoint which either returns 404 not found or redirects to the streaming url when on a different host. I'd argue this is in "the REST API" and not the streaming API ... though I generally agree that drawing these lines might not be useful. Was mostly trying to sort out how to resolve this issue.

the confusion seems to have been about nesting the streaming methods indented underneath timelines, which was done because streaming is conceptually related to timelines.

Would making this change (move it out of timelines) resolve this issue?

i think the better solution is to make the theme support search, and make the sidebar collapsible so that it's not so intimidating.

I'm not sure how to get an opinion from anyone on the desirability of things like search, theme changes, collapse, etc - but I agree those would all be improvements.

tangentially, it seems that some new methods were added that could be conceptually grouped together [...]

They could ... there's sort of a mix of grouping just by API method endpoint/path, or grouping thematically.

[andypiper]

I'm not sure how to get an opinion from anyone on the desirability of things like search, theme changes, collapse, etc - but I agree those would all be improvements.

Off topic for this specific issue, but it is a relevant point.

• Yes, having search, and an alternative theme, would be great.
• A fully modernised docs site is desirable++! Choose your adventure:
  • there is a suggestion to move the docs into the same repo as the source, which might reduce the gap between functionality and what is documented, but would be very disruptive (at a point in time) to ongoing development
  • it would be great to have an OpenAPI spec with generated docs, but there are historical reasons why that didn't happen, and it would mean cutting off older docs and legacy options to focus on a fresh start
  • a full UX review and revised information architecture would be very helpful (but our UX folks are focused on the "product" user experience rather than our developer docs)
  • we would like the docs and project website and blog etc to share common theme and elements and be light-and-dark media friendly, this is a even (!) larger effort.

The tl;dr here is that we're not likely to be able to take any changes of this kind on, very quickly or lightly.

An imperfect and unhappy response, but we're a bit stuck in the patches-on-patches situation for the moment.

[mjankowski]

Thanks for that summary and clarification of where things are at. Those are all good goals/improvements!

At risk of further derailing -- and maybe this is way too nuanced a question -- but just to be clear:

• Are we in a "hard no" scenario, where nothing along these lines will be at all contemplated/reviewed/etc (until the team can eventually do this in a coordinated way, "correctly" in-house), and where some issues/PRs can probably close? Light mode #1068 / Search-able documentation #1075 / Add changelog / deprecations page to REST API section #1443 / Collapse sections of the sidebar by default #1061 / Update theme #1130 / maybe more?
• More of a "soft no", where efforts at light/dark mode, search, sidebar, etc - MIGHT be considered ... but only if easy to review and only if they kept things on the path to those larger goals?

Basically ... should we close all current "docs site meta issues" sort of stuff and essentially only entertain content changes for now? (maybe update README and/or contribute guide along those lines...)

[trwnh]

@mjankowski said in #1237 (comment):

the confusion seems to have been about nesting the streaming methods indented underneath timelines, which was done because streaming is conceptually related to timelines.

Would making this change (move it out of timelines) resolve this issue?

in the narrowest sense, it would address the request that led to this issue being filed, but it would not really be a valid solution because it undoes the conceptual grouping. the real issue is that some people cannot find certain methods because there are a lot of method pages. the conceptual grouping is a bit of a hack to let the reader navigate more quickly. if you're looking for the lowest-effort change that would allow this issue to be closed, then i would suggest something like the following:

1. ungroup the table-of-contents, so that methods pages are presented in alphabetical order, and someone looking for streaming can scroll down to S in the alphabetized list. note that the admin apis should either remain grouped, or the pages should be relabeled to admin/* in the table of contents and subsequently flattened.
2. recreate the conceptual groups in a different way, e.g. maybe by expanding or reworking the client developer guide with a "features" subcategory (and move/merge the "implementing quote posts" page into this new structure), and re-evaluating overlaps with the more generic "rest api" category.

i suppose the 1st point could be done in its own PR, but the conceptual grouping is very useful to me and i suspect others, so getting rid of it without immediately recreating it would represent a loss of overall value to the documentation.

---

@andypiper said in #1237 (comment):

A fully modernised docs site is desirable++! [...] a full UX review and revised information architecture would be very helpful (but our UX folks are focused on the "product" user experience rather than our developer docs) [...] The tl;dr here is that we're not likely to be able to take any changes of this kind on, very quickly or lightly.

If there's room in the Mastodon organizational budget or strategy for undertaking this at some point, I'm open to working out some kind of proposal/contract as long as there's a clear scope and someone I can contact or liaison with as a stakeholder within the Mastodon organization.

there is a suggestion to move the docs into the same repo as the source, which might reduce the gap between functionality and what is documented, but would be very disruptive (at a point in time) to ongoing development

this could be done, but probably not as the first thing. having things in the same repo doesn't guarantee that the docs will be in sync with the code. if done right now, it would probably not only be disruptive to the main project, but it would also make it harder to make doc changes since the main repo has more restrictive requirements on merging PRs. i'd suggest holding off on this and prioritizing a more rapid modernization of the documentation first.

it would be great to have an OpenAPI spec with generated docs, but there are historical reasons why that didn't happen, and it would mean cutting off older docs and legacy options to focus on a fresh start

i don't think it necessarily involves cutting off older docs, but at the very least it involves a lot of work. i can say right now that it is possible to use OpenAPI definitions to generate themed documentation pages with Hugo using content adapters, so it's not necessary to use tools like Swagger if not desired. this isn't the hardest or biggest part of the work, though -- the main barrier to OpenAPI definitions in Mastodon has historically been the difficulty in mapping the Mastodon codebase to OpenAPI definitions, as well as the inherent inexpressibility of certain aspects of the Mastodon API in OpenAPI's schema. i think we have gotten to a point where OpenAPI definitions are at least possible, but there are caveats.

• Previously, OpenAPI required defining one or more servers to provide a base path for the API routes. This seems to no longer be an issue, because we can use Server Variables without hardcoding a certain host.
• In the absence of a good library to generate OpenAPI definitions from Ruby code, I suppose we can generate OpenAPI definitions manually... as long as we have stable targets. To simplify this, let's say our target is "released versions of Mastodon", and to trim it down further, let's say "currently supported versions of Mastodon". We can backfill our archive with older versions later at a lower priority.
• Eventually, we can add a feature to the Mastodon codebase to serve an OpenAPI definition for each instance, similar to Akkoma and Misskey. In the longer term, we can start hooking up those OpenAPI definitions to the actual source code. But in the immediate present, the existing documentation can be adapted and double-checked for correctness along the way.

we would like the docs and project website and blog etc to share common theme and elements and be light-and-dark media friendly, this is a even (!) larger effort.

this could somewhat contradict the desire to merge the docs into the main repo, if the docs are also expected to share project-level assets with the website and blog. i would leave this until later in the lifecycle of the docs modernization project.

[mjankowski]

@andypiper any thoughts on my query from a few months ago RE: search, light/dark modes, theme stuff, sidebar re-org, etc...

should we close all current "docs site meta issues" sort of stuff and essentially only entertain content changes for now? (maybe update README and/or contribute guide along those lines...)

[andypiper]

We'll possibly start on a new docs site project in the coming month or so (still getting this aligned and funded). I think closing the meta issues for things we know would go into a future site like search and theme, would make sense at this point.