Generator versioning

[rohitst]

Given that:

• Changes (sometimes breaking) to generators are happening frequently
• We want to encourage active development on the generators
• We have production or almost in production projects that heavily depend on api-builder generated clients.
• We can't always depend on someone who knows the source of the generator being available to fix issues asap, and hence development/release progress is blocked

I propose some kind of versioning for api-builder generators that let unstable dev work go on, but let projects that require some level of stability stick to known generator versions

Some possible solutions

• each generator can have a 'stable lts' and a 'nightly' variant. This can probably be achieved with no infrastructure changes using prefixes to the generator keys, and some cumbersome merging when needed (so http4s_stable and http4s_nightly)
• build in proper versioning to generators. This is probably a lot of work because we'd have to store and load the versions (or at least jar binaries) somewhere, and current structure has all generators in one active project. + all the necessary changes to the cli and api, etc.
• split apart the generators each into their own git repos. Use lambdas to host generators and all old versions

[rlmartin]

I agree that generator versioning would be helpful for stability.

I think the hard part will be that, since generator authors own the actual generation, the onus will be on them to properly serve and support legacy generators. So ApiBuilder can dictate the model behavior via the generator API definition, but we might see spotty support of old versions (especially since new versions will likely be patchsets on top of legacy versions).

There could be a generator url structure below the generator, something like /generators/:name/versions and /generators/:name/versions/:version. This might be maintainable internally in the generator by freezing code in an old version and working on a new version elsewhere in the project. This also raises the question of how to deprecate/remove unsupported legacy versions; would be nice to surface some metrics around version usage to enable generator authors to see which versions can be removed.

Side note: the apibuilder-cli config would also need to be expanded to support a version (or latest) of the requested generator.

[fiadliel]

A suggestion: a way of doing this might to actually snapshot the generated code and store after building (a code owner may want a way of controlling this).

The value of this is that a certain code version will always return the same code (byte for byte). You can get updates if the generators have been improved, by bumping version.

It is, in some ways, just reproducing what already exists with current packaging systems. An even better option would be to emulate (or proxy) a Maven repository for Java/Scala apps, etc. where you actually also get dependency management (upstream libraries, etc.). But this would be a lot more work.

[fiadliel]

Above comment is to snapshot non-snapshot versions (haha) - snapshot/dev versions would not be cached.

[rohitst]

If I understand correctly... snapshotting generated code is probably not as useful (...a snapshot is already probably stored in the project's git repo). What is needed is to run the old version of the generator against a new api.json.