feat: simple and fast method for obtaining project overview for spaces, operations, and resource relationships

[michael-johnston]

Is your feature request related to a problem? Please describe.

When working with an ado project, I routinely need two pictures at once:

• Campaign design e.g. which discovery spaces exist, what experiments they use, how large or structured the entity space is.
• Work done e.g. how much has been measured, which operations ran, and what downstream resources (samplestores, datacontainers) exist.

Neither picture is easy to get in one pass today.

For spaces I start with ado get spaces (sometimes --details), but the default listing is thin, and richer structure — experiments, space size, measurement coverage — still requires fetching YAML per space or running additional commands (requiring copy/pastings many ids)

For progress you need to use either ado show summary and ado show details, but:

• details is rich but scoped to one space or operation at a time; for single instance show details operation is relatively quick; however show details space is much slower; when querying many spaces/operations time adds up quickly.
• summary requires explicit space IDs or filters rather than a single project-wide view, it only gives high-level percentages, and is slows to execute across many spaces (appears same as serial execution of show details space)

Understanding how resources connect (for example whether an operation produced a datacontainer) means calling ado show related per resource - no way to see the project connections as a connected whole c.f. a tree

In summary, getting a coherent overview of "what's been designed" and "what has been run" currently requires many commands, copy-pasting many IDs, and obtaining overlapping outputs that partially duplicate each other.

Describe the solution you'd like

A CLI method to get a fast, project (context) overview that answers in one place:

1. What spaces exist and enough design context to understand and compare them without dumping full configs (experiment types, entity space size, key parameters).
2. Measurement and execution activity for those spaces and their associated operations — at a useful level of detail, not only percentages.
3. (Optionally), a relationship view across the main resource types (samplestore → spaces → operations → datacontainers) so I can see how resource connect without issuing a separate query per resource.

and that is responsive on real projects.

As a starter I imagine an enhanced ado get e.g. that integrates partial or fully ado show details output, along with an enhanced ado show related (maybe as ado tree)

Describe alternatives you've considered

• Status quo with tooling around it — shell scripts or notebooks that call the CLI repeatedly. This works but preserves the fragmentation, duplication, and latency I want to avoid.
• Dedicated ado project status or ado overview command — a single aggregate entry point, keeping ado get for raw listing and ado show for deep dives, so the separation of concerns is clear. However need a clean separation of the commands.
  - Unify and clarify ado show summary vs ado show details — one command with explicit modes, or a cleaner split between "tabular, many spaces" and "narrative, one resource," so overlap is reduced and multi-space overviews (including "all spaces in project") are first-class without hand-assembling ID lists.

Additional context

• Key "HUD" metrics: A core part of a workable solution is deciding what the key metrics are and what the boundary is between HUD metrics and deep dive metrics -> suggest counting/statistics like metrics can be HUD, anything requiring fetching of actual measurements is not.
• Overlap today: ado show summary (table/CSV/Markdown, multiple spaces via IDs or label filters) and ado show details (one space or operation, narrative) both address "how much is measured / what's related," so you end up moving between them for similar questions.
• ado show details and ado show related both surfaces relationships for a single anchor resource separately.
• ado get today: The default space table is intentionally minimal (identifier, name, age); --details adds some columns. Because ado get options are shared across resource kinds, adding expensive or resource-specific columns is awkward without affecting every get use case.
• Performance: Space-oriented summaries currently build full in-memory views to compute counts and percentages, which does not scale comfortably across many spaces. Operation paths already use SQL aggregation for entity statistics; space-level and cross-space bulk statistics likely need similar treatment to keep a project overview responsive.
• Relationship visibility: Related resources are currently resolved per resource through the metastore; a project-level view implies composing those edges in bulk rather than repeated single-resource calls from the user side.

[michael-johnston]

@AlessandroPomponio FYI

[michael-johnston]

@danielelotito @christian-pinto @mgazz @srikumar003 @VassilisVassiliadis

[VassilisVassiliadis]

Most of the times I use tools like ado summary and the like it's to find the details of experiments I ran in the past. The proposed feature definitely helps but ultimately what I really want is a super easy way to query. I know there already exists a query command but it's not intuitive. I always have to look up the docs and think about jsonpaths etc.

I'd prefer a feature where I could point ado to a discoveryspace YAML file and say "list all the spaces that are a superset of this space" etc

[AlessandroPomponio]

I'd prefer a feature where I could point ado to a discoveryspace YAML file and say "list all the spaces that are a superset of this space" etc

It's already there and it's ado get space --matching-space https://ibm.github.io/ado/getting-started/ado/#searching-and-filtering

[VassilisVassiliadis]

Perfect! Then the proposed feature in combination with ado get space --matching-space is what I'm looking for when I set out to do this kind of analysis

[danielelotito]

I agree with your observations, Michael, I first comment some of your points and then add mine.

Comments:
The main issue I also face is that a clear separation of scope/purpose of different commands is missing.

• ado get for raw listing and ado show for deep dives
• ado show details and ado show related and ado show summary, should be clearly defined
  • In relation to this I am also concerned about performance (I could suggest to keep one command to have answers faster)

I do not like it "Status quo with tooling around it", you have to maintain and it does not really solve the problem imo.

My observations:

• Sometimes I want info and I need to set the output format either because without the -o flag I do not get all info or because the information printed on the terminal is not easily consumable.
  • NOTE: often this does not depend on ado, but on the fact that the terminal I use has small space dedicated to it (see below), I can increase the window size and rerun the command. In the screenshot below I cannot copy the operation ID without running another command.

If I modify the terminal size after I run the program the table loses formatting and is not consumable anymore.
So I end up generating tons of files (whose name is difficult to interpret because it contains the hash, so I even spend time to rename it if I need it) and I pay the additional effort to delete them (if needed).
I do not have a solution to this.

[michael-johnston]

@danielelotito FYI we will by end May force users to set unique ids for resources themselves which should help with the naming issue.

What info do you need btw - is it common enough we could create an operator that wraps it. Then you apply the operator to the space and there is no intermediate file?

@AlessandroPomponio Is there a way with Rich to ensure the identifier columns are not truncated?

[AlessandroPomponio]

@AlessandroPomponio Is there a way with Rich to ensure the identifier columns are not truncated?

I think we can set a minimum width for that column, but it might be a bit of a hack, as we go from dataframe to rich table. We'll likely have to have an if that checks if the column name is identifier