Skip to content

Updater: change excluded repos handling#719

Open
sale3 wants to merge 5 commits intomasterfrom
sbojanic/change-excluded-target-repos-handling
Open

Updater: change excluded repos handling#719
sale3 wants to merge 5 commits intomasterfrom
sbojanic/change-excluded-target-repos-handling

Conversation

@sale3
Copy link
Copy Markdown
Contributor

@sale3 sale3 commented Apr 1, 2026

Description (e.g. "Related to ...", etc.)

Target repo exclusion is set only in clone operation and the pattern is stored in last_validated_commit file.
Excluded target repos' last validated commits are set to None.
--exclude-target is removed in favor of --exclude-filter.
--exclude-filter is extended to filter repos by name to match --exclude-target implementation.

In auth_repo.py:
Removed excluded repos handling for methods targets_data_by_auth_commit and sorted_commits_and_branches_per_repositories since it is not used anywhere. Introducing --exclude-filter would lead to coupling with repositoriesdb.

Tests:
Removed test_update_after_update_with_exclude, test_update_after_update_with_exclude_with_invalid_commit and test_full_update_after_partial_update since update with exclude is no longer feasible.
Added verification for expected last_validated_commit entires - verify_excluded_lvc_entries function.

Closes: #716

Code review checklist (for code reviewer to complete)

  • Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
  • Title summarizes what is changing
  • Commit messages are meaningful (see this for details)
  • Tests have been included and/or updated, as appropriate
  • Docstrings have been included and/or updated, as appropriate
  • Changelog has been updated, as needed (see CHANGELOG.md)

@sale3 sale3 requested review from n-dusan and renatav April 1, 2026 23:32
@sale3 sale3 self-assigned this Apr 1, 2026
@sale3 sale3 force-pushed the sbojanic/change-excluded-target-repos-handling branch 4 times, most recently from 68cb5c0 to bcca125 Compare April 2, 2026 00:00
sale3 added 3 commits April 1, 2026 20:02
Target repo exclusion is set only in clone operation and the pattern is stored in last_validated_commits file.
Excluded repos' last validated commits are set to None.
--exclude-target is removed in favor of --exclude-filter.
--exclude-filter is extended to filter repos by name to match --exclude-target implementation.
@sale3 sale3 force-pushed the sbojanic/change-excluded-target-repos-handling branch from 0fbb74f to ebc018c Compare April 2, 2026 00:03
last_validated_data = client_auth_repo.last_validated_data
assert last_validated_data is not None
if last_validated_data.get("exclude_filter") is not None:
assert last_validated_data["exclude_filter"]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assert does not really check anything given the if condition. Maybe if excluded: assert "exclude_filter" in last_validated_data else assert exclude_filter" not in last_validated_data would be more useful.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy was complaining if there was no null check, so that's why I put it like that.

all_repositories = repositoriesdb.load_repositories_json(
self.state.users_auth_repo
)
if all_repositories is not None:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was my old way of handling this, so without saving exclude_filter. Just look for repositories not in last_validated_data and skip them. If are going to start storing exclude_filter, we should either validate rely on it primarily and just check if repositories not in last validated data and not on disk match the filter? What if there is a discrepancy here? Do we even need excluded_target_globs if we have exclude_filter? I added that automatic detection of excluded target repositories since there was no way to read this information.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that we can get rid of this completely. I think that it's not likely to happen that repositories.json doesn't have one of the target repos as an entry?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It very unlikely, but I wouldn't make that assumption.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 6, 2026

CLA assistant check
All committers have signed the CLA.

@sale3 sale3 force-pushed the sbojanic/change-excluded-target-repos-handling branch from 0a44f03 to 4aef841 Compare April 6, 2026 17:37
Copy link
Copy Markdown
Collaborator

@renatav renatav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and works well. Found a minor edge-case, might not even be addressing now. When you clone with an exclude-filter and then remove the filter line from the last_validated_commit file and run update, it clones the missing repository but prints All repositories are up-to-date with no changes or errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Updater: change excluded target repos handling

3 participants