Skip to content

Fix extra_files paths for subfolder Quarto documents#762

Open
anders-wiggers wants to merge 2 commits intoposit-dev:mainfrom
anders-wiggers:main
Open

Fix extra_files paths for subfolder Quarto documents#762
anders-wiggers wants to merge 2 commits intoposit-dev:mainfrom
anders-wiggers:main

Conversation

@anders-wiggers
Copy link
Copy Markdown

@anders-wiggers anders-wiggers commented Mar 30, 2026

Intent

Fix incorrect file paths in the manifest when rendering standalone Quarto documents whose .qmd file and extra_files are located in a subdirectory. Previously, the subdirectory name could be duplicated when joining base_dir and rel_path, causing extra files to be resolved to non-existent paths (e.g. subdir/subdir/file.ext).

Type of Change

  • Bug Fix
  • New Feature
  • Breaking Change

Approach

For standalone Quarto documents (the else branch where file_or_directory is not a directory):

  • Compute a common base directory across the main .qmd file and all extra_files:

    all_files = [file_or_directory] + extra_files
    abs_paths = [os.path.abspath(p) for p in all_files]
    base_dir = os.path.commonpath(abs_paths)
  • Convert all relevant file paths to be relative to this common base_dir:

    relevant_files = [os.path.relpath(p, base_dir) for p in abs_paths]
  • Keep the directory-based Quarto project branch (isdir(file_or_directory)) unchanged, still delegating to _create_quarto_file_list.

This ensures:

  • When both the .qmd and extra_files live in the same subfolder (e.g. subdir/report.qmd and subdir/data.csv) and extra_files is specified with the subfolder component (["subdir/data.csv"]), the manifest now uses:

    • base_dir = <common root>
    • rel_path = "subdir/data.csv"
      avoiding the duplicated subdir/subdir path.
  • Extra files located outside the .qmd’s subdirectory continue to work: the common path becomes the deepest shared parent directory, and all paths are consistently relative to that parent.

Directions for Reviewers

To validate manually:

  1. Create a structure like:

    project-root/
      subdir/
        report.qmd
        test/
          data.csv
    
  2. From project-root, invoke the packaging logic with:

    • file_or_directory="subdir/report.qmd"
    • extra_files=["subdir/test/data.csv"]
  3. Inspect the generated manifest (or log / debug around manifest_add_file) and verify:

    • data.csv is added once.
    • The final path used to locate data.csv contains subdir/test/data.csv (not subdir/subdir/test/data.csv).

Checklist

  • I have updated CHANGELOG.md to cover notable changes.
  • I have updated all related GitHub issues to reflect their current state.
  • I have run the rsconnect-python-tests-at-night workflow in Connect against this feature branch.

refactoring of extra_files to handle building bundle for deployment which done in a sub-directory.
Copy link
Copy Markdown
Collaborator

@jonkeane jonkeane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for submitting a PR for this. Overall this looks ok, but would you mind also adding some tests to confirm this behavior?

I suspect using something like

def test_make_quarto_manifest_project_with_extra_files(self):
temp_proj = tempfile.mkdtemp()
# include extra_files parameter
fp = open(join(temp_proj, "a"), "w")
fp.write("This is file a\n")
fp.close()
fp = open(join(temp_proj, "b"), "w")
fp.write("This is file b\n")
fp.close()
fp = open(join(temp_proj, "c"), "w")
fp.write("This is file c\n")
fp.close()
manifest, _ = make_quarto_manifest(
temp_proj,
{
"quarto": {"version": "0.9.16"},
"engines": ["jupyter"],
"config": {"project": {"title": "quarto-proj-py"}, "editor": "visual", "language": {}},
},
AppModes.SHINY_QUARTO,
None,
["a", "b", "c"],
[],
None,
)
if sys.platform == "win32":
a_hash = "f4751c084b3ade4d736c6293ab8468c9"
b_hash = "4976d559975b5232cf09a10afaf8d0a8"
c_hash = "09c56e1b9e6ae34c6662717c47a7e187"
else:
a_hash = "4a3eb92956aa3e16a9f0a84a43c943e7"
b_hash = "b249e5b536d30e6282cea227f3a73669"
c_hash = "53b36f1d5b6f7fb2cfaf0c15af7ffb2d"
self.assertEqual(
manifest,
{
"version": 1,
"metadata": {"appmode": "quarto-shiny"},
"quarto": {"version": "0.9.16", "engines": ["jupyter"]},
"files": {
"a": {"checksum": a_hash},
"b": {"checksum": b_hash},
"c": {"checksum": c_hash},
},
},
)
as inspiration would be a good starting point.

base_dir = os.path.commonpath(abs_paths)

# Store paths relative to base_dir
relevant_files = [os.path.relpath(p, base_dir) for p in abs_paths]
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I reading this correctly that this is using absolute paths at first then shifts them to relative paths (relative to the base_dir)? Is there a way to avoid that back and forth and just go directly to paths relative to the base_dir? It's not the end of the world if not, just seems like a bit of extra back and forth that is slightly distracting.

Comment on lines -1446 to +1449
manifest_add_buffer(manifest, environment.filename, environment.contents)

manifest_add_buffer(manifest, environment.filename, environment.contents)
for rel_path in relevant_files:
manifest_add_file(manifest, rel_path, base_dir)

manifest_add_file(manifest, rel_path, base_dir)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are just whitespace changes, yeah?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants