Updating code to release 0.2.1

.flake8

@@ -27,4 +27,5 @@ exclude =
     tracking*.json,
     conftest.py,
     playwright.config.py,
-    .csv
+    .csv,
+    .vulture_whitelist.py

.github/workflows/check_headers.yml

@@ -40,7 +40,7 @@ jobs:
           #   this file, may be copied, modified, propagated, or distributed except according to
           #   the terms contained in the file 'LICENCE.txt'."
 
-          MARKDOWN_HEADER="[//]: # (Copyright (c) European Space Agency, 2025.)
+          MARKDOWN_HEADER="[//]: # (Copyright © European Space Agency, 2025.)
           [//]: # ()
           [//]: # (This file is subject to the terms and conditions defined in file 'LICENCE.txt', which)
           [//]: # (is part of this source code package. No part of the package, including)

.github/workflows/dead_code.yml

@@ -0,0 +1,44 @@
+#   Copyright (c) European Space Agency, 2025.
+#
+#   This file is subject to the terms and conditions defined in file 'LICENCE.txt', which
+#   is part of this source code package. No part of the package, including
+#   this file, may be copied, modified, propagated, or distributed except according to
+#   the terms contained in the file 'LICENCE.txt'.
+name: Dead Code Detection
+
+on: [pull_request]
+
+jobs:
+  vulture-strict:
+    name: Vulture (100% confidence - blocking)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+      - name: Install vulture
+        run: pip install vulture>=2.10
+      - name: Run vulture (100% confidence)
+        run: |
+          echo "Running vulture dead code detection (100% confidence - blocking)..."
+          vulture cutana/ cutana_ui/ .vulture_whitelist.py --min-confidence 100
+
+  vulture-warnings:
+    name: Vulture (60% confidence - not required)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+      - name: Install vulture
+        run: pip install vulture>=2.10
+      - name: Run vulture (60% confidence)
+        run: |
+          echo "Running vulture dead code detection (60% confidence)..."
+          echo "This check fails if potential dead code is found, but is not required to pass."
+          echo ""
+          vulture cutana/ cutana_ui/ .vulture_whitelist.py --min-confidence 60

.github/workflows/ruff_imports.yml

@@ -0,0 +1,19 @@
+#   Copyright (c) European Space Agency, 2025.
+#
+#   This file is subject to the terms and conditions defined in file 'LICENCE.txt', which
+#   is part of this source code package. No part of the package, including
+#   this file, may be copied, modified, propagated, or distributed except according to
+#   the terms contained in the file 'LICENCE.txt'.
+name: Ruff Import Sorting Check
+
+on: [pull_request]
+
+jobs:
+  ruff-imports:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install ruff
+        run: pip install ruff
+      - name: Check import sorting with ruff
+        run: ruff check --select I --diff .

.gitignore

@@ -232,4 +232,8 @@ job_tracking.json
 cutana_output/
 test_workflow_*
 examples/*.csv
+paper_scripts/results
+paper_scripts/figures
+paper_scripts/latex
+paper_scripts/catalogues
 demo

.vulture_whitelist.py

@@ -0,0 +1,74 @@
+#   Copyright (c) European Space Agency, 2025.
+#
+#   This file is subject to the terms and conditions defined in file 'LICENCE.txt', which
+#   is part of this source code package. No part of the package, including
+#   this file, may be copied, modified, propagated, or distributed except according to
+#   the terms contained in the file 'LICENCE.txt'.
+"""
+Vulture whitelist - false positives for legitimate code.
+
+These entries represent code that IS used but vulture cannot detect the usage:
+- WCS attributes: Written by our code, read by external libraries (astropy, drizzle)
+- DotMap config: Accessed dynamically via attribute access
+- ipywidgets attributes: Set by our code, read by the UI framework
+- Public API: Exported for external use, documented in README
+
+Usage: vulture cutana cutana_ui .vulture_whitelist.py --min-confidence 60
+"""
+
+# WCS attributes - written and consumed by external libraries (astropy, drizzle)
+_.crpix  # noqa
+_.cdelt  # noqa
+_.crval  # noqa
+_.ctype  # noqa
+_.array_shape  # noqa
+
+# Config attributes accessed dynamically via DotMap
+_.num_unique_fits_files  # noqa
+_.preview_samples  # noqa
+_.preview_size  # noqa
+_.auto_regenerate_preview  # noqa
+_.created_at  # noqa
+
+# UI widget attributes (ipywidgets style/layout attributes set but read by framework)
+_.button_style  # noqa
+_.button_color  # noqa
+_.preview_cutouts  # noqa
+_.original_layout  # noqa
+_.channel_matrix  # noqa
+_.max_width  # noqa
+_.margin  # noqa
+_.crop_enable_label  # noqa
+_.disabled  # noqa
+_.default_filename  # noqa
+
+# UI methods used for polling/event handling
+get_processing_status  # noqa - called via asyncio polling from main_screen
+
+# UI style constants imported by test files that verify theming
+ESA_BLUE_DEEP  # noqa
+ESA_GREEN  # noqa
+ESA_RED  # noqa
+
+# PreviewCache class attribute - accessed dynamically within class methods
+_.config_cache  # noqa
+
+# StreamingOrchestrator public API - documented in README, used in examples/async_streaming.py
+init_streaming  # noqa - public API for batch streaming workflow
+next_batch  # noqa - public API for getting next batch of cutouts
+get_batch_count  # noqa - public API for getting total batch count
+get_batch  # noqa - public API for random access to batches
+
+# SystemMonitor utility methods - public API for resource monitoring
+check_memory_constraints  # noqa - utility for checking available memory
+estimate_memory_usage  # noqa - utility for estimating memory requirements
+record_resource_snapshot  # noqa - utility for recording resource history
+get_resource_history  # noqa - utility for retrieving resource history
+get_conservative_cpu_limit  # noqa - utility for conservative CPU allocation
+
+# UILogManager public API - imported and used by app.py, main_screen.py, start_screen.py
+setup_ui_logging  # noqa - public API for setting up UI logging with file handler
+set_console_log_level  # noqa - public API for dynamically changing console log level
+
+# Styles module public API - utility functions for UI scaling
+scale_vh  # noqa - public API for scaling viewport height values (symmetric with scale_px)

CHANGELOG.md

@@ -0,0 +1,62 @@
+[//]: # (Copyright © European Space Agency, 2025.)
+[//]: # ()
+[//]: # (This file is subject to the terms and conditions defined in file 'LICENCE.txt', which)
+[//]: # (is part of this source code package. No part of the package, including)
+[//]: # (this file, may be copied, modified, propagated, or distributed except according to)
+[//]: # (the terms contained in the file 'LICENCE.txt'.)
+
+# Changelog
+
+## [v0.2.1] – 2025-01-21
+
+### Changed
+- **Default max_workers** now uses available CPU count instead of hardcoded 16
+
+### Fixed
+- **Status panel worker display** now shows "16 workers" before processing starts instead of misleading "0/16 workers"
+- **Help panel README handling** now uses `importlib.metadata` to load main README from package metadata in pip-installed environments
+
+---
+
+## [v0.2.0] – 2025-01-12
+
+### Added
+- **Streaming mode** with `StreamingOrchestrator` for in-memory cutout processing using shared memory, enabling direct processing without disk I/O
+- **Flux-conserved resizing** using the drizzle algorithm to preserve photometric accuracy during image resampling
+- **Parquet input support** allowing source catalogues to be provided in Parquet format in addition to CSV
+- **Raw cutout extraction** (`cutout_only` mode) for outputting unprocessed cutouts directly from FITS tiles
+- **External FITSBolt configuration** support with TOML serialization for seamless integration with FITSBolt pipelines
+- **Log level selector** dropdown in the UI header for runtime log verbosity control
+- **Vulture dead code detection** CI workflow to identify and prevent unused code accumulation
+- **Ruff import sorting** CI check to enforce consistent import ordering across the codebase
+- **Comprehensive benchmarking suite** (`paper_scripts/`) for performance evaluation and reproducibility of paper results
+- **Async streaming example** (`examples/async_streaming.py`) demonstrating programmatic streaming mode usage
+
+### Changed
+- **Catalogue streaming architecture** with `CatalogueStreamer` enabling memory-efficient processing of catalogues with 10M+ sources through atomic tile batching
+- **Default output folder** changed from `cutana/output` to `cutana_output` for cleaner project structure
+- **Default resizing mode** changed to symmetric for more intuitive cutout dimensions
+- **Logging configuration** now follows loguru best practices: disabled by default, users opt-in via `logger.enable("cutana")`
+- **WCS handling** optimized for FITS output with correct pixel scale and WCS - no SIP distortions implemented
+- **Documentation** updated for Euclid DR1 compatibility with improved README and markdown formatting
+- **Source mapping output** now written as Parquet instead of CSV for better performance with large catalogues
+- **Dependencies** updated: `fitsbolt>=0.1.6`, `images-to-zarr>=0.3.5`, added `drizzle>=2.0.1`, `scikit-image>=0.21`
+
+### Fixed
+- **WCS pixel offset** corrected 1-based indexing and half-pixel offset issues affecting cutout positioning
+- **Subprocess logging** resolved ANSI escape codes and duplicate log folder creation
+- **Windows compatibility** fixed temp file permission issues in streaming mode tests
+- **Parquet file selection** in UI now properly filters and displays Parquet files
+- **Flux conservation integration** properly applied in `cutout_process_utils.py`
+- **Normalisation bypass** allowing `"none"` config value to skip normalisation entirely
+
+### Performance
+- **10x memory reduction** for large catalogue processing through true streaming implementation
+- **WCS computation optimisation** reducing overhead for FITS output generation
+- **Single-threaded FITSBolt** mode for improved stability in multi-process environments
+
+### Removed
+- **`JobCreator` class** and associated dead code identified through vulture static analysis
+- **Obsolete example notebooks** (`Cutana_IDR1_Setup.ipynb`, `backend_demo.ipynb`) replaced with updated documentation
+
+---

MANIFEST.in

@@ -0,0 +1,5 @@
+include README.md
+include LICENCE.txt
+include CHANGELOG.md
+recursive-include cutana_ui *.md
+recursive-include assets *.svg