refactor: client-first architecture, extended builder API, and CLI improvements

.github/copilot-instructions.md

@@ -8,9 +8,8 @@ OneIO is a Rust library providing unified IO operations for reading and writing
 ### Before Committing - Always Run:
 1. **Format code**: `cargo fmt`
 2. **Run linter**: `cargo clippy --all-features` and fix all warnings
-3. **Update README if lib.rs docs changed**: `cargo readme > README.md`
-4. **Update CHANGELOG.md**: Add entries under `[Unreleased]` section
-5. **Run tests**: `cargo test --all-features`
+3. **Update CHANGELOG.md**: Add entries under `[Unreleased]` section
+4. **Run tests**: `cargo test --all-features`
 
 ### Formatting Rules
 - Always run `cargo fmt` before completing any task
@@ -19,10 +18,8 @@ OneIO is a Rust library providing unified IO operations for reading and writing
 - Follow Rust standard formatting conventions
 
 ### Documentation Requirements
-- When modifying `src/lib.rs` documentation, always regenerate README:
-  ```bash
-  cargo readme > README.md
-  ```
+- Keep lib.rs documentation concise and API-focused (for docs.rs)
+- README.md is maintained separately with more detailed examples
 - Keep documentation examples up-to-date
 - Add doc comments for all public APIs
 - Include usage examples in module-level documentation
@@ -135,7 +132,7 @@ cargo test --features http,gz,bz
 1. Add feature flag to `Cargo.toml`
 2. Implement feature-gated code with `#[cfg(feature = "...")]`
 3. Add tests for the feature
-4. Document in lib.rs and regenerate README
+4. Document in lib.rs (API docs) and README.md (user guide)
 5. Update CHANGELOG.md
 6. Add example if applicable
 
@@ -179,4 +176,4 @@ cargo test --features http,gz,bz
 - **Always run cargo fmt and cargo clippy before committing**
 - **Always update CHANGELOG.md with changes**
 - **No emojis in commits or PRs**
-- **Regenerate README.md when lib.rs docs change**
+- **Maintain both lib.rs (API docs) and README.md (user guide) separately**

.github/workflows/release.yml

@@ -17,9 +17,6 @@ jobs:
       - name: Run format check
         run: cargo fmt --check
 
-      - name: Run cargo-readme check
-        run: cargo install cargo-readme && cargo readme > TMP_README.md && diff -b TMP_README.md README.md
-
   create-release:
     needs: release-format-check
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -4,16 +4,58 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+### Breaking changes
+- `OneIoError` is now `#[non_exhaustive]`; `match` expressions without a wildcard `_` arm will fail to compile
+- `OneIoBuilder::header()` now accepts typed `HeaderName`/`HeaderValue` (infallible) instead of `(K, V) -> Result<Self>`
+- `OneIoBuilder::user_agent()` now accepts a typed `HeaderValue` (infallible) instead of `V -> Result<Self>`
+- `oneio::download()` no longer accepts an `Option<reqwest::blocking::Client>` parameter
+- `oneio::remote` module is now `pub(crate)`; `create_client_with_headers` is deprecated (use `OneIo::builder().header_str()`)
+- `ProgressReader` and `ProgressCallback` are no longer part of the public API
+
 ### Changed
+- Flattened module layout: `src/oneio/` sub-directory removed; all modules are now at `src/` level
+- `OneIo` and `OneIoBuilder` are now the primary API surface; free-standing functions delegate to a shared default client
+- Compression detection strips URL query parameters and fragments before reading the file extension
+- `download_with_retry()` uses exponential backoff between retry attempts (100ms × 2^attempt, capped at 6400ms)
 - Stream cache writes to disk via `std::io::copy` instead of buffering the full payload in memory
 - `download_async()` now preserves raw bytes, matching `download()`
 - Default blocking HTTP clients are reused across reads and content-length probes
+- Stateless read and download helpers now delegate to a reusable `OneIo` client internally
 - S3 status failures now use structured errors instead of string parsing
 - S3 readers now stream data through a bounded channel instead of materializing the full object in memory
 
 ### Added
+- `OneIoBuilder::header_str(name, value)` — string convenience for adding headers (panics on invalid input, matching reqwest convention)
+- `OneIoBuilder::configure_http(f)` — escape hatch for setting any `reqwest::blocking::ClientBuilder` option
+- `OneIoBuilder::timeout()`, `connect_timeout()` — request and connect timeouts
+- `OneIoBuilder::proxy()`, `no_proxy()` — proxy configuration
+- `OneIoBuilder::redirect()` — redirect policy
+- `OneIoBuilder::add_root_certificate_pem()`, `add_root_certificate_der()` — load CA certs from raw bytes
+- `OneIo::get_reader_with_type(path, compression)` — explicit compression override, useful for URLs with query parameters
+- `OneIo::from_client(client)` — construct a `OneIo` from an existing `reqwest::blocking::Client`
+- `OneIoError::NetworkWithContext` — network errors now carry the URL that failed
+- `OneIoError::InvalidHeader`, `OneIoError::InvalidCertificate` — specific error variants for header and certificate construction failures
+- `ONEIO_CA_BUNDLE` environment variable — path to a PEM file added to the HTTP trust store on startup
+- `get_cache_reader()` free-standing shortcut kept at crate root for convenience
 - Added `bzip2_decompress` benchmark coverage
 - Added a benchmark helper script for comparing gzip backend feature flags and bz2 decompression
+- Added reusable `OneIo` and `OneIoBuilder` APIs for sharing headers and TLS certificate configuration across requests
+
+### CLI
+- Added `-H`/`--header` flag for custom HTTP headers (`"Name: Value"` or `"Name:Value"`), can be repeated
+- Added `--compression` flag to override compression detection (gz, bz2, lz4, xz, zst); no effect with `--download`
+- Added progress bar for `--download` and `s3 download`, shown when stderr is a terminal; uses spinner when file size is unknown
+- Added `s3 download <BUCKET> <S3_PATH> [--outfile]` subcommand
+- Fixed S3 upload syntax: `oneio s3 upload <LOCAL_FILE> <BUCKET> <S3_PATH>` (local file is now the first positional arg under the subcommand)
+- Terminal detection uses `std::io::IsTerminal` from the standard library; no extra dependency needed
+- Added `indicatif` to the `cli` feature for progress bars
+
+### Bug fixes
+- LZ4 compressed writes were silently truncated: `lz4::Encoder` has no `Drop` impl and requires an explicit `finish()` call to write the end-of-stream marker. Fixed with a `Lz4Writer` wrapper that calls `finish()` on drop.
+
+### Documentation
+- `lib.rs` docstring documents the `native-tls` feature as the fix for Cloudflare WARP and corporate proxy environments
+- `ONEIO_ACCEPT_INVALID_CERTS` and `ONEIO_CA_BUNDLE` environment variables documented at crate root
 
 ## v0.20.1 -- 2025-12-18
 

Cargo.toml

@@ -53,6 +53,7 @@ rust-s3 = { version = "0.37", optional = true, default-features = false, feature
 # feature: cli
 clap = { version = "4.4", features = ["derive"], optional = true }
 tracing = { version = "0.1", optional = true }
+indicatif = { version = "0.18", optional = true }
 
 # feature: async (Phase 3)
 tokio = { version = "1.0", features = ["rt", "rt-multi-thread", "io-util", "fs"], optional = true }
@@ -94,7 +95,7 @@ json = ["serde", "serde_json"]
 digest = ["ring", "hex"]
 
 # CLI tool (includes common features)
-cli = ["clap", "tracing", "gz", "bz", "lz", "xz", "http", "s3", "digest"]
+cli = ["clap", "tracing", "indicatif", "gz", "bz", "lz", "xz", "http", "s3", "digest"]
 
 # Advanced: TLS selection (only if explicitly needed)
 native-tls = [