Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/workflows/validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,32 +75,32 @@ jobs:
echo "" >> $GITHUB_STEP_SUMMARY

if [ -f validation_reference_results.json ]; then
python -c "
python -c '
import json
import sys

with open('validation_reference_results.json') as f:
with open("validation_reference_results.json") as f:
results = json.load(f)

summary = results['summary']
targets = results['targets']
summary = results["summary"]
targets = results["targets"]

print(f\"**Test Cases Run:** {summary['test_cases_run']}\")
print(f\"**Total Comparisons:** {summary['total']}\")
print(f\"**Passed:** {summary['passed']} ({summary['pass_rate']:.1f}%)\")
print(f\"**Failed:** {summary['failed']}\")
print(f"**Test Cases Run:** {summary[\"test_cases_run\"]}")
print(f"**Total Comparisons:** {summary[\"total\"]}")
print(f"**Passed:** {summary[\"passed\"]} ({summary[\"pass_rate\"]:.1f}%)")
print(f"**Failed:** {summary[\"failed\"]}")
print()

pass_rate = summary['pass_rate']
if pass_rate >= targets['excellent_pass_rate']:
print(f\"✅ **EXCELLENT** - Pass rate {pass_rate:.1f}% exceeds target {targets['excellent_pass_rate']}%\")
elif pass_rate >= targets['target_pass_rate']:
print(f\"✅ **VERY GOOD** - Pass rate {pass_rate:.1f}% exceeds target {targets['target_pass_rate']}%\")
elif pass_rate >= targets['minimum_pass_rate']:
print(f\"✅ **PASSED** - Pass rate {pass_rate:.1f}% meets minimum {targets['minimum_pass_rate']}%\")
pass_rate = summary["pass_rate"]
if pass_rate >= targets["excellent_pass_rate"]:
print(f"✅ **EXCELLENT** - Pass rate {pass_rate:.1f}% exceeds target {targets[\"excellent_pass_rate\"]}%")
elif pass_rate >= targets["target_pass_rate"]:
print(f"✅ **VERY GOOD** - Pass rate {pass_rate:.1f}% exceeds target {targets[\"target_pass_rate\"]}%")
elif pass_rate >= targets["minimum_pass_rate"]:
print(f"✅ **PASSED** - Pass rate {pass_rate:.1f}% meets minimum {targets[\"minimum_pass_rate\"]}%")
else:
print(f\"❌ **BELOW TARGET** - Pass rate {pass_rate:.1f}% below minimum {targets['minimum_pass_rate']}%\")
" >> $GITHUB_STEP_SUMMARY
print(f"❌ **BELOW TARGET** - Pass rate {pass_rate:.1f}% below minimum {targets[\"minimum_pass_rate\"]}%")
' >> $GITHUB_STEP_SUMMARY
else
echo "❌ Validation failed to produce results" >> $GITHUB_STEP_SUMMARY
fi
Expand Down
14 changes: 7 additions & 7 deletions NEXT_STEPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ DVOACAP-Python v1.0.1 is production-ready with **86.6% validation pass rate** ac

## ~~Priority 1: Fix Phase 5 Integration (Weeks 1-2)~~ ✅ **COMPLETED**

**Status:** Phase 5 validation achieved **83.8% pass rate** (181/216 tests), exceeding the 80% target.
**Status:** Phase 5 validation achieved **86.6% pass rate** (226/261 tests), exceeding the 85% target.

**Completed Work:**
- ✅ Reliability calculation verified against FORTRAN RELBIL.FOR
Expand All @@ -69,7 +69,7 @@ DVOACAP-Python v1.0.1 is production-ready with **86.6% validation pass rate** ac
- ✅ All core algorithms match FORTRAN reference

**Results:**
- Tangier → Belgrade test path: 83.8% pass rate (181/216 comparisons)
- Total validation: 86.6% pass rate (226/261 comparisons across 11 test cases)
- Predictions show valid reliability percentages (0-100%)
- No crashes or exceptions on valid inputs
- Signal strength predictions in expected range
Expand All @@ -85,7 +85,7 @@ DVOACAP-Python v1.0.1 is production-ready with **86.6% validation pass rate** ac
**Completed Work:**
- ✅ Created reference validation test suite (`test_voacap_reference.py`)
- ✅ Established tolerance specifications (SNR ±10 dB, Reliability ±20%, MUF ±2 MHz)
- ✅ Achieved 83.8% pass rate on baseline test path
- ✅ Achieved 86.6% pass rate across 11 diverse test paths
- ✅ CI/CD workflow implemented (`.github/workflows/validation.yml`)
- ✅ Validation status badge added to README

Expand Down Expand Up @@ -493,7 +493,7 @@ DVOACAP-Python v1.0.1 is production-ready with **86.6% validation pass rate** ac
### ~~Weeks 1-2: Critical Bug Fixes~~ ✅ **COMPLETED**
- ✅ Fixed Phase 5 reliability calculation
- ✅ Validated signal strength computations
- ✅ Basic predictions working correctly (83.8% pass rate)
- ✅ Basic predictions working correctly (86.6% pass rate)
- **Milestone:** Predictions show >0% reliability, one path validates ✓

### ~~Weeks 3-4: Systematic Validation~~ ✅ **COMPLETED**
Expand Down Expand Up @@ -625,7 +625,7 @@ python validate_predictions.py

---

**Last Updated:** 2025-11-15
**Status:** Phase 5 complete (83.8% validation). Focus: Expand test coverage, documentation maintenance
**Last Updated:** 2025-11-18
**Status:** Phase 5 complete (86.6% validation, exceeds 85% target). v1.0.1 released with 2.3x performance boost. Focus: Documentation polish, PyPI preparation

**Next Review:** After expanding test coverage to 7+ diverse paths
**Next Review:** Before PyPI public release
42 changes: 21 additions & 21 deletions wiki/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,11 @@ DVOACAP-Python is a Python port of the DVOACAP HF propagation prediction engine,

### How accurate is DVOACAP-Python?

**Current validation status:**
**Current validation status (v1.0.1):**
- **Phase 1-4 (Geometry, Solar, Ionosphere, Raytracing):** >95% validation pass rate
- **Phase 5 (Signal Predictions):** 83.8% validation pass rate
- **Phase 5 (Signal Predictions):** 86.6% validation pass rate

This means 83.8% of predictions match reference VOACAP output within acceptable tolerances. The remaining 16.2% show discrepancies being investigated and resolved.
This means 86.6% of predictions match reference VOACAP output within acceptable tolerances. The remaining 13.4% show minor discrepancies at edge cases.

See [Validation Status](Validation-Status) and [Known Issues](Known-Issues) for details.

Expand Down Expand Up @@ -106,12 +106,12 @@ Yes! DVOACAP-Python is open source under the **MIT License**. You can use it fre
### What are the system requirements?

**Minimum:**
- Python 3.8 or higher
- Python 3.11 or higher
- 200 MB disk space
- 512 MB RAM

**Recommended:**
- Python 3.10 or higher
- Python 3.11 or higher
- 500 MB disk space (with dependencies)
- 1 GB RAM

Expand Down Expand Up @@ -155,15 +155,15 @@ The only data files needed are the CCIR/URSI coefficient maps, which are include

### Can I install DVOACAP-Python with pip from PyPI?

**Not yet.** DVOACAP-Python is currently in active development and not published to PyPI. Install from source using:
**Not yet.** DVOACAP-Python v1.0.1 is production-ready but not yet published to PyPI. Install from source using:

```bash
pip install -e .
```

Once the project reaches v1.0, it will be published to PyPI for easier installation with:
PyPI publication is planned for a future release:
```bash
pip install dvoacap # Future release
pip install dvoacap # Planned future release
```

---
Expand Down Expand Up @@ -277,16 +277,16 @@ print(f"Long path: {long_distance:.0f} km")

## Accuracy & Validation

### Why is the validation pass rate only 83.8%?
### Why is the validation pass rate 86.6%?

The 83.8% pass rate for Phase 5 indicates that **83.8% of predictions match reference VOACAP within tolerances**. The remaining 16.2% show discrepancies due to:
The 86.6% pass rate for Phase 5 indicates that **86.6% of predictions match reference VOACAP within tolerances** across 11 diverse test cases. The remaining 13.4% show minor discrepancies due to:

1. **Numerical differences** - Python vs. Pascal floating-point operations
2. **Mode selection variations** - Different propagation mode choices
3. **Edge cases** - Extreme solar conditions, high latitudes, very short/long paths
4. **Over-the-MUF handling** - Different approaches when frequency > MUF

**Important:** Even "failed" test cases typically show SNR differences < 3 dB, which is operationally acceptable for amateur radio.
**Important:** This validation rate exceeds the 85% target threshold and is considered production-ready. Even "failed" test cases typically show acceptable operational differences.

See [Known Issues](Known-Issues) for details.

Expand All @@ -306,22 +306,22 @@ The dashboard includes optional PSKreporter validation.

## Performance

### Why are predictions slow?
### How fast are predictions in v1.0.1?

DVOACAP predictions are **computationally intensive** because they perform full raytracing through a 3D ionospheric model.
DVOACAP v1.0.1 delivers a **2.3x performance boost** over v1.0.0 through algorithmic optimizations. While predictions remain computationally intensive (full raytracing through 3D ionospheric model), they are now much faster.

**Typical times:**
- Single prediction: ~200-500 ms
- Full band sweep (7 frequencies): ~2-3 seconds
- 24-hour forecast (12 time points): ~25-35 seconds
- Full dashboard (10 regions × 7 bands): ~60-90 seconds
**Typical times (v1.0.1):**
- Single prediction: ~4 ms (was ~8 ms in v1.0.0)
- Multi-frequency (9 frequencies): ~48 ms (was ~111 ms, 2.3x faster)
- 24-hour scan: ~118 ms (was ~282 ms, 2.4x faster)
- Area coverage (100 predictions): ~350 ms (was ~820 ms, 2.3x faster)

**Why it's slow:**
**Computational bottlenecks:**
- Iterative raytracing (Phase 4)
- CCIR coefficient processing (Phase 3)
- Python overhead vs. compiled code

See [Performance Tips](Performance-Tips) for optimization strategies.
See [Performance Tips](Performance-Tips) for optimization strategies and benchmarks.

---

Expand Down Expand Up @@ -442,7 +442,7 @@ See [Contributing Guide](Contributing) for guidelines.
### What development tools do I need?

**Essential:**
- Python 3.8+
- Python 3.11+
- pytest (testing)
- black (formatting)
- flake8 (linting)
Expand Down
6 changes: 3 additions & 3 deletions wiki/Getting-Started.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This guide will help you install DVOACAP-Python and run your first propagation p

## Prerequisites

- Python 3.8 or higher
- Python 3.11 or higher
- pip (Python package manager)
- git (for cloning the repository)

Expand Down Expand Up @@ -96,7 +96,7 @@ pytest tests/test_path_geometry.py -v
This example computes ionospheric parameters at a single location:

```python
from dvoacap import FourierMaps, ControlPoint, GeographicPoint, compute_iono_params
from dvoacap import FourierMaps, ControlPoint, IonoPoint, compute_iono_params
import math

# Load CCIR/URSI ionospheric maps
Expand All @@ -105,7 +105,7 @@ maps.set_conditions(month=6, ssn=100, utc_fraction=0.5) # June, SSN=100, noon U

# Create control point at Philadelphia
pnt = ControlPoint(
location=GeographicPoint.from_degrees(40.0, -75.0),
location=IonoPoint.from_degrees(40.0, -75.0),
east_lon=-75.0 * math.pi/180,
distance_rad=0.0,
local_time=0.5, # Noon local
Expand Down
20 changes: 10 additions & 10 deletions wiki/Known-Issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,20 +64,20 @@ Focus on SNR and reliability values rather than mode names.

## Performance Limitations

### Slow Prediction Generation
### Prediction Performance (Optimized in v1.0.1)

**Status:** Known Limitation
**Impact:** Medium
**Status:** Significantly Improved
**Impact:** Low (after v1.0.1 optimization)
**Affects:** Dashboard generation, batch processing

**Description:**
Full VOACAP predictions are computationally intensive. Generating predictions for 10 regions × 7 bands × 12 time points can take 60-90 seconds.
Version 1.0.1 introduced comprehensive algorithmic optimizations providing a 2.3x speedup. Full VOACAP predictions remain computationally intensive but are now much faster.

**Timing Breakdown:**
- Single frequency prediction: ~200-500ms
- Full band sweep (7 frequencies): ~2-3 seconds
- 24-hour forecast (12 time points): ~25-35 seconds
- Full dashboard (10 regions × 7 bands × 12 hours): ~60-90 seconds
**Timing Breakdown (v1.0.1):**
- Single frequency prediction: ~4ms (was ~8ms in v1.0.0)
- Full band sweep (9 frequencies): ~48ms (was ~111ms, 2.3x faster)
- 24-hour forecast (12 time points): ~118ms (was ~282ms, 2.4x faster)
- Area coverage (100 predictions): ~350ms (was ~820ms, 2.3x faster)

**Root Causes:**
1. Raytracing calculations (Phase 4) - iterative path finding
Expand Down Expand Up @@ -380,4 +380,4 @@ If you encounter issues not listed here:
---

**Last Updated:** 2025-11-18
**Project Status:** v1.0.0 Production Ready (86.6% validation accuracy)
**Project Status:** v1.0.1 Production Ready (86.6% validation accuracy, 2.3x performance boost)