diff --git a/.dockerignore b/.dockerignore deleted file mode 100644 index b694934fb..000000000 --- a/.dockerignore +++ /dev/null @@ -1 +0,0 @@ -.venv \ No newline at end of file diff --git a/Procfile b/Procfile deleted file mode 100644 index 2486669cb..000000000 --- a/Procfile +++ /dev/null @@ -1 +0,0 @@ -web: python3 -m transfers.transfer diff --git a/RENDER_DEPLOYMENT.md b/RENDER_DEPLOYMENT.md new file mode 100644 index 000000000..0110499ce --- /dev/null +++ b/RENDER_DEPLOYMENT.md @@ -0,0 +1,248 @@ +# Render Deployment Guide + +This guide explains how to deploy OcotilloAPI to Render using the `render.yaml` blueprint. + +## Overview + +The `render.yaml` file defines two complete environments: +- **Staging**: Auto-deploys from `staging` branch +- **Production**: Manual deploys from `main` branch (requires approval) + +## Architecture + +### Services Created + +1. **Web Services** (2) + - `ocotillo-api-staging` - Staging environment + - `ocotillo-api-production` - Production environment + +2. **Databases** (2) + - `ocotillo-db-staging` - PostgreSQL 17 with PostGIS + - `ocotillo-db-production` - PostgreSQL 17 with PostGIS + +3. **Environment Variable Groups** (3) + - `ocotillo-shared` - Common configuration for all environments + - `ocotillo-staging` - Staging-specific settings + - `ocotillo-production` - Production-specific settings + +## Initial Deployment + +### Step 1: Connect Repository + +1. Log in to [Render Dashboard](https://dashboard.render.com) +2. Click **"New Blueprint Instance"** +3. Connect your GitHub repository +4. Select the branch containing `render.yaml` (e.g., `render-deploy`) + +### Step 2: Configure Environment Variables + +Render will automatically create the environment variable groups, but you need to set values for `sync: false` variables: + +#### Staging Environment (`ocotillo-staging` group) + +Set these in the Render dashboard: + +```bash +# Authentik OAuth Configuration +AUTHENTIK_URL=https://your-staging-authentik-instance.com +AUTHENTIK_CLIENT_ID=your_staging_client_id +AUTHENTIK_AUTHORIZE_URL=https://your-staging-authentik-instance.com/application/o/authorize/ +AUTHENTIK_TOKEN_URL=https://your-staging-authentik-instance.com/application/o/token/ + +# Google Cloud Storage (if using assets) +GCS_BUCKET_NAME=your-staging-bucket +GOOGLE_APPLICATION_CREDENTIALS=/path/to/staging/credentials.json +``` + +#### Production Environment (`ocotillo-production` group) + +```bash +# Authentik OAuth Configuration +AUTHENTIK_URL=https://your-production-authentik-instance.com +AUTHENTIK_CLIENT_ID=your_production_client_id +AUTHENTIK_AUTHORIZE_URL=https://your-production-authentik-instance.com/application/o/authorize/ +AUTHENTIK_TOKEN_URL=https://your-production-authentik-instance.com/application/o/token/ + +# Google Cloud Storage (if using assets) +GCS_BUCKET_NAME=your-production-bucket +GOOGLE_APPLICATION_CREDENTIALS=/path/to/production/credentials.json +``` + +**Note**: `SESSION_SECRET_KEY` is auto-generated by Render (`generateValue: true`) + +### Step 3: Review and Deploy + +1. Review the services that will be created +2. Click **"Apply"** to create all resources +3. Render will: + - Create PostgreSQL databases with PostGIS extension + - Build Docker images + - Run database migrations + - Deploy the applications + +## Database Configuration + +### PostGIS Extension + +The `preDeployCommand` automatically installs PostGIS: +```bash +psql $DATABASE_URL -c "CREATE EXTENSION IF NOT EXISTS postgis;" +``` + +### Connection Details + +- **DATABASE_URL** is automatically provided by Render +- Connections are internal-only by default (`ipAllowList: []`) +- To allow external connections, add IP addresses to `ipAllowList` in `render.yaml` + +## Health Checks + +Both environments use `/health/ready` endpoint: +- Checks application responsiveness +- Verifies database connectivity +- Returns 200 if healthy, 503 if not ready + +## Deployment Workflow + +### Staging Deployments + +```bash +git checkout staging +git merge render-deploy # or your feature branch +git push origin staging +``` + +**Result**: Automatic deployment to staging environment + +### Production Deployments + +```bash +git checkout main +git merge staging +git push origin main +``` + +**Result**: Build triggered, but requires manual approval in Render dashboard + +## Environment-Specific Configuration + +### Staging +- `MODE=development` +- Auto-deploy enabled +- Authentication enabled (Authentik) +- Smaller instance size (starter plan) + +### Production +- `MODE=production` +- Manual deploy (requires approval) +- Authentication enabled (Authentik) +- Larger instance size (standard plan) + +## Service Plans + +Current configuration in `render.yaml`: + +| Service | Plan | Notes | +|---------|------|-------| +| Staging Web | Starter | Suitable for testing | +| Production Web | Standard | Recommended for production traffic | +| Staging DB | Standard | Can downgrade to Starter if needed | +| Production DB | Standard | Recommended for production data | + +To change plans, edit the `plan` field in `render.yaml`. + +## Scaling Configuration + +The application uses **Gunicorn with 4 workers** by default (configured in `entrypoint.sh`). + +To adjust workers: +1. Edit `entrypoint.sh` +2. Change `--workers 4` to desired number +3. Commit and push changes + +**Recommended**: 2-4 workers per container, scale horizontally by adding more instances. + +## Monitoring + +### Health Check Endpoints + +- `/health/live` - Basic liveness check +- `/health/ready` - Readiness check with DB connectivity +- `/health/status` - Detailed status with pool metrics + +### Logs + +Access logs in Render dashboard: +1. Navigate to your service +2. Click **"Logs"** tab +3. Filter by severity or search terms + +## Troubleshooting + +### Database Connection Issues + +Check if PostGIS extension is installed: +```bash +# Via Render Shell +psql $DATABASE_URL -c "SELECT PostGIS_Version();" +``` + +### Migration Failures + +Run migrations manually via Render Shell: +```bash +alembic upgrade head +``` + +### Port Binding Issues + +Ensure `$PORT` environment variable is being used (already configured in `entrypoint.sh`). + +## Updating the Blueprint + +After modifying `render.yaml`: + +1. Commit and push changes +2. Go to Render dashboard +3. Navigate to Blueprint instance +4. Click **"Sync"** to apply changes + +**Warning**: Some changes may cause service restarts. + +## Cost Optimization + +### Staging Environment + +Consider downgrading if budget-constrained: +- Web service: `starter` → `free` (with limitations) +- Database: `standard` → `starter` (1GB storage, no point-in-time recovery) + +### Production Environment + +Recommended minimum: +- Web service: `standard` (for multiple workers) +- Database: `standard` (for backups and PITR) + +## Security Considerations + +1. **Authentication**: Authentik OAuth is enabled by default +2. **Database**: Internal-only access (no public IP by default) +3. **Secrets**: All sensitive values use `sync: false` (not in git) +4. **HTTPS**: Automatically provided by Render + +## Next Steps + +After deployment: + +1. ✅ Verify health checks are passing +2. ✅ Test API endpoints +3. ✅ Configure custom domains (optional) +4. ✅ Set up monitoring/alerting +5. ✅ Configure backups schedule +6. ✅ Review and optimize database performance + +## Support + +- [Render Documentation](https://render.com/docs) +- [Render Status Page](https://status.render.com) +- [Community Forum](https://community.render.com) diff --git a/alembic/env.py b/alembic/env.py index f0bd9e778..fb33a6c57 100644 --- a/alembic/env.py +++ b/alembic/env.py @@ -56,7 +56,17 @@ def build_database_url(): return f"postgresql+pg8000://{user}@/{database}" return f"postgresql+pg8000://{user}:{password}@/{database}" - # Default/Postgres + # Check for DATABASE_URL first (Render/Heroku standard) + database_url = os.environ.get("DATABASE_URL", "") + if database_url: + # Convert to psycopg2 driver for alembic + if database_url.startswith("postgres://"): + return database_url.replace("postgres://", "postgresql+psycopg2://", 1) + elif database_url.startswith("postgresql://"): + return database_url.replace("postgresql://", "postgresql+psycopg2://", 1) + return database_url + + # Fall back to individual env vars (backward compatible) user = os.environ.get("POSTGRES_USER", "") password = os.environ.get("POSTGRES_PASSWORD", "") db = os.environ.get("POSTGRES_DB", "") diff --git a/api/health.py b/api/health.py new file mode 100644 index 000000000..4a2c77629 --- /dev/null +++ b/api/health.py @@ -0,0 +1,121 @@ +"""Health check endpoints for deployment monitoring and container orchestration.""" + +from datetime import datetime +from typing import Any + +from fastapi import APIRouter, HTTPException, status +from sqlalchemy import text + +from core.dependencies import session_dependency + +router = APIRouter(prefix="/health", tags=["Health"]) + + +@router.get("/live", summary="Liveness probe") +async def liveness_check() -> dict[str, str]: + """ + Basic liveness probe - checks if the application is running. + + This endpoint should always return 200 if the app is alive. + No dependencies checked - just confirms the web server is responding. + + Use this for Kubernetes/Docker liveness probes. + """ + return {"status": "ok"} + + +@router.get("/ready", summary="Readiness probe") +async def readiness_check(session: session_dependency) -> dict[str, Any]: + """ + Readiness probe - checks if the application is ready to serve traffic. + + Verifies: + - Application is running + - Database connection is available + + Returns 200 if ready, 503 if not ready. + Use this for Kubernetes/Docker readiness probes and load balancer health checks. + """ + try: + # Test database connectivity with a simple query + result = await session.execute(text("SELECT 1")) + result.scalar() + + return { + "status": "ready", + "database": "connected", + "checks": {"db_connection": True}, + } + except Exception as e: + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, + detail={ + "status": "not_ready", + "database": "disconnected", + "error": str(e), + "checks": {"db_connection": False}, + }, + ) + + +@router.get("/status", summary="Detailed status information") +async def status_check(session: session_dependency) -> dict[str, Any]: + """ + Detailed status endpoint - provides comprehensive health information. + + Returns: + - Application status + - Database connectivity and pool information + - Timestamp + - Additional service status + + Use this for monitoring dashboards and detailed health reports. + """ + timestamp = datetime.utcnow().isoformat() + + # Check database connectivity + db_status = "disconnected" + db_connected = False + pool_info = {} + + try: + result = await session.execute(text("SELECT 1")) + result.scalar() + db_status = "connected" + db_connected = True + + # Get connection pool information + pool = session.get_bind().pool + pool_info = { + "size": pool.size(), + "overflow": pool.overflow() if hasattr(pool, "overflow") else None, + "checked_in": pool.checkedin() if hasattr(pool, "checkedin") else None, + "checked_out": pool.checkedout() if hasattr(pool, "checkedout") else None, + } + # Remove None values + pool_info = {k: v for k, v in pool_info.items() if v is not None} + + except Exception as e: + db_status = f"error: {str(e)}" + + response = { + "status": "healthy" if db_connected else "degraded", + "timestamp": timestamp, + "database": { + "status": db_status, + "connected": db_connected, + }, + "services": {"admin": "available", "cors": "enabled"}, + } + + # Add pool info if available + if pool_info: + response["database"]["pool_info"] = pool_info + + # Return 503 if database is not connected + if not db_connected: + raise HTTPException( + status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=response + ) + + return response diff --git a/core/initializers.py b/core/initializers.py index 02ecc968f..5728b774d 100644 --- a/core/initializers.py +++ b/core/initializers.py @@ -108,6 +108,7 @@ def init_lexicon(path: str = None) -> None: def register_routes(app): from admin.auth_routes import router as admin_auth_router + from api.health import router as health_router from api.group import router as group_router from api.contact import router as contact_router from api.location import router as location_router @@ -126,6 +127,7 @@ def register_routes(app): from api.geospatial import router as geospatial_router from api.ngwmn import router as ngwmn_router + app.include_router(health_router) app.include_router(asset_router) app.include_router(admin_auth_router) app.include_router(author_router) diff --git a/db/README_DATABASE_SETUP.md b/db/README_DATABASE_SETUP.md new file mode 100644 index 000000000..71049593f --- /dev/null +++ b/db/README_DATABASE_SETUP.md @@ -0,0 +1,255 @@ +# Database Setup Guide + +This directory contains SQL files to help you set up and populate a new OcotilloAPI database instance. + +## Files + +- **`schema_dump.sql`** - Complete PostgreSQL database schema with PostGIS support +- **`sample_data.sql`** - Sample data for testing and demonstration +- **`README_DATABASE_SETUP.md`** - This file + +## Quick Start + +### Option 1: Using Alembic Migrations (Recommended) + +The preferred method is to use Alembic, which will create the schema and keep it in sync: + +```bash +# 1. Set up environment variables +export DATABASE_URL="postgresql://user:password@host:5432/dbname" + +# 2. Run migrations +alembic upgrade head + +# 3. Initialize lexicon and parameters +python -c "from core.initializers import init_lexicon, init_parameter; init_lexicon(); init_parameter()" + +# 4. (Optional) Load sample data +psql $DATABASE_URL -f db/sample_data.sql +``` + +### Option 2: Using SQL Dump Files + +If you prefer to use the SQL dump files directly: + +```bash +# 1. Create database +createdb ocotillo_dev + +# 2. Load schema +psql ocotillo_dev -f db/schema_dump.sql + +# 3. Initialize lexicon and parameters (via Python) +export DATABASE_URL="postgresql://user:password@localhost:5432/ocotillo_dev" +python -c "from core.initializers import init_lexicon, init_parameter; init_lexicon(); init_parameter()" + +# 4. (Optional) Load sample data +psql ocotillo_dev -f db/sample_data.sql +``` + +## Schema Details + +### Extensions Required + +- **PostGIS** - For geographic data types and spatial queries +- **uuid-ossp** - For UUID generation + +### Key Table Groups + +1. **Vocabulary Tables** - Lexicon terms, categories, and semantic relationships +2. **Geographic Tables** - Locations, aquifer systems, geologic formations +3. **Things** - Wells, springs, and other monitoring points +4. **Contacts** - People and organizations with emails, phones, addresses +5. **Sensors** - Equipment inventory and deployment history +6. **Sampling** - Field events, activities, samples, and observations +7. **Parameters** - Measurable properties and analysis methods +8. **Groups** - Project and monitoring network organization +9. **Publications** - Research papers and authorship +10. **Polymorphic Tables** - Status history, notes, data provenance + +### Spatial Reference System + +All geographic coordinates use **SRID 4326** (WGS84): +- Longitude range: -180 to 180 +- Latitude range: -90 to 90 + +### Sample Data + +The `sample_data.sql` file includes: + +- **5 Contacts** - Scientists and technicians from various agencies + - Complete with emails, phone numbers, and addresses + +- **5 Locations** - Geographic points across New Mexico + - Albuquerque, Santa Fe, Las Cruces, Los Alamos, Carlsbad + +- **5 Wells (Things)** - Monitoring wells with complete metadata + - Depth, casing, completion details + - Associated with locations, aquifers, and formations + +- **5 Sensors** - Pressure transducers, barometers, acoustic probes + - Serial numbers, equipment status + +- **5 Deployments** - Sensors installed at wells + +- **5 Field Events** - Site visits with participants + +- **5 Samples** - Water samples collected during field events + +- **10 Observations** - Water level and temperature measurements + +- **3 Aquifer Systems** - Santa Fe Group, Ogallala, Roswell Basin + +- **3 Geologic Formations** - Alluvium, Santa Fe Group, Bandelier Tuff + +- **3 Groups/Projects** - Monitoring networks and research projects + +## Prerequisites for Sample Data + +Before loading `sample_data.sql`, you must initialize: + +1. **Lexicon Terms** - Run `init_lexicon()` from `core/initializers.py` + - Required terms: "water well", "monitoring", "public", "unconfined", etc. + +2. **Parameters** - Run `init_parameter()` from `core/initializers.py` + - Parameter ID 1: depth to water + - Parameter ID 2: water temperature + - Additional parameters for chemistry + +These are automatically initialized via: +```python +from core.initializers import init_lexicon, init_parameter +init_lexicon() +init_parameter() +``` + +Or via the seed function: +```python +from transfers.seed import seed_all +seed_all(n=5) # Creates 5 of each entity type +``` + +## Using with Docker + +If running in Docker: + +```bash +# Access the app container +docker compose exec app bash + +# Run migrations +alembic upgrade head + +# Initialize lexicon and parameters +python -c "from core.initializers import init_lexicon, init_parameter; init_lexicon(); init_parameter()" + +# Load sample data +psql $DATABASE_URL -f db/sample_data.sql +``` + +## Using with Render + +On Render, the `preDeployCommand` in `render.yaml` automatically: +1. Creates PostGIS extension +2. Runs Alembic migrations + +To load sample data on Render: +1. Access the database via Render Shell +2. Run the SQL file: `psql $DATABASE_URL -f db/sample_data.sql` + +## Verification + +After loading data, verify with these queries: + +```sql +-- Count records +SELECT + (SELECT COUNT(*) FROM contact) as contacts, + (SELECT COUNT(*) FROM location) as locations, + (SELECT COUNT(*) FROM thing) as things, + (SELECT COUNT(*) FROM sensor) as sensors, + (SELECT COUNT(*) FROM observation) as observations; + +-- List all wells with locations +SELECT t.name, l.county, l.state, ST_AsText(l.point) as coordinates +FROM thing t +JOIN location_thing_association lta ON t.id = lta.thing_id +JOIN location l ON lta.location_id = l.id +WHERE lta.effective_end IS NULL; + +-- Check parameter initialization +SELECT COUNT(*) FROM parameter; +SELECT COUNT(*) FROM lexicon_term; +``` + +## Resetting the Database + +To start fresh: + +```bash +# Drop and recreate +dropdb ocotillo_dev +createdb ocotillo_dev + +# Reload schema and data +psql ocotillo_dev -f db/schema_dump.sql +python -c "from core.initializers import init_lexicon, init_parameter; init_lexicon(); init_parameter()" +psql ocotillo_dev -f db/sample_data.sql +``` + +Or with Alembic: + +```bash +# Downgrade all migrations +alembic downgrade base + +# Upgrade to latest +alembic upgrade head + +# Reinitialize +python -c "from core.initializers import init_lexicon, init_parameter; init_lexicon(); init_parameter()" +psql $DATABASE_URL -f db/sample_data.sql +``` + +## Troubleshooting + +### PostGIS Extension Not Found + +```sql +-- Check if PostGIS is available +SELECT name, default_version FROM pg_available_extensions WHERE name = 'postgis'; + +-- Enable PostGIS +CREATE EXTENSION IF NOT EXISTS postgis; +``` + +### Lexicon Terms Not Found + +If you get foreign key errors referencing `lexicon_term`, make sure to run: +```python +from core.initializers import init_lexicon +init_lexicon() +``` + +### Parameter Not Found + +If observations fail to insert due to missing parameters: +```python +from core.initializers import init_parameter +init_parameter() +``` + +## Additional Resources + +- **Alembic Migrations**: `/alembic/versions/` +- **Model Definitions**: `/db/*.py` +- **Seed Script**: `/transfers/seed.py` +- **Lexicon Data**: `/core/lexicon.json` +- **Parameter Data**: `/core/parameter.json` + +## Support + +For questions or issues with database setup, refer to: +- Main README.md +- RENDER_DEPLOYMENT.md (for Render-specific setup) +- Alembic documentation: https://alembic.sqlalchemy.org/ diff --git a/db/engine.py b/db/engine.py index 4fa1e638d..11c783cfa 100644 --- a/db/engine.py +++ b/db/engine.py @@ -150,22 +150,32 @@ def getconn(): # async_engine = asyncio.run(get_async_engine()) else: - # if driver == "sqlite": - # name = os.environ.get("DB_NAME", "development.db") - # url = f"sqlite:///{name}" - # elif driver == "postgres": - password = os.environ.get("POSTGRES_PASSWORD", "") - host = os.environ.get("POSTGRES_HOST", "localhost") - port = os.environ.get("POSTGRES_PORT", "5432") - # Default to current OS user if POSTGRES_USER not set or empty - user = os.environ.get("POSTGRES_USER", "").strip() or getpass.getuser() - name = os.environ.get("POSTGRES_DB", "postgres") - - auth = f"{user}:{password}@" if user and password else "" - port_part = f":{port}" if port else "" - url = f"postgresql+pg8000://{auth}{host}{port_part}/{name}" - # else: - # url = "sqlite:///./development.db" + # Check for DATABASE_URL first (Render/Heroku standard) + # Falls back to individual env vars for backward compatibility + database_url = os.environ.get("DATABASE_URL", "") + + if database_url: + # Use DATABASE_URL if provided (e.g., from Render) + # Convert postgresql:// to postgresql+pg8000:// for SQLAlchemy + if database_url.startswith("postgres://"): + # Handle legacy postgres:// scheme (some platforms use this) + url = database_url.replace("postgres://", "postgresql+pg8000://", 1) + elif database_url.startswith("postgresql://"): + url = database_url.replace("postgresql://", "postgresql+pg8000://", 1) + else: + url = database_url + else: + # Fall back to individual environment variables (backward compatible) + password = os.environ.get("POSTGRES_PASSWORD", "") + host = os.environ.get("POSTGRES_HOST", "localhost") + port = os.environ.get("POSTGRES_PORT", "5432") + # Default to current OS user if POSTGRES_USER not set or empty + user = os.environ.get("POSTGRES_USER", "").strip() or getpass.getuser() + name = os.environ.get("POSTGRES_DB", "postgres") + + auth = f"{user}:{password}@" if user and password else "" + port_part = f":{port}" if port else "" + url = f"postgresql+pg8000://{auth}{host}{port_part}/{name}" # Configure connection pool for parallel transfers # pool_size: number of persistent connections diff --git a/db/sample_data.sql b/db/sample_data.sql new file mode 100644 index 000000000..c02fb94b5 --- /dev/null +++ b/db/sample_data.sql @@ -0,0 +1,379 @@ +-- ============================================================================ +-- OcotilloAPI Sample Data +-- ============================================================================ +-- This file contains sample data to populate a new OcotilloAPI instance +-- Run this AFTER running schema_dump.sql and initializing lexicon/parameters +-- ============================================================================ + +-- Prerequisites: +-- 1. PostGIS extension must be enabled +-- 2. Lexicon terms must be initialized (via core/lexicon.json) +-- 3. Parameters must be initialized (via core/parameter.json) + +BEGIN; + +-- ============================================================================ +-- SAMPLE CONTACTS +-- ============================================================================ + +INSERT INTO contact (id, name, organization, role, contact_type, release_status, created_at) +VALUES + (1, 'John Smith', 'USGS', 'Hydrologist', 'person', 'public', NOW()), + (2, 'Jane Doe', 'New Mexico Bureau of Geology', 'Field Technician', 'person', 'public', NOW()), + (3, 'Robert Johnson', 'NM Environment Department', 'Water Quality Specialist', 'person', 'public', NOW()), + (4, 'Maria Garcia', 'Los Alamos National Laboratory', 'Environmental Scientist', 'person', 'public', NOW()), + (5, 'David Chen', 'Sandia National Laboratories', 'Geologist', 'person', 'public', NOW()); + +-- Set sequence to continue from last ID +SELECT setval('contact_id_seq', 5); + +-- Contact Emails +INSERT INTO email (contact_id, email, release_status) +VALUES + (1, 'john.smith@usgs.gov', 'public'), + (2, 'jane.doe@nmbg.nmt.edu', 'public'), + (3, 'robert.johnson@env.nm.gov', 'public'), + (4, 'maria.garcia@lanl.gov', 'public'), + (5, 'david.chen@sandia.gov', 'public'); + +-- Contact Phones +INSERT INTO phone (contact_id, phone_number, release_status) +VALUES + (1, '505-123-4567', 'public'), + (2, '575-234-5678', 'public'), + (3, '505-345-6789', 'public'), + (4, '505-456-7890', 'public'), + (5, '505-567-8901', 'public'); + +-- Contact Addresses +INSERT INTO address (contact_id, street, city, state, zip_code, country, release_status) +VALUES + (1, '5338 Montgomery Blvd NE', 'Albuquerque', 'NM', '87109', 'USA', 'public'), + (2, '801 Leroy Place', 'Socorro', 'NM', '87801', 'USA', 'public'), + (3, '1190 Saint Francis Dr', 'Santa Fe', 'NM', '87505', 'USA', 'public'), + (4, 'TA-3, SM-29', 'Los Alamos', 'NM', '87545', 'USA', 'public'), + (5, '1515 Eubank Blvd SE', 'Albuquerque', 'NM', '87123', 'USA', 'public'); + +-- ============================================================================ +-- SAMPLE LOCATIONS (New Mexico coordinates) +-- ============================================================================ + +INSERT INTO location (id, description, point, elevation, county, state, release_status, created_at) +VALUES + (1, 'Albuquerque East Mesa', ST_SetSRID(ST_MakePoint(-106.5419, 35.1107), 4326), 1620.5, 'Bernalillo', 'NM', 'public', NOW()), + (2, 'Santa Fe Buckman Well Field', ST_SetSRID(ST_MakePoint(-106.0031, 35.7381), 4326), 1850.2, 'Santa Fe', 'NM', 'public', NOW()), + (3, 'Las Cruces East Mesa', ST_SetSRID(ST_MakePoint(-106.7369, 32.3199), 4326), 1220.8, 'Dona Ana', 'NM', 'public', NOW()), + (4, 'Los Alamos Canyon', ST_SetSRID(ST_MakePoint(-106.3198, 35.8869), 4326), 2134.6, 'Los Alamos', 'NM', 'public', NOW()), + (5, 'Carlsbad Caverns Area', ST_SetSRID(ST_MakePoint(-104.4457, 32.1476), 4326), 1095.3, 'Eddy', 'NM', 'public', NOW()); + +SELECT setval('location_id_seq', 5); + +-- ============================================================================ +-- SAMPLE AQUIFER SYSTEMS +-- ============================================================================ + +INSERT INTO aquifer_system (id, name, description, primary_aquifer_type, geographic_scale, release_status, created_at) +VALUES + (1, 'Santa Fe Group Aquifer System', 'Basin-fill aquifer underlying the Rio Grande rift', 'unconfined', 'major', 'public', NOW()), + (2, 'Ogallala Aquifer', 'High Plains aquifer in eastern New Mexico', 'unconfined', 'major', 'public', NOW()), + (3, 'Roswell Artesian Basin', 'Confined aquifer in Pecos River valley', 'confined', 'regional', 'public', NOW()); + +SELECT setval('aquifer_system_id_seq', 3); + +-- ============================================================================ +-- SAMPLE GEOLOGIC FORMATIONS +-- ============================================================================ + +INSERT INTO geologic_formation (id, formation_code, description, lithology, release_status, created_at) +VALUES + (1, 'alluvium', 'Quaternary alluvial deposits', 'alluvium', 'public', NOW()), + (2, 'santa fe group', 'Tertiary basin-fill sediments', 'sandstone', 'public', NOW()), + (3, 'bandelier tuff', 'Volcanic tuff from Valles Caldera', 'tuff', 'public', NOW()); + +SELECT setval('geologic_formation_id_seq', 3); + +-- ============================================================================ +-- SAMPLE SENSORS +-- ============================================================================ + +INSERT INTO sensor (id, name, sensor_type, model, serial_no, pcn_number, owner_agency, sensor_status, release_status, created_at) +VALUES + (1, 'Pressure Transducer PT-001', 'pressure transducer', 'In-Situ Level TROLL 500', 'SN123456', 'PCN-2024-001', 'USGS', 'in service', 'public', NOW()), + (2, 'Barometer BAR-001', 'barometer', 'In-Situ Baro TROLL', 'SN789012', 'PCN-2024-002', 'USGS', 'in service', 'public', NOW()), + (3, 'Acoustic Probe AC-001', 'acoustic probe', 'Solinst Model 107', 'SN345678', 'PCN-2024-003', 'NMBG', 'in service', 'public', NOW()), + (4, 'Electric Tape ET-001', 'electric tape', 'Solinst Model 101', 'SN901234', 'PCN-2024-004', 'NMED', 'in service', 'public', NOW()), + (5, 'Water Level Logger WL-001', 'water level logger', 'Onset HOBO U20L', 'SN567890', 'PCN-2024-005', 'LANL', 'in service', 'public', NOW()); + +SELECT setval('sensor_id_seq', 5); + +-- ============================================================================ +-- SAMPLE THINGS (Wells) +-- ============================================================================ + +INSERT INTO thing (id, name, thing_type, well_depth, hole_depth, well_casing_diameter, well_casing_depth, + well_completion_date, well_driller_name, is_suitable_for_datalogger, release_status, created_at) +VALUES + (1, 'ABQ-EAST-001', 'water well', 350.0, 365.0, 6.0, 340.0, '2015-06-15', 'ABC Drilling Company', true, 'public', NOW()), + (2, 'SF-BUCKMAN-042', 'water well', 800.0, 825.0, 12.0, 780.0, '2010-03-22', 'New Mexico Well Drilling', true, 'public', NOW()), + (3, 'LC-MESA-108', 'water well', 450.0, 465.0, 8.0, 440.0, '2018-11-08', 'Southwest Drilling Inc', true, 'public', NOW()), + (4, 'LA-CANYON-019', 'water well', 280.0, 295.0, 6.0, 270.0, '2012-08-30', 'Mountain Drilling LLC', false, 'public', NOW()), + (5, 'CARLSBAD-055', 'water well', 520.0, 535.0, 10.0, 500.0, '2020-01-17', 'Pecos Valley Drilling', true, 'public', NOW()); + +SELECT setval('thing_id_seq', 5); + +-- Link Things to Locations (with current effective dates) +INSERT INTO location_thing_association (location_id, thing_id, effective_start, effective_end) +VALUES + (1, 1, NOW(), NULL), + (2, 2, NOW(), NULL), + (3, 3, NOW(), NULL), + (4, 4, NOW(), NULL), + (5, 5, NOW(), NULL); + +-- ============================================================================ +-- WELL DETAILS +-- ============================================================================ + +-- Well Purposes +INSERT INTO well_purpose (thing_id, purpose, release_status) +VALUES + (1, 'monitoring', 'public'), + (2, 'public supply', 'public'), + (3, 'monitoring', 'public'), + (4, 'monitoring', 'public'), + (5, 'monitoring', 'public'); + +-- Well Casing Materials +INSERT INTO well_casing_material (thing_id, material, release_status) +VALUES + (1, 'steel', 'public'), + (2, 'steel', 'public'), + (3, 'pvc', 'public'), + (4, 'steel', 'public'), + (5, 'steel', 'public'); + +-- Well Screens +INSERT INTO well_screen (thing_id, aquifer_system_id, geologic_formation_id, screen_depth_top, screen_depth_bottom, + screen_type, screen_description, release_status) +VALUES + (1, 1, 2, 320.0, 350.0, 'slotted', '30 ft slotted screen in Santa Fe Group', 'public'), + (2, 1, 2, 750.0, 800.0, 'slotted', '50 ft slotted screen in Santa Fe Group', 'public'), + (3, 1, 1, 420.0, 450.0, 'slotted', '30 ft slotted screen in alluvium', 'public'), + (4, 1, 3, 250.0, 280.0, 'slotted', '30 ft slotted screen in Bandelier Tuff', 'public'), + (5, 3, 1, 490.0, 520.0, 'slotted', '30 ft slotted screen in Roswell Basin', 'public'); + +-- Thing-Aquifer Associations +INSERT INTO thing_aquifer_association (thing_id, aquifer_system_id, release_status) +VALUES + (1, 1, 'public'), + (2, 1, 'public'), + (3, 1, 'public'), + (4, 1, 'public'), + (5, 3, 'public'); + +-- Aquifer Types for each association +INSERT INTO aquifer_type (thing_aquifer_association_id, aquifer_type, release_status) +VALUES + (1, 'unconfined', 'public'), + (2, 'unconfined', 'public'), + (3, 'unconfined', 'public'), + (4, 'fractured', 'public'), + (5, 'confined', 'public'); + +-- Measuring Point History (current) +INSERT INTO measuring_point_history (thing_id, measuring_point_height, measuring_point_description, start_date, end_date, release_status) +VALUES + (1, 2.5, 'Top of casing, 2.5 ft above ground surface', '2015-06-15', NULL, 'public'), + (2, 3.0, 'Top of casing, 3.0 ft above ground surface', '2010-03-22', NULL, 'public'), + (3, 2.0, 'Top of casing, 2.0 ft above ground surface', '2018-11-08', NULL, 'public'), + (4, 1.5, 'Top of casing, 1.5 ft above ground surface', '2012-08-30', NULL, 'public'), + (5, 2.8, 'Top of casing, 2.8 ft above ground surface', '2020-01-17', NULL, 'public'); + +-- Monitoring Frequency History +INSERT INTO monitoring_frequency_history (thing_id, monitoring_frequency, start_date, end_date, release_status) +VALUES + (1, 'quarterly', '2015-06-15', NULL, 'public'), + (2, 'monthly', '2010-03-22', NULL, 'public'), + (3, 'quarterly', '2018-11-08', NULL, 'public'), + (4, 'annual', '2012-08-30', NULL, 'public'), + (5, 'quarterly', '2020-01-17', NULL, 'public'); + +-- Thing-Contact Associations (well ownership/access) +INSERT INTO thing_contact_association (thing_id, contact_id) +VALUES + (1, 1), + (2, 2), + (3, 3), + (4, 4), + (5, 5); + +-- ============================================================================ +-- SAMPLE DEPLOYMENTS +-- ============================================================================ + +INSERT INTO deployment (thing_id, sensor_id, installation_date, removal_date, recording_interval, + recording_interval_units, hanging_cable_length, release_status, created_at) +VALUES + (1, 1, '2024-01-15', NULL, 15, 'minutes', 325.0, 'public', NOW()), + (2, 3, '2024-02-20', NULL, 60, 'minutes', 785.0, 'public', NOW()), + (3, 1, '2024-03-10', '2024-06-10', 30, 'minutes', 445.0, 'public', NOW()), + (4, 5, '2024-04-05', NULL, 30, 'minutes', 275.0, 'public', NOW()), + (5, 1, '2024-05-01', NULL, 15, 'minutes', 515.0, 'public', NOW()); + +-- ============================================================================ +-- SAMPLE FIELD EVENTS & SAMPLING +-- ============================================================================ + +-- Field Events (site visits) +INSERT INTO field_event (id, thing_id, event_date, notes, release_status) +VALUES + (1, 1, '2024-06-15 10:30:00+00', 'Quarterly monitoring event', 'public'), + (2, 2, '2024-06-18 09:00:00+00', 'Monthly water level measurement', 'public'), + (3, 3, '2024-06-20 14:15:00+00', 'Quarterly sampling and water level', 'public'), + (4, 4, '2024-06-22 11:00:00+00', 'Annual inspection and measurement', 'public'), + (5, 5, '2024-06-25 08:30:00+00', 'Quarterly monitoring', 'public'); + +SELECT setval('field_event_id_seq', 5); + +-- Field Event Participants +INSERT INTO field_event_participant (id, field_event_id, contact_id, participant_role, release_status) +VALUES + (1, 1, 1, 'field technician', 'public'), + (2, 2, 2, 'field technician', 'public'), + (3, 3, 3, 'sampler', 'public'), + (4, 4, 4, 'field technician', 'public'), + (5, 5, 5, 'field technician', 'public'); + +SELECT setval('field_event_participant_id_seq', 5); + +-- Field Activities +INSERT INTO field_activity (id, field_event_id, activity_type, notes, release_status) +VALUES + (1, 1, 'water level measurement', 'Measured depth to water', 'public'), + (2, 2, 'water level measurement', 'Measured depth to water', 'public'), + (3, 3, 'water quality sampling', 'Collected sample for metals analysis', 'public'), + (4, 4, 'water level measurement', 'Annual monitoring measurement', 'public'), + (5, 5, 'water level measurement', 'Quarterly monitoring', 'public'); + +SELECT setval('field_activity_id_seq', 5); + +-- Samples +INSERT INTO sample (id, field_activity_id, field_event_participant_id, sample_date, sample_name, + sample_matrix, sample_method, qc_type, release_status, created_at) +VALUES + (1, 1, '1', '2024-06-15 10:45:00+00', 'ABQ-EAST-001-20240615', 'water', 'bailer', 'Normal', 'public', NOW()), + (2, 2, '2', '2024-06-18 09:15:00+00', 'SF-BUCKMAN-042-20240618', 'water', 'electric tape', 'Normal', 'public', NOW()), + (3, 3, '3', '2024-06-20 14:30:00+00', 'LC-MESA-108-20240620', 'water', 'bailer', 'Normal', 'public', NOW()), + (4, 4, '4', '2024-06-22 11:15:00+00', 'LA-CANYON-019-20240622', 'water', 'electric tape', 'Normal', 'public', NOW()), + (5, 5, '5', '2024-06-25 08:45:00+00', 'CARLSBAD-055-20240625', 'water', 'bailer', 'Normal', 'public', NOW()); + +SELECT setval('sample_id_seq', 5); + +-- ============================================================================ +-- SAMPLE OBSERVATIONS (Water Levels) +-- ============================================================================ +-- Note: This assumes parameter IDs exist from initialization +-- Common water parameter IDs: 1=depth to water, 2=water temperature, 3=pH, etc. + +INSERT INTO observation (sample_id, parameter_id, observation_datetime, value, unit, + measuring_point_height, release_status, created_at) +VALUES + -- Depth to water observations + (1, 1, '2024-06-15 10:45:00+00', 145.32, 'feet', 2.5, 'public', NOW()), + (2, 1, '2024-06-18 09:15:00+00', 287.56, 'feet', 3.0, 'public', NOW()), + (3, 1, '2024-06-20 14:30:00+00', 178.91, 'feet', 2.0, 'public', NOW()), + (4, 1, '2024-06-22 11:15:00+00', 98.44, 'feet', 1.5, 'public', NOW()), + (5, 1, '2024-06-25 08:45:00+00', 223.67, 'feet', 2.8, 'public', NOW()), + + -- Water temperature observations + (1, 2, '2024-06-15 10:45:00+00', 15.8, 'celsius', NULL, 'public', NOW()), + (2, 2, '2024-06-18 09:15:00+00', 14.2, 'celsius', NULL, 'public', NOW()), + (3, 2, '2024-06-20 14:30:00+00', 16.5, 'celsius', NULL, 'public', NOW()), + (4, 2, '2024-06-22 11:15:00+00', 13.9, 'celsius', NULL, 'public', NOW()), + (5, 2, '2024-06-25 08:45:00+00', 17.1, 'celsius', NULL, 'public', NOW()); + +-- ============================================================================ +-- SAMPLE GROUPS/PROJECTS +-- ============================================================================ + +INSERT INTO "group" (id, name, description, group_type, release_status) +VALUES + (1, 'Albuquerque Basin Monitoring Network', 'Long-term monitoring of groundwater levels in Albuquerque Basin', 'monitoring network', 'public'), + (2, 'Santa Fe Basin Study', 'Water resources assessment for Santa Fe Basin', 'research project', 'public'), + (3, 'Statewide Baseline Network', 'Wells monitored for baseline water quality', 'monitoring network', 'public'); + +SELECT setval('group_id_seq', 3); + +INSERT INTO group_thing_association (group_id, thing_id) +VALUES + (1, 1), + (1, 3), + (2, 2), + (2, 4), + (3, 1), + (3, 2), + (3, 3), + (3, 4), + (3, 5); + +-- ============================================================================ +-- SAMPLE NOTES +-- ============================================================================ + +INSERT INTO notes (target_id, target_table, note_type, content, release_status) +VALUES + (1, 'thing', 'general', 'Well is located on public land. Easy access via maintained road.', 'public'), + (2, 'thing', 'construction', 'Well drilled using air rotary method. Casing is cemented to surface.', 'public'), + (3, 'thing', 'sampling procedure', 'Purge 3 well volumes before sampling. Use low-flow sampling method.', 'public'), + (4, 'thing', 'general', 'Well is on LANL property. Requires security clearance for access.', 'public'), + (5, 'thing', 'water quality', 'Historical elevated nitrate levels detected. Monitor quarterly.', 'public'); + +-- ============================================================================ +-- SAMPLE DATA PROVENANCE +-- ============================================================================ + +INSERT INTO data_provenance (target_id, target_table, field_name, origin_type, origin_source, collection_method, accuracy_value, accuracy_unit) +VALUES + (1, 'location', 'point', 'GPS', 'Field survey 2015-06-15', 'gps survey grade', 0.5, 'meters'), + (1, 'location', 'elevation', 'GPS', 'Field survey 2015-06-15', 'gps survey grade', 0.3, 'meters'), + (2, 'location', 'point', 'GPS', 'Field survey 2010-03-22', 'gps survey grade', 0.5, 'meters'), + (3, 'location', 'point', 'GPS', 'Field survey 2018-11-08', 'gps handheld', 3.0, 'meters'), + (1, 'thing', 'well_depth', 'drillers log', 'ABC Drilling Company completion report', NULL, NULL, NULL); + +-- ============================================================================ +-- SAMPLE STATUS HISTORY +-- ============================================================================ + +INSERT INTO status_history (status_type, status_value, start_date, end_date, target_id, target_table, release_status) +VALUES + ('well status', 'active', '2015-06-15', NULL, 1, 'thing', 'public'), + ('monitoring status', 'routine monitoring', '2015-06-15', NULL, 1, 'thing', 'public'), + ('well status', 'active', '2010-03-22', NULL, 2, 'thing', 'public'), + ('monitoring status', 'routine monitoring', '2010-03-22', NULL, 2, 'thing', 'public'), + ('well status', 'active', '2018-11-08', NULL, 3, 'thing', 'public'); + +COMMIT; + +-- ============================================================================ +-- VERIFICATION QUERIES +-- ============================================================================ +-- Run these to verify data was inserted correctly: + +-- SELECT COUNT(*) as contact_count FROM contact; +-- SELECT COUNT(*) as location_count FROM location; +-- SELECT COUNT(*) as thing_count FROM thing; +-- SELECT COUNT(*) as sensor_count FROM sensor; +-- SELECT COUNT(*) as observation_count FROM observation; +-- SELECT COUNT(*) as sample_count FROM sample; +-- SELECT COUNT(*) as deployment_count FROM deployment; + +-- List all wells with their locations: +-- SELECT t.name, l.county, l.state, ST_AsText(l.point) as coordinates +-- FROM thing t +-- JOIN location_thing_association lta ON t.id = lta.thing_id +-- JOIN location l ON lta.location_id = l.id +-- WHERE lta.effective_end IS NULL; + +-- ============================================================================ +-- END OF SAMPLE DATA +-- ============================================================================ diff --git a/db/schema_dump.sql b/db/schema_dump.sql new file mode 100644 index 000000000..f25146c70 --- /dev/null +++ b/db/schema_dump.sql @@ -0,0 +1,721 @@ +-- ============================================================================ +-- OcotilloAPI Database Schema Dump +-- PostgreSQL 17 with PostGIS Extension +-- ============================================================================ +-- This file contains the complete database schema for OcotilloAPI +-- Use this to recreate the database structure on a new instance +-- ============================================================================ + +-- Enable required extensions +CREATE EXTENSION IF NOT EXISTS postgis; +CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; + +-- ============================================================================ +-- CORE VOCABULARY TABLES +-- ============================================================================ + +-- Lexicon Categories +CREATE TABLE lexicon_category ( + id SERIAL PRIMARY KEY, + name VARCHAR(100) NOT NULL UNIQUE, + description VARCHAR(255), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- Lexicon Terms (Controlled Vocabulary) +CREATE TABLE lexicon_term ( + id SERIAL PRIMARY KEY, + term VARCHAR NOT NULL UNIQUE, + definition VARCHAR, + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- Lexicon Term-Category Association +CREATE TABLE lexicon_term_category_association ( + id SERIAL PRIMARY KEY, + term_id INTEGER NOT NULL REFERENCES lexicon_term(id) ON DELETE CASCADE, + category_id INTEGER NOT NULL REFERENCES lexicon_category(id) ON DELETE CASCADE +); + +-- Lexicon Triples (Semantic Relationships) +CREATE TABLE lexicon_triple ( + id SERIAL PRIMARY KEY, + subject VARCHAR(100) REFERENCES lexicon_term(term) ON DELETE CASCADE, + predicate VARCHAR(100), + object_ VARCHAR(100) REFERENCES lexicon_term(term) ON DELETE CASCADE, + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- ============================================================================ +-- GEOGRAPHIC & REFERENCE DATA +-- ============================================================================ + +-- Locations (Geographic Points) +CREATE TABLE location ( + id SERIAL PRIMARY KEY, + nma_pk_location VARCHAR(36), + description VARCHAR, + point GEOMETRY(POINT, 4326), + elevation FLOAT, + county VARCHAR(100), + state VARCHAR(100), + quad_name VARCHAR(100), + nma_notes_location TEXT, + nma_coordinate_notes TEXT, + nma_date_created DATE, + nma_site_date DATE, + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + search_vector TSVECTOR +); + +CREATE INDEX idx_location_point ON location USING GIST (point); +CREATE INDEX idx_location_search ON location USING GIN (search_vector); + +-- Aquifer Systems +CREATE TABLE aquifer_system ( + id SERIAL PRIMARY KEY, + name VARCHAR NOT NULL UNIQUE, + description TEXT, + primary_aquifer_type VARCHAR(100) REFERENCES lexicon_term(term), + geographic_scale VARCHAR(100) REFERENCES lexicon_term(term), + boundary GEOMETRY(MULTIPOLYGON, 4326), + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +CREATE INDEX idx_aquifer_system_name ON aquifer_system(name); +CREATE INDEX idx_aquifer_system_boundary ON aquifer_system USING GIST (boundary); + +-- Geologic Formations +CREATE TABLE geologic_formation ( + id SERIAL PRIMARY KEY, + formation_code VARCHAR(100) UNIQUE REFERENCES lexicon_term(term), + description TEXT, + lithology VARCHAR(100) REFERENCES lexicon_term(term), + boundary GEOMETRY(MULTIPOLYGON, 4326), + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +CREATE INDEX idx_geologic_formation_code ON geologic_formation(formation_code); +CREATE INDEX idx_geologic_formation_boundary ON geologic_formation USING GIST (boundary); + +-- ============================================================================ +-- THING (Well/Spring/Stream Gauge) +-- ============================================================================ + +CREATE TABLE thing ( + id SERIAL PRIMARY KEY, + name VARCHAR, + thing_type VARCHAR REFERENCES lexicon_term(term), + first_visit_date DATE, + nma_pk_welldata VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + + -- Well-specific columns + well_depth FLOAT, + hole_depth FLOAT, + well_casing_diameter FLOAT, + well_casing_depth FLOAT, + well_completion_date DATE, + well_driller_name VARCHAR(200), + well_construction_method VARCHAR REFERENCES lexicon_term(term), + well_pump_type VARCHAR REFERENCES lexicon_term(term), + well_pump_depth FLOAT, + formation_completion_code VARCHAR REFERENCES lexicon_term(term), + is_suitable_for_datalogger BOOLEAN, + + -- Spring-specific columns + spring_type VARCHAR REFERENCES lexicon_term(term), + + -- Audit columns + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + search_vector TSVECTOR +); + +CREATE INDEX idx_thing_search ON thing USING GIN (search_vector); + +-- Location-Thing Association (Time-Series) +CREATE TABLE location_thing_association ( + id SERIAL PRIMARY KEY, + location_id INTEGER NOT NULL REFERENCES location(id), + thing_id INTEGER NOT NULL REFERENCES thing(id), + effective_start TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + effective_end TIMESTAMP WITH TIME ZONE +); + +-- Well Screens +CREATE TABLE well_screen ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + aquifer_system_id INTEGER REFERENCES aquifer_system(id) ON DELETE SET NULL, + geologic_formation_id INTEGER REFERENCES geologic_formation(id) ON DELETE SET NULL, + screen_depth_top FLOAT, + screen_depth_bottom FLOAT, + screen_type VARCHAR(100) REFERENCES lexicon_term(term), + screen_description VARCHAR(1000), + nma_pk_wellscreens VARCHAR(100), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Well Purpose +CREATE TABLE well_purpose ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + purpose VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_well_purpose_search ON well_purpose USING GIN (search_vector); + +-- Well Casing Material +CREATE TABLE well_casing_material ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + material VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_well_casing_material_search ON well_casing_material USING GIN (search_vector); + +-- Thing Aquifer Association +CREATE TABLE thing_aquifer_association ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + aquifer_system_id INTEGER NOT NULL REFERENCES aquifer_system(id) ON DELETE CASCADE, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Aquifer Type +CREATE TABLE aquifer_type ( + id SERIAL PRIMARY KEY, + thing_aquifer_association_id INTEGER NOT NULL REFERENCES thing_aquifer_association(id) ON DELETE CASCADE, + aquifer_type VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Thing Geologic Formation Association +CREATE TABLE thing_geologic_formation_association ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + geologic_formation_id INTEGER REFERENCES geologic_formation(id) ON DELETE SET NULL, + top_depth FLOAT NOT NULL, + bottom_depth FLOAT NOT NULL, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Thing ID Links (External System IDs) +CREATE TABLE thing_id_link ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + relation VARCHAR(100) REFERENCES lexicon_term(term), + alternate_id VARCHAR(100), + alternate_organization VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Measuring Point History +CREATE TABLE measuring_point_history ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + measuring_point_height NUMERIC NOT NULL, + measuring_point_description TEXT, + start_date DATE NOT NULL, + end_date DATE, + reason TEXT, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Monitoring Frequency History +CREATE TABLE monitoring_frequency_history ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + monitoring_frequency VARCHAR(100) REFERENCES lexicon_term(term), + start_date DATE NOT NULL, + end_date DATE, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- ============================================================================ +-- CONTACTS & ORGANIZATIONS +-- ============================================================================ + +CREATE TABLE contact ( + id SERIAL PRIMARY KEY, + name VARCHAR(100), + organization VARCHAR(100) REFERENCES lexicon_term(term), + role VARCHAR(100) REFERENCES lexicon_term(term), + contact_type VARCHAR(100) REFERENCES lexicon_term(term), + nma_pk_owners VARCHAR(100), + nma_pk_waterlevels VARCHAR(100), + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + search_vector TSVECTOR, + CONSTRAINT uq_contact_name_organization UNIQUE (name, organization) +); + +CREATE INDEX idx_contact_search ON contact USING GIN (search_vector); + +-- Phone Numbers +CREATE TABLE phone ( + id SERIAL PRIMARY KEY, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + phone_number VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_phone_search ON phone USING GIN (search_vector); + +-- Email Addresses +CREATE TABLE email ( + id SERIAL PRIMARY KEY, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + email VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_email_search ON email USING GIN (search_vector); + +-- Physical Addresses +CREATE TABLE address ( + id SERIAL PRIMARY KEY, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + street VARCHAR, + city VARCHAR, + state VARCHAR, + zip_code VARCHAR, + country VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_address_search ON address USING GIN (search_vector); + +-- Incomplete NMA Phone (legacy migration support) +CREATE TABLE incomplete_nma_phone ( + id SERIAL PRIMARY KEY, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + phone_number VARCHAR +); + +-- Thing-Contact Association +CREATE TABLE thing_contact_association ( + id SERIAL PRIMARY KEY, + thing_id INTEGER REFERENCES thing(id), + contact_id INTEGER REFERENCES contact(id) +); + +-- ============================================================================ +-- SENSORS & EQUIPMENT +-- ============================================================================ + +CREATE TABLE sensor ( + id SERIAL PRIMARY KEY, + nma_pk_equipment VARCHAR(36), + name VARCHAR(255) NOT NULL, + sensor_type VARCHAR(100) REFERENCES lexicon_term(term), + model VARCHAR(50), + serial_no VARCHAR(50) UNIQUE, + pcn_number VARCHAR(50) UNIQUE, + owner_agency VARCHAR(100) REFERENCES lexicon_term(term), + sensor_status VARCHAR(100) REFERENCES lexicon_term(term), + notes TEXT, + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- Sensor Deployments +CREATE TABLE deployment ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + sensor_id INTEGER NOT NULL REFERENCES sensor(id), + installation_date DATE NOT NULL, + removal_date DATE, + recording_interval INTEGER, + recording_interval_units VARCHAR(100) REFERENCES lexicon_term(term), + hanging_cable_length NUMERIC, + hanging_point_height NUMERIC, + hanging_point_description TEXT, + notes TEXT, + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- ============================================================================ +-- PARAMETERS & ANALYSIS +-- ============================================================================ + +CREATE TABLE parameter ( + id SERIAL PRIMARY KEY, + parameter_name VARCHAR(100) REFERENCES lexicon_term(term), + matrix VARCHAR(100) REFERENCES lexicon_term(term), + parameter_type VARCHAR(100) REFERENCES lexicon_term(term), + cas_number VARCHAR, + default_unit VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + CONSTRAINT uq_parameter_name_matrix UNIQUE (parameter_name, matrix) +); + +CREATE TABLE analysis_method ( + id SERIAL PRIMARY KEY, + analysis_method_code VARCHAR UNIQUE, + analysis_method_name VARCHAR, + analysis_method_type VARCHAR(100) REFERENCES lexicon_term(term), + source_organization VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE TABLE regulatory_limit ( + id SERIAL PRIMARY KEY, + parameter_id INTEGER NOT NULL REFERENCES parameter(id), + limit_source VARCHAR(100) REFERENCES lexicon_term(term), + limit_value NUMERIC NOT NULL, + limit_unit VARCHAR(100) REFERENCES lexicon_term(term), + limit_type VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- ============================================================================ +-- FIELD ACTIVITIES & SAMPLING +-- ============================================================================ + +CREATE TABLE field_event ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + event_date TIMESTAMP WITH TIME ZONE NOT NULL, + notes TEXT, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE TABLE field_event_participant ( + id SERIAL PRIMARY KEY, + field_event_id INTEGER NOT NULL REFERENCES field_event(id) ON DELETE CASCADE, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + participant_role VARCHAR(100) REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE TABLE field_activity ( + id SERIAL PRIMARY KEY, + field_event_id INTEGER NOT NULL REFERENCES field_event(id) ON DELETE CASCADE, + activity_type VARCHAR(100) REFERENCES lexicon_term(term), + notes TEXT, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE TABLE sample ( + id SERIAL PRIMARY KEY, + field_activity_id INTEGER NOT NULL REFERENCES field_activity(id) ON DELETE CASCADE, + field_event_participant_id VARCHAR REFERENCES field_event_participant(id), + sample_date TIMESTAMP WITH TIME ZONE NOT NULL, + sample_name VARCHAR NOT NULL UNIQUE, + sample_matrix VARCHAR(100) REFERENCES lexicon_term(term), + sample_method VARCHAR(100) REFERENCES lexicon_term(term), + qc_type VARCHAR DEFAULT 'Normal', + depth_top FLOAT, + depth_bottom FLOAT, + notes TEXT, + nma_pk_waterlevels VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +CREATE TABLE observation ( + id SERIAL PRIMARY KEY, + nma_pk_waterlevels VARCHAR, + sample_id INTEGER NOT NULL REFERENCES sample(id) ON DELETE CASCADE, + sensor_id INTEGER REFERENCES sensor(id), + analysis_method_id INTEGER REFERENCES analysis_method(id), + parameter_id INTEGER NOT NULL REFERENCES parameter(id), + observation_datetime TIMESTAMP WITH TIME ZONE NOT NULL, + value FLOAT, + unit VARCHAR(100) REFERENCES lexicon_term(term), + notes TEXT, + measuring_point_height FLOAT, + groundwater_level_reason VARCHAR REFERENCES lexicon_term(term), + release_status VARCHAR REFERENCES lexicon_term(term), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255) +); + +-- ============================================================================ +-- TRANSDUCER (Continuous Monitoring) +-- ============================================================================ + +CREATE TABLE transducer_observation_block ( + id SERIAL PRIMARY KEY, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE, + parameter_id INTEGER NOT NULL REFERENCES parameter(id) ON DELETE CASCADE, + review_status VARCHAR(100) REFERENCES lexicon_term(term), + start_datetime TIMESTAMP WITH TIME ZONE NOT NULL, + end_datetime TIMESTAMP WITH TIME ZONE NOT NULL, + comment TEXT, + reviewer_id VARCHAR REFERENCES contact(id) ON DELETE CASCADE, + CONSTRAINT uq_transducer_block_thing_status_parameter_time UNIQUE (thing_id, review_status, parameter_id, start_datetime, end_datetime), + CONSTRAINT ck_transducer_block_end_after_start CHECK (end_datetime > start_datetime) +); + +CREATE INDEX idx_transducer_block_thing ON transducer_observation_block(thing_id); +CREATE INDEX idx_transducer_block_parameter ON transducer_observation_block(parameter_id); +CREATE INDEX idx_transducer_block_time ON transducer_observation_block(start_datetime, end_datetime); + +CREATE TABLE transducer_observation ( + id SERIAL PRIMARY KEY, + parameter_id INTEGER NOT NULL REFERENCES parameter(id) ON DELETE CASCADE, + deployment_id INTEGER NOT NULL REFERENCES deployment(id) ON DELETE CASCADE, + observation_datetime TIMESTAMP WITH TIME ZONE NOT NULL, + value FLOAT NOT NULL, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE INDEX idx_transducer_observation_datetime ON transducer_observation(observation_datetime); +CREATE INDEX idx_transducer_observation_parameter ON transducer_observation(parameter_id); + +-- ============================================================================ +-- GROUPS & PROJECTS +-- ============================================================================ + +CREATE TABLE "group" ( + id SERIAL PRIMARY KEY, + name VARCHAR(100) NOT NULL UNIQUE, + description VARCHAR(255), + project_area GEOMETRY(MULTIPOLYGON, 4326), + group_type VARCHAR(100) REFERENCES lexicon_term(term), + parent_group_id INTEGER REFERENCES "group"(id), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE INDEX idx_group_project_area ON "group" USING GIST (project_area); + +CREATE TABLE group_thing_association ( + id SERIAL PRIMARY KEY, + group_id INTEGER NOT NULL REFERENCES "group"(id) ON DELETE CASCADE, + thing_id INTEGER NOT NULL REFERENCES thing(id) ON DELETE CASCADE +); + +-- ============================================================================ +-- ASSETS (File Storage) +-- ============================================================================ + +CREATE TABLE asset ( + id SERIAL PRIMARY KEY, + name VARCHAR, + label VARCHAR, + storage_service VARCHAR, + storage_path VARCHAR, + mime_type VARCHAR, + size INTEGER, + uri VARCHAR, + release_status VARCHAR REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_asset_search ON asset USING GIN (search_vector); + +CREATE TABLE asset_thing_association ( + id SERIAL PRIMARY KEY, + asset_id INTEGER REFERENCES asset(id) ON DELETE CASCADE, + thing_id INTEGER REFERENCES thing(id) ON DELETE CASCADE +); + +-- ============================================================================ +-- PUBLICATIONS & AUTHORS +-- ============================================================================ + +CREATE TABLE pub_author ( + id SERIAL PRIMARY KEY, + name VARCHAR, + affiliation VARCHAR, + search_vector TSVECTOR +); + +CREATE INDEX idx_pub_author_search ON pub_author USING GIN (search_vector); + +CREATE TABLE publication ( + id SERIAL PRIMARY KEY, + title TEXT, + abstract TEXT, + doi VARCHAR UNIQUE, + year INTEGER, + publisher VARCHAR, + url VARCHAR, + publication_type VARCHAR(100) REFERENCES lexicon_term(term), + search_vector TSVECTOR +); + +CREATE INDEX idx_publication_search ON publication USING GIN (search_vector); + +CREATE TABLE pub_author_contact_association ( + author_id INTEGER REFERENCES pub_author(id), + contact_id INTEGER REFERENCES contact(id), + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + PRIMARY KEY (author_id, contact_id) +); + +CREATE TABLE pub_author_publication_association ( + publication_id INTEGER REFERENCES publication(id), + author_id INTEGER REFERENCES pub_author(id), + author_order INTEGER, + created_at TIMESTAMP WITH TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), + created_by_name VARCHAR(255), + created_by_id VARCHAR(255), + updated_by_name VARCHAR(255), + updated_by_id VARCHAR(255), + PRIMARY KEY (publication_id, author_id) +); + +-- ============================================================================ +-- POLYMORPHIC TABLES +-- ============================================================================ + +-- Status History (for Thing, Location, etc.) +CREATE TABLE status_history ( + id SERIAL PRIMARY KEY, + status_type VARCHAR(100) REFERENCES lexicon_term(term), + status_value VARCHAR(100) REFERENCES lexicon_term(term), + start_date DATE NOT NULL, + end_date DATE, + reason TEXT, + target_id INTEGER, + target_table VARCHAR(50), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- Notes (for Thing, Location, etc.) +CREATE TABLE notes ( + id SERIAL PRIMARY KEY, + target_id INTEGER, + target_table VARCHAR, + note_type VARCHAR(100) REFERENCES lexicon_term(term), + content TEXT, + release_status VARCHAR REFERENCES lexicon_term(term) +); + +CREATE INDEX idx_notes_polymorphic_link ON notes(target_id, target_table); + +-- Data Provenance (for Thing, Location, etc.) +CREATE TABLE data_provenance ( + id SERIAL PRIMARY KEY, + target_id INTEGER, + target_table VARCHAR, + field_name VARCHAR, + origin_type VARCHAR(100) REFERENCES lexicon_term(term), + origin_source VARCHAR, + collection_method VARCHAR(100) REFERENCES lexicon_term(term), + accuracy_value FLOAT, + accuracy_unit VARCHAR(100) REFERENCES lexicon_term(term) +); + +CREATE INDEX idx_provenance_targets ON data_provenance(target_id, target_table); + +-- Permission History +CREATE TABLE permission_history ( + id SERIAL PRIMARY KEY, + contact_id INTEGER NOT NULL REFERENCES contact(id) ON DELETE CASCADE, + permission_type VARCHAR(100) REFERENCES lexicon_term(term), + permission_allowed BOOLEAN DEFAULT false, + start_date DATE NOT NULL, + end_date DATE, + notes TEXT, + target_id INTEGER, + target_table VARCHAR(50), + release_status VARCHAR REFERENCES lexicon_term(term) +); + +-- ============================================================================ +-- LEGACY SUPPORT TABLES +-- ============================================================================ + +CREATE TABLE geochronology_age ( + id SERIAL PRIMARY KEY, + location_id INTEGER REFERENCES location(id), + age FLOAT, + age_error FLOAT, + method VARCHAR(100) REFERENCES lexicon_term(term) +); + +CREATE TABLE collaborative_network_well ( + id SERIAL PRIMARY KEY, + thing_id INTEGER REFERENCES thing(id), + actively_monitored BOOLEAN +); + +-- ============================================================================ +-- COMMENTS +-- ============================================================================ + +COMMENT ON TABLE thing IS 'Physical monitoring locations (wells, springs, stream gauges)'; +COMMENT ON TABLE location IS 'Geographic coordinates and metadata for sites'; +COMMENT ON TABLE observation IS 'Individual measurements from samples or sensors'; +COMMENT ON TABLE sample IS 'Samples collected during field activities'; +COMMENT ON TABLE parameter IS 'Controlled vocabulary for measurable properties'; +COMMENT ON TABLE sensor IS 'Equipment inventory for data collection devices'; +COMMENT ON TABLE deployment IS 'Installation history of sensors at things'; +COMMENT ON TABLE lexicon_term IS 'Controlled vocabulary terms'; +COMMENT ON TABLE contact IS 'People and organizations'; +COMMENT ON TABLE field_event IS 'Field visits to monitoring locations'; + +-- ============================================================================ +-- END OF SCHEMA +-- ============================================================================ diff --git a/docker-compose.yml b/docker-compose.yml deleted file mode 100644 index 1c6dec4ef..000000000 --- a/docker-compose.yml +++ /dev/null @@ -1,43 +0,0 @@ -# keep docker-compose.yml in root directory to configure with root .env - -services: - db: - image: postgis/postgis:17-3.5 - platform: linux/amd64 - environment: - - POSTGRES_USER=${POSTGRES_USER} - - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} - - POSTGRES_DB=${POSTGRES_DB} - ports: - - 54321:5432 - volumes: - - postgres_data:/var/lib/postgresql/data - healthcheck: - test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"] - interval: 2s - timeout: 5s - retries: 20 - - app: - build: - context: . - dockerfile: ./docker/app/Dockerfile - environment: - - POSTGRES_USER=${POSTGRES_USER} - - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} - - POSTGRES_DB=${POSTGRES_DB} - - POSTGRES_HOST=db - - MODE=${MODE} - - AUTHENTIK_DISABLE_AUTHENTICATION=${AUTHENTIK_DISABLE_AUTHENTICATION} - ports: - - 8000:8000 - depends_on: - db: - condition: service_healthy # <-- wait for DB to be ready - links: - - db - volumes: - - .:/app - -volumes: - postgres_data: diff --git a/docker/app/Dockerfile b/docker/app/Dockerfile deleted file mode 100644 index c39141b2a..000000000 --- a/docker/app/Dockerfile +++ /dev/null @@ -1,43 +0,0 @@ -FROM python:3.13-slim AS base - -# install system dependencies for psycopg2 and uv -RUN apt-get update && apt-get install -y curl gcc libpq-dev - -# install curl -RUN apt-get update && apt-get install -y curl - -# install uv (fast dependency manager) -RUN curl -Ls https://astral.sh/uv/install.sh | sh -s -- v0.7.17 && \ - cp /root/.local/bin/uv /usr/local/bin/uv - -# install postgresql client -RUN apt-get update && apt-get install -y postgresql-client - -# set workdir -WORKDIR /app - -# copy the full project source -COPY . . - -# Define an optional build argument -ARG INSTALL_DEV=false -ENV INSTALL_DEV=${INSTALL_DEV} - -# Install dependencies using uv (conditionally include dev dependencies) -ENV UV_PROJECT_ENVIRONMENT="/usr/local/" -RUN if [ "$INSTALL_DEV" = "true" ]; then \ - echo "Installing all groups (including dev)..." && \ - uv sync --locked --all-groups; \ - else \ - echo "Installing only production dependencies..." && \ - uv sync --locked; \ - fi - -# expose FastAPI's default dev port -EXPOSE 8000 - -# set environment variables for database connection -RUN chmod +x entrypoint.sh - -# default command (run database migrations and the FastAPI development server) -CMD ["sh", "entrypoint.sh"] diff --git a/entrypoint.sh b/entrypoint.sh index 662487618..15fab3c42 100644 --- a/entrypoint.sh +++ b/entrypoint.sh @@ -1,12 +1,31 @@ #!/bin/sh -# Wait for PostgreSQL to be ready -until PGPASSWORD="$POSTGRES_PASSWORD" pg_isready -h db -p 5432 -U "$POSTGRES_USER"; do - echo "Waiting for postgres..." - sleep 2 -done -echo "PostgreSQL is ready!" +# Wait for PostgreSQL to be ready (only in local docker-compose environment) +# Skip this check if DATABASE_URL is set (e.g., on Render) or POSTGRES_HOST is not "db" +if [ -z "$DATABASE_URL" ] && [ "${POSTGRES_HOST:-db}" = "db" ]; then + echo "Checking local PostgreSQL readiness..." + until PGPASSWORD="$POSTGRES_PASSWORD" pg_isready -h db -p 5432 -U "$POSTGRES_USER"; do + echo "Waiting for postgres..." + sleep 2 + done + echo "PostgreSQL is ready!" +else + echo "Using DATABASE_URL or external PostgreSQL, skipping local db wait..." +fi echo "Applying migrations..." alembic upgrade head + echo "Starting the application..." -uvicorn main:app --host 0.0.0.0 --port 8000 --reload \ No newline at end of file +# Use PORT environment variable (Render provides this), default to 8000 +PORT=${PORT:-8000} +echo "Starting server on port $PORT" + +# Production server: gunicorn with uvicorn workers +# Use --reload only if RELOAD_ENABLED=true (for local development) +if [ "${RELOAD_ENABLED:-false}" = "true" ]; then + echo "Running in development mode with auto-reload" + uvicorn main:app --host 0.0.0.0 --port "$PORT" --reload +else + echo "Running in production mode with gunicorn" + gunicorn main:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:"$PORT" +fi \ No newline at end of file diff --git a/render.yaml b/render.yaml new file mode 100644 index 000000000..a3ef31bda --- /dev/null +++ b/render.yaml @@ -0,0 +1,99 @@ +# Render Blueprint for OcotilloAPI - FREE TIER (Native Python) +# NOTE: Free tier has 30-day database expiration and service spin-down after 15 min inactivity + +envVarGroups: + - name: ocotillo-shared + envVars: + - key: DB_POOL_SIZE + value: 5 + - key: DB_MAX_OVERFLOW + value: 5 + - key: TRANSFER_PARALLEL + value: 1 + - key: TRANSFER_WELL_SCREENS + value: "True" + - key: TRANSFER_SENSORS + value: "True" + - key: TRANSFER_CONTACTS + value: "True" + - key: TRANSFER_WATERLEVELS + value: "True" + - key: TRANSFER_WATERLEVELS_PRESSURE + value: "True" + - key: TRANSFER_WATERLEVELS_ACOUSTIC + value: "True" + - key: TRANSFER_LINK_IDS + value: "True" + - key: TRANSFER_GROUPS + value: "True" + - key: TRANSFER_ASSETS + value: "False" + - key: TRANSFER_SURFACE_WATER_DATA + value: "True" + - key: TRANSFER_HYDRAULICS_DATA + value: "True" + - key: TRANSFER_CHEMISTRY_SAMPLEINFO + value: "True" + - key: TRANSFER_RADIONUCLIDES + value: "True" + - key: TRANSFER_NGWMN_VIEWS + value: "True" + - key: TRANSFER_WATERLEVELS_PRESSURE_DAILY + value: "True" + - key: TRANSFER_WEATHER_DATA + value: "True" + - key: TRANSFER_MINOR_TRACE_CHEMISTRY + value: "True" + + - name: ocotillo-staging + envVars: + - key: MODE + value: development + - key: AUTHENTIK_DISABLE_AUTHENTICATION + value: 1 + - key: AUTHENTIK_URL + sync: false + - key: AUTHENTIK_CLIENT_ID + sync: false + - key: AUTHENTIK_AUTHORIZE_URL + sync: false + - key: AUTHENTIK_TOKEN_URL + sync: false + - key: SESSION_SECRET_KEY + generateValue: true + - key: GCS_BUCKET_NAME + sync: false + - key: GOOGLE_APPLICATION_CREDENTIALS + sync: false + +# Database - FREE TIER +databases: + - name: ocotillo-db-staging + databaseName: ocotillo_staging + user: ocotillo_staging + plan: free + region: oregon + postgresMajorVersion: "17" + ipAllowList: [] + +# Web service - FREE TIER with Native Python +services: + - type: web + name: ocotillo-api-staging + runtime: python + region: oregon + plan: free + branch: render-deploy + buildCommand: pip install uv && uv sync --locked + startCommand: alembic upgrade head && gunicorn main:app --workers 2 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT + envVars: + - key: DATABASE_URL + fromDatabase: + name: ocotillo-db-staging + property: connectionString + - key: PYTHON_VERSION + value: "3.13.1" + - fromGroup: ocotillo-shared + - fromGroup: ocotillo-staging + healthCheckPath: /health/ready + autoDeploy: true