mirror of
https://origin.cursor.com/mrdevmx/panels.git
synced 2026-10-09 10:43:18 +00:00
db/infra: ETL de datos, backups y limpieza de despliegue (fase 5-7)
Fase 5 (migracion de datos): - api/scripts/migrate-sqlite-to-postgres.ts: ETL unico SQLite -> Postgres. Pre-flight (CURP/RFC duplicados case-insensitive, FKs huerfanas, tenant_id invalido, fechas mal formateadas) -> carga por base/esquema con credenciales _owner (setval de secuencias, OVERRIDING SYSTEM VALUE, ON CONFLICT DO NOTHING idempotente) -> verificacion de conteos y sumas de dinero con tolerancia. Probado end-to-end contra un dataset SQLite legacy sintetico y un Postgres limpio (solo schema+catalogos, sin contexto dev): 100% de las filas migradas, sumas de dinero exactas, snapshot de uploaded_by/created_by resuelto correctamente. - db/RUNBOOK-corte.md: checklist go/no-go para el corte por ambiente. - Fix de bug real encontrado al probar: sin --context-filter explicito, Liquibase corre TAMBIEN los changesets context=dev (comportamiento por defecto, no "modo seguro") -- documentado en db/README.md con el ejemplo correcto (--context-filter='!dev' para staging/produccion). Fase 5b: patron de compensacion para createTenant ya resuelto en el rewrite de saas.ts (fase 3/4). Fase 6 (backups): db/backups/ con plantillas pgBackRest por base, politica de retencion, nota de persistencia minima de Redis (RDB, sin retencion de negocio), checklist de simulacro de restauracion mensual, y la validacion pendiente de si Coolify permite WAL archiving antes de comprometerse a PITR real. Fase 7 (limpieza y CI): - Dockerfile.api simplificado (sin JRE/Liquibase/FFI). Dockerfile.migrate y Dockerfile.provision nuevos, de un solo uso, para el paso explicito de deploy (nunca sirven trafico). - docker-compose.yml: postgres+redis+provision+migrate para dev local end-to-end; api ya no arranca hasta que migrate termina bien. - docs/coolify.md y .env.example actualizados al modelo de 2 bases + Redis + Contabo. - api/schema.sql eliminado (desincronizado, competia con Liquibase como fuente de verdad). - .github/workflows/ci.yml: Postgres+Redis de servicio, aprovisiona roles/ACLs, dry-run de Liquibase (updateSQL) antes de aplicar, verify-isolation.sh, deno check + deno test, build de los dos frontends. Co-authored-by: alberto.martinez <alberto.martinez@mrdev.mx>
This commit is contained in:
parent
493829d028
commit
d36c287a82
19 changed files with 1223 additions and 451 deletions
56
.env.example
56
.env.example
|
|
@ -1,17 +1,16 @@
|
|||
# =============================================================================
|
||||
# Copiar a `.env` (gitignored) para local / Coolify.
|
||||
# NUNCA subas `.env` ni `data/*.db` al repositorio.
|
||||
# NUNCA subas `.env` ni `data/*` al repositorio.
|
||||
#
|
||||
# Generar cada password/secreto con: openssl rand -hex 24
|
||||
# =============================================================================
|
||||
|
||||
# --- OBLIGATORIAS (producción) ---
|
||||
# Generar ejemplos:
|
||||
# SESSION_SECRET: openssl rand -hex 32
|
||||
# DOCS_KEY: openssl rand -hex 32 (exactamente 64 hex)
|
||||
SESSION_SECRET=
|
||||
DOCS_KEY=
|
||||
# ^ DOCS_KEY debe ser EXACTAMENTE 64 hex (32 bytes): openssl rand -hex 32
|
||||
SEED_PASSWORD=
|
||||
# Si la contraseña tiene `$`, en Docker Compose escríbelo como `$$` (ej. `p$$ass` → `p$ass`).
|
||||
# Opcional one-shot: SEED_SYNC_PASSWORD=true (actualiza password de admin al arrancar; luego quítalo)
|
||||
|
||||
PANEL_LOGIN_URL=https://app.tudominio/login
|
||||
PORT=8000
|
||||
|
|
@ -25,9 +24,54 @@ CORS_ORIGINS=
|
|||
API_KEY=
|
||||
VCARD_BASE=https://vcard.arctec.com.mx?info=
|
||||
|
||||
# SMTP (opcional; también se puede configurar en SaaS → /smtp y queda en platform.db)
|
||||
# SMTP (opcional; también se puede configurar en SaaS → /smtp y queda en panels_platform)
|
||||
SMTP_HOST=
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=
|
||||
SMTP_PASS=
|
||||
SMTP_FROM=PANELS <noreply@tudominio>
|
||||
|
||||
# =============================================================================
|
||||
# Postgres (monolito modular: panels_platform separada de panels_product)
|
||||
# En Coolify, estos apuntan al recurso Postgres gestionado del ambiente
|
||||
# (ver db/provision/README.md). Para `docker compose up` local, además hay
|
||||
# que definir las contraseñas de cada rol (siguiente bloque) -- el compose
|
||||
# arma las URLs solo.
|
||||
# =============================================================================
|
||||
DATABASE_URL_PLATFORM=
|
||||
DATABASE_URL_IAM=
|
||||
DATABASE_URL_CORE=
|
||||
# _OWNER: solo para el paso de deploy (Liquibase) y scripts (bootstrap/ETL).
|
||||
# Nunca usar el rol _owner para el tráfico normal de la API.
|
||||
DATABASE_URL_PLATFORM_OWNER=
|
||||
DATABASE_URL_IAM_OWNER=
|
||||
DATABASE_URL_CORE_OWNER=
|
||||
|
||||
# --- Solo para `docker compose up` local (provisiona Postgres/Redis propios) ---
|
||||
POSTGRES_SUPERUSER_PASSWORD=
|
||||
PLATFORM_OWNER_PASSWORD=
|
||||
PLATFORM_APP_PASSWORD=
|
||||
IAM_OWNER_PASSWORD=
|
||||
IAM_APP_PASSWORD=
|
||||
CORE_OWNER_PASSWORD=
|
||||
CORE_APP_PASSWORD=
|
||||
|
||||
# =============================================================================
|
||||
# Redis (sesiones + cache, con ACL por módulo -- ver db/provision/05-redis-acl.sh)
|
||||
# =============================================================================
|
||||
REDIS_URL_IAM=
|
||||
REDIS_URL_CORE=
|
||||
# --- Solo para `docker compose up` local ---
|
||||
IAM_REDIS_PASSWORD=
|
||||
CORE_REDIS_PASSWORD=
|
||||
|
||||
# =============================================================================
|
||||
# Contabo Object Storage (Fase 4c) -- expedientes/PDFs/logos cifrados.
|
||||
# Sin esto, cae a disco local bajo el volumen panel-data (solo dev; en
|
||||
# producción un volumen local no escala horizontalmente, ver el plan).
|
||||
# =============================================================================
|
||||
S3_ENDPOINT=
|
||||
S3_BUCKET=
|
||||
S3_REGION=
|
||||
S3_ACCESS_KEY_ID=
|
||||
S3_SECRET_ACCESS_KEY=
|
||||
|
|
|
|||
111
.github/workflows/ci.yml
vendored
Normal file
111
.github/workflows/ci.yml
vendored
Normal file
|
|
@ -0,0 +1,111 @@
|
|||
name: CI
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
api:
|
||||
runs-on: ubuntu-latest
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_PASSWORD: postgres
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd "pg_isready -U postgres"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
redis:
|
||||
image: redis:7
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 10
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: denoland/setup-deno@v2
|
||||
with:
|
||||
deno-version: v2.x
|
||||
|
||||
- uses: actions/setup-java@v4
|
||||
with:
|
||||
distribution: temurin
|
||||
java-version: "21"
|
||||
|
||||
- name: Aprovisionar roles/esquemas Postgres + ACLs Redis
|
||||
env:
|
||||
PGHOST: localhost
|
||||
PGPASSWORD: postgres
|
||||
PLATFORM_OWNER_PASSWORD: ci-platform-owner
|
||||
PLATFORM_APP_PASSWORD: ci-platform-app
|
||||
IAM_OWNER_PASSWORD: ci-iam-owner
|
||||
IAM_APP_PASSWORD: ci-iam-app
|
||||
CORE_OWNER_PASSWORD: ci-core-owner
|
||||
CORE_APP_PASSWORD: ci-core-app
|
||||
REDIS_ADMIN_URL: redis://localhost:6379
|
||||
IAM_REDIS_PASSWORD: ci-iam-redis
|
||||
CORE_REDIS_PASSWORD: ci-core-redis
|
||||
run: |
|
||||
sudo apt-get update -qq && sudo apt-get install -y -qq postgresql-client redis-tools
|
||||
./db/provision/docker-provision.sh
|
||||
./db/provision/05-redis-acl.sh
|
||||
|
||||
- name: Dry-run de Liquibase (updateSQL) -- no debe fallar antes de aplicar
|
||||
env:
|
||||
DATABASE_URL_PLATFORM_OWNER: postgresql://panels_platform_owner:ci-platform-owner@localhost:5432/panels_platform
|
||||
DATABASE_URL_IAM_OWNER: postgresql://panels_iam_owner:ci-iam-owner@localhost:5432/panels_product
|
||||
DATABASE_URL_CORE_OWNER: postgresql://panels_core_owner:ci-core-owner@localhost:5432/panels_product
|
||||
run: ./db/update.sh all --context-filter='!dev' -- updateSQL
|
||||
|
||||
- name: Aplicar migraciones (con datos de demo, para los tests)
|
||||
env:
|
||||
DATABASE_URL_PLATFORM_OWNER: postgresql://panels_platform_owner:ci-platform-owner@localhost:5432/panels_platform
|
||||
DATABASE_URL_IAM_OWNER: postgresql://panels_iam_owner:ci-iam-owner@localhost:5432/panels_product
|
||||
DATABASE_URL_CORE_OWNER: postgresql://panels_core_owner:ci-core-owner@localhost:5432/panels_product
|
||||
run: ./db/update.sh all --context-filter=dev
|
||||
|
||||
- name: Verificar aislamiento (roles/RLS/ACLs)
|
||||
env:
|
||||
DATABASE_URL_PLATFORM: postgresql://panels_platform_app:ci-platform-app@localhost:5432/panels_platform
|
||||
DATABASE_URL_IAM: postgresql://panels_iam_app:ci-iam-app@localhost:5432/panels_product
|
||||
DATABASE_URL_CORE: postgresql://panels_core_app:ci-core-app@localhost:5432/panels_product
|
||||
REDIS_URL_IAM: redis://panels_iam_redis:ci-iam-redis@localhost:6379
|
||||
REDIS_URL_CORE: redis://panels_core_redis:ci-core-redis@localhost:6379
|
||||
run: ./db/provision/verify-isolation.sh
|
||||
|
||||
- name: deno check
|
||||
working-directory: api
|
||||
run: deno check main.ts
|
||||
|
||||
- name: deno test
|
||||
working-directory: api
|
||||
env:
|
||||
DATABASE_URL_CORE_OWNER: postgresql://panels_core_owner:ci-core-owner@localhost:5432/panels_product
|
||||
run: deno test --allow-net --allow-read --allow-write --allow-env
|
||||
|
||||
web:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
app: [web-panel, web-saas]
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
cache: npm
|
||||
cache-dependency-path: ${{ matrix.app }}/package-lock.json
|
||||
- run: npm ci
|
||||
working-directory: ${{ matrix.app }}
|
||||
- run: npm run build
|
||||
working-directory: ${{ matrix.app }}
|
||||
|
|
@ -1,26 +1,14 @@
|
|||
# API PANELS — Deno + SQLite FFI + Liquibase/JRE (Debian slim, no Alpine)
|
||||
# API PANELS — Deno + Postgres (postgres.js) + Redis. Ya no necesita FFI de
|
||||
# SQLite ni JRE/Liquibase en la imagen de runtime -- las migraciones corren
|
||||
# como paso explícito de deploy (ver db/update.sh), no al arrancar la app.
|
||||
FROM denoland/deno:2.9.5
|
||||
|
||||
USER root
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
openjdk-21-jre-headless \
|
||||
curl \
|
||||
unzip \
|
||||
bash \
|
||||
ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY db/ ./db/
|
||||
COPY api/ ./api/
|
||||
COPY web-panel/public/brand/panel-icon-symmetric.png ./api/assets/email/logo.png
|
||||
|
||||
RUN chmod +x ./db/bootstrap-tools.sh \
|
||||
&& ./db/bootstrap-tools.sh \
|
||||
&& mkdir -p /app/data \
|
||||
&& chown -R deno:deno /app
|
||||
RUN mkdir -p /app/data && chown -R deno:deno /app
|
||||
|
||||
USER deno
|
||||
WORKDIR /app/api
|
||||
|
|
@ -29,7 +17,7 @@ RUN deno cache main.ts
|
|||
ENV PORT=8000
|
||||
EXPOSE 8000
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=45s --retries=3 \
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
|
||||
CMD ["deno", "eval", "const r=await fetch('http://127.0.0.1:'+(Deno.env.get('PORT')||'8000')+'/v1/health'); if(!r.ok) Deno.exit(1)"]
|
||||
|
||||
CMD ["deno", "run", "--allow-net", "--allow-read", "--allow-write", "--allow-env", "--allow-ffi", "--allow-run", "main.ts"]
|
||||
CMD ["deno", "run", "--allow-net", "--allow-read", "--allow-write", "--allow-env", "main.ts"]
|
||||
|
|
|
|||
17
Dockerfile.migrate
Normal file
17
Dockerfile.migrate
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
# PANELS — imagen mínima para el paso EXPLÍCITO de deploy que aplica
|
||||
# Liquibase (Postgres). Separada de Dockerfile.api a propósito: la imagen
|
||||
# que sirve tráfico real no necesita JRE/Liquibase (Fase 2/7 del plan de
|
||||
# migración) -- esto solo se usa como job de un solo uso antes de levantar
|
||||
# `api`, nunca como servicio de larga duración.
|
||||
FROM eclipse-temurin:21-jre-jammy
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends curl unzip bash ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
COPY db/ ./db/
|
||||
RUN chmod +x ./db/bootstrap-tools.sh ./db/update.sh && ./db/bootstrap-tools.sh
|
||||
|
||||
ENTRYPOINT ["./db/update.sh"]
|
||||
CMD ["all", "--context-filter=dev"]
|
||||
11
Dockerfile.provision
Normal file
11
Dockerfile.provision
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
# PANELS — imagen mínima de un solo uso para aprovisionar roles/bases/
|
||||
# esquemas de Postgres y ACLs de Redis en desarrollo local (ver
|
||||
# db/provision/). No se usa en producción -- ahí este paso se corre a
|
||||
# mano una vez contra el Postgres/Redis reales de Coolify (ver
|
||||
# db/provision/README.md).
|
||||
FROM alpine:3.20
|
||||
RUN apk add --no-cache postgresql16-client redis bash
|
||||
WORKDIR /app
|
||||
COPY db/provision/ ./db/provision/
|
||||
RUN chmod +x ./db/provision/*.sh
|
||||
ENTRYPOINT ["/bin/bash"]
|
||||
|
|
@ -1,9 +1,19 @@
|
|||
{
|
||||
"version": "5",
|
||||
"specifiers": {
|
||||
"jsr:@db/sqlite@0.12": "0.12.0",
|
||||
"jsr:@denosaurs/plug@1": "1.1.0",
|
||||
"jsr:@hono/hono@4": "4.13.2",
|
||||
"jsr:@std/assert@0.217": "0.217.0",
|
||||
"jsr:@std/assert@1": "1.0.19",
|
||||
"jsr:@std/encoding@1": "1.0.11",
|
||||
"jsr:@std/fmt@1": "1.0.10",
|
||||
"jsr:@std/fs@1": "1.0.24",
|
||||
"jsr:@std/internal@^1.0.12": "1.0.14",
|
||||
"jsr:@std/internal@^1.0.14": "1.0.14",
|
||||
"jsr:@std/path@0.217": "0.217.0",
|
||||
"jsr:@std/path@1": "1.1.6",
|
||||
"jsr:@std/path@^1.1.5": "1.1.6",
|
||||
"npm:@aws-sdk/client-s3@3": "3.1124.0",
|
||||
"npm:@types/node@*": "22.15.15",
|
||||
"npm:nodemailer@6.9.16": "6.9.16",
|
||||
|
|
@ -14,17 +24,61 @@
|
|||
"npm:xlsx@0.18.5": "0.18.5"
|
||||
},
|
||||
"jsr": {
|
||||
"@db/sqlite@0.12.0": {
|
||||
"integrity": "dd1ef7f621ad50fc1e073a1c3609c4470bd51edc0994139c5bf9851de7a6d85f",
|
||||
"dependencies": [
|
||||
"jsr:@denosaurs/plug",
|
||||
"jsr:@std/path@0.217"
|
||||
]
|
||||
},
|
||||
"@denosaurs/plug@1.1.0": {
|
||||
"integrity": "eb2f0b7546c7bca2000d8b0282c54d50d91cf6d75cb26a80df25a6de8c4bc044",
|
||||
"dependencies": [
|
||||
"jsr:@std/encoding",
|
||||
"jsr:@std/fmt",
|
||||
"jsr:@std/fs",
|
||||
"jsr:@std/path@1"
|
||||
]
|
||||
},
|
||||
"@hono/hono@4.13.2": {
|
||||
"integrity": "715d8cc1b6b5d6b9e6a7519059778d775b2d79fffdc026ac0ccdd9971e75809c"
|
||||
},
|
||||
"@std/assert@0.217.0": {
|
||||
"integrity": "c98e279362ca6982d5285c3b89517b757c1e3477ee9f14eb2fdf80a45aaa9642"
|
||||
},
|
||||
"@std/assert@1.0.19": {
|
||||
"integrity": "eaada96ee120cb980bc47e040f82814d786fe8162ecc53c91d8df60b8755991e",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal"
|
||||
"jsr:@std/internal@^1.0.12"
|
||||
]
|
||||
},
|
||||
"@std/encoding@1.0.11": {
|
||||
"integrity": "e7cef2f0b3153bccc17431e7c864a1de03a4b6d9647389c43e08aaa775d37385"
|
||||
},
|
||||
"@std/fmt@1.0.10": {
|
||||
"integrity": "90dfba288802ac6de82fb31d0917eb9e4450b9925b954d5e51fc29ac07419db5"
|
||||
},
|
||||
"@std/fs@1.0.24": {
|
||||
"integrity": "f3061b45b81673a2bece689da041df32d174be064c89eb6397fb5718d3fb7877",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal@^1.0.14",
|
||||
"jsr:@std/path@^1.1.5"
|
||||
]
|
||||
},
|
||||
"@std/internal@1.0.14": {
|
||||
"integrity": "291516b3d4c35024d6ffbc0a9df5bf4c64116e05b50012cf846710152d2ffdf7"
|
||||
},
|
||||
"@std/path@0.217.0": {
|
||||
"integrity": "1217cc25534bca9a2f672d7fe7c6f356e4027df400c0e85c0ef3e4343bc67d11",
|
||||
"dependencies": [
|
||||
"jsr:@std/assert@0.217"
|
||||
]
|
||||
},
|
||||
"@std/path@1.1.6": {
|
||||
"integrity": "c68485c2a4dfbb5ae3cc74fae4e8c4e5d874cf8a8ed12927917235c758b46cbe",
|
||||
"dependencies": [
|
||||
"jsr:@std/internal@^1.0.14"
|
||||
]
|
||||
}
|
||||
},
|
||||
"npm": {
|
||||
|
|
|
|||
384
api/schema.sql
384
api/schema.sql
|
|
@ -1,384 +0,0 @@
|
|||
PRAGMA foreign_keys = ON;
|
||||
PRAGMA journal_mode = WAL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS users (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
username TEXT NOT NULL UNIQUE,
|
||||
password_hash TEXT NOT NULL,
|
||||
display_name TEXT NOT NULL,
|
||||
company_id INTEGER REFERENCES companies(id),
|
||||
tenant_id INTEGER,
|
||||
role TEXT NOT NULL DEFAULT 'user',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS companies (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
code TEXT NOT NULL UNIQUE,
|
||||
name TEXT NOT NULL,
|
||||
parent_id INTEGER REFERENCES companies(id),
|
||||
kind TEXT NOT NULL DEFAULT 'sub' CHECK (kind IN ('principal', 'sub')),
|
||||
status TEXT NOT NULL DEFAULT 'activo' CHECK (status IN ('activo', 'inactivo')),
|
||||
tenant_id INTEGER,
|
||||
registro_patronal TEXT NOT NULL DEFAULT '',
|
||||
razon_social TEXT NOT NULL DEFAULT '',
|
||||
nombre_comercial TEXT NOT NULL DEFAULT '',
|
||||
rfc TEXT NOT NULL DEFAULT '',
|
||||
regimen_fiscal TEXT NOT NULL DEFAULT '',
|
||||
clase_riesgo TEXT NOT NULL DEFAULT '',
|
||||
domicilio_fiscal TEXT NOT NULL DEFAULT '',
|
||||
codigo_postal TEXT NOT NULL DEFAULT '',
|
||||
ciudad TEXT NOT NULL DEFAULT '',
|
||||
estado TEXT NOT NULL DEFAULT '',
|
||||
telefono TEXT NOT NULL DEFAULT '',
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
representante_legal TEXT NOT NULL DEFAULT '',
|
||||
giro TEXT NOT NULL DEFAULT '',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS risk_levels (
|
||||
code TEXT PRIMARY KEY,
|
||||
label TEXT NOT NULL,
|
||||
color TEXT NOT NULL,
|
||||
text_color TEXT NOT NULL DEFAULT '#FFFFFF'
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS badge_themes (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
layout TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS projects (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
code TEXT NOT NULL UNIQUE,
|
||||
name TEXT NOT NULL,
|
||||
address TEXT NOT NULL DEFAULT '',
|
||||
stage TEXT NOT NULL DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'activo' CHECK (status IN ('activo', 'pausado', 'concluido', 'cancelado')),
|
||||
theme_id TEXT NOT NULL REFERENCES badge_themes(id),
|
||||
logo_left_path TEXT,
|
||||
logo_right_path TEXT,
|
||||
company_id INTEGER REFERENCES companies(id),
|
||||
tenant_id INTEGER,
|
||||
contract_amount REAL,
|
||||
start_date TEXT,
|
||||
end_date TEXT,
|
||||
resident_name TEXT NOT NULL DEFAULT '',
|
||||
siroc TEXT NOT NULL DEFAULT '',
|
||||
payroll_tax_pct REAL NOT NULL DEFAULT 4,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS workers (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
first_name TEXT NOT NULL,
|
||||
middle_name TEXT,
|
||||
last_name_p TEXT NOT NULL,
|
||||
last_name_m TEXT NOT NULL,
|
||||
curp TEXT NOT NULL COLLATE NOCASE,
|
||||
rfc TEXT NOT NULL COLLATE NOCASE,
|
||||
nss TEXT NOT NULL,
|
||||
phone TEXT NOT NULL,
|
||||
email TEXT NOT NULL,
|
||||
address TEXT NOT NULL,
|
||||
blood_type TEXT,
|
||||
hire_type TEXT NOT NULL,
|
||||
company_id INTEGER REFERENCES companies(id),
|
||||
tenant_id INTEGER,
|
||||
position TEXT NOT NULL,
|
||||
risk_code TEXT NOT NULL REFERENCES risk_levels(code),
|
||||
work_type TEXT NOT NULL CHECK (work_type IN ('N', 'D')),
|
||||
daily_wage REAL NOT NULL DEFAULT 0,
|
||||
needs_badge INTEGER NOT NULL DEFAULT 1,
|
||||
status TEXT NOT NULL DEFAULT 'activo' CHECK (status IN ('activo', 'baja')),
|
||||
pipeline_status TEXT NOT NULL DEFAULT 'incompleto',
|
||||
imss_status TEXT NOT NULL DEFAULT 'sin_alta' CHECK (imss_status IN ('sin_alta', 'alta', 'baja_imss')),
|
||||
imss_company_id INTEGER REFERENCES companies(id),
|
||||
imss_alta_at TEXT,
|
||||
imss_baja_at TEXT,
|
||||
last_rehire_at TEXT,
|
||||
vcard_password_enc TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_workers_curp ON workers(curp);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_workers_rfc ON workers(rfc);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_workers_nss ON workers(nss);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS assignments (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id) ON DELETE CASCADE,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
start_date TEXT NOT NULL,
|
||||
end_date TEXT,
|
||||
UNIQUE (worker_id, project_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS document_types (
|
||||
code TEXT PRIMARY KEY,
|
||||
label TEXT NOT NULL,
|
||||
required INTEGER NOT NULL DEFAULT 1,
|
||||
validity_mode TEXT NOT NULL DEFAULT 'none' CHECK (validity_mode IN ('none', 'freshness', 'expiry')),
|
||||
freshness_days INTEGER,
|
||||
requires_issued_at INTEGER NOT NULL DEFAULT 0,
|
||||
requires_expires_at INTEGER NOT NULL DEFAULT 0,
|
||||
category TEXT NOT NULL DEFAULT 'identidad'
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS documents (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id) ON DELETE CASCADE,
|
||||
type_code TEXT NOT NULL REFERENCES document_types(code),
|
||||
original_name TEXT NOT NULL,
|
||||
mime TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
iv TEXT NOT NULL,
|
||||
storage_name TEXT NOT NULL,
|
||||
is_current INTEGER NOT NULL DEFAULT 1,
|
||||
parse_status TEXT NOT NULL DEFAULT 'manual',
|
||||
issued_at TEXT,
|
||||
expires_at TEXT,
|
||||
imss_company_id INTEGER REFERENCES companies(id),
|
||||
imss_alta_at TEXT,
|
||||
uploaded_by INTEGER REFERENCES users(id),
|
||||
uploaded_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS project_document_types (
|
||||
code TEXT PRIMARY KEY,
|
||||
label TEXT NOT NULL,
|
||||
required INTEGER NOT NULL DEFAULT 1,
|
||||
category TEXT NOT NULL DEFAULT 'contrato'
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS project_documents (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
type_code TEXT NOT NULL REFERENCES project_document_types(code),
|
||||
original_name TEXT NOT NULL,
|
||||
mime TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
iv TEXT NOT NULL,
|
||||
storage_name TEXT NOT NULL,
|
||||
is_current INTEGER NOT NULL DEFAULT 1,
|
||||
parse_status TEXT NOT NULL DEFAULT 'manual',
|
||||
uploaded_by INTEGER REFERENCES users(id),
|
||||
uploaded_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS company_document_types (
|
||||
code TEXT PRIMARY KEY,
|
||||
label TEXT NOT NULL,
|
||||
required INTEGER NOT NULL DEFAULT 0,
|
||||
category TEXT NOT NULL DEFAULT 'otro'
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS company_documents (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
company_id INTEGER NOT NULL REFERENCES companies(id) ON DELETE CASCADE,
|
||||
type_code TEXT NOT NULL REFERENCES company_document_types(code),
|
||||
original_name TEXT NOT NULL,
|
||||
mime TEXT NOT NULL,
|
||||
size_bytes INTEGER NOT NULL,
|
||||
sha256 TEXT NOT NULL,
|
||||
iv TEXT NOT NULL,
|
||||
storage_name TEXT NOT NULL,
|
||||
is_current INTEGER NOT NULL DEFAULT 1,
|
||||
parse_status TEXT NOT NULL DEFAULT 'manual',
|
||||
uploaded_by INTEGER REFERENCES users(id),
|
||||
uploaded_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS badge_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id),
|
||||
status TEXT NOT NULL DEFAULT 'done',
|
||||
pdf_path TEXT,
|
||||
created_by INTEGER REFERENCES users(id),
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS badge_job_people (
|
||||
job_id INTEGER NOT NULL REFERENCES badge_jobs(id) ON DELETE CASCADE,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id),
|
||||
delivered INTEGER NOT NULL DEFAULT 0,
|
||||
delivered_at TEXT,
|
||||
PRIMARY KEY (job_id, worker_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS loans (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id) ON DELETE CASCADE,
|
||||
amount REAL NOT NULL,
|
||||
delivered REAL NOT NULL DEFAULT 0,
|
||||
balance REAL NOT NULL,
|
||||
weekly_payment REAL NOT NULL,
|
||||
note TEXT,
|
||||
commission_pct REAL NOT NULL DEFAULT 0,
|
||||
commission_amount REAL NOT NULL DEFAULT 0,
|
||||
plan TEXT NOT NULL DEFAULT 'single',
|
||||
installments_n INTEGER NOT NULL DEFAULT 1,
|
||||
first_due TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS attendance (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id) ON DELETE CASCADE,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
work_date TEXT NOT NULL,
|
||||
present INTEGER NOT NULL DEFAULT 1,
|
||||
UNIQUE (worker_id, project_id, work_date)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_periods (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id),
|
||||
week_start TEXT NOT NULL,
|
||||
week_end TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'draft',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_lines (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
period_id INTEGER NOT NULL REFERENCES payroll_periods(id) ON DELETE CASCADE,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id),
|
||||
days REAL NOT NULL DEFAULT 0,
|
||||
daily_wage REAL NOT NULL,
|
||||
gross REAL NOT NULL,
|
||||
discounts REAL NOT NULL DEFAULT 0,
|
||||
loan_payment REAL NOT NULL DEFAULT 0,
|
||||
net REAL NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_settings (
|
||||
tenant_id INTEGER NOT NULL PRIMARY KEY,
|
||||
loan_commission_enabled INTEGER NOT NULL DEFAULT 1,
|
||||
loan_commission_pct REAL NOT NULL DEFAULT 10,
|
||||
loan_small_max REAL NOT NULL DEFAULT 500
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_weeks (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
week_start TEXT NOT NULL,
|
||||
week_end TEXT NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'draft' CHECK (status IN ('draft', 'assembled', 'paid')),
|
||||
required_net REAL NOT NULL DEFAULT 0,
|
||||
payable_net REAL NOT NULL DEFAULT 0,
|
||||
assembled_at TEXT,
|
||||
paid_at TEXT,
|
||||
tenant_id INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_sheets (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
week_id INTEGER NOT NULL REFERENCES payroll_weeks(id) ON DELETE CASCADE,
|
||||
kind TEXT NOT NULL CHECK (kind IN ('obra', 'destajo', 'admin')),
|
||||
project_id INTEGER REFERENCES projects(id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS payroll_week_lines (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
sheet_id INTEGER NOT NULL REFERENCES payroll_sheets(id) ON DELETE CASCADE,
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id),
|
||||
destajo_cut_line_id INTEGER,
|
||||
project_id INTEGER REFERENCES projects(id),
|
||||
days REAL NOT NULL DEFAULT 0,
|
||||
daily_wage REAL NOT NULL DEFAULT 0,
|
||||
qty_planned REAL NOT NULL DEFAULT 0,
|
||||
qty_actual REAL NOT NULL DEFAULT 0,
|
||||
qty_extra REAL NOT NULL DEFAULT 0,
|
||||
unit_price REAL NOT NULL DEFAULT 0,
|
||||
unit_code TEXT,
|
||||
concepto TEXT,
|
||||
amount REAL NOT NULL DEFAULT 0,
|
||||
gross REAL NOT NULL DEFAULT 0,
|
||||
discounts REAL NOT NULL DEFAULT 0,
|
||||
loan_id INTEGER REFERENCES loans(id),
|
||||
loan_discount REAL NOT NULL DEFAULT 0,
|
||||
loan_label TEXT,
|
||||
required_net REAL NOT NULL DEFAULT 0,
|
||||
payable_net REAL NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS destajo_units (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
tenant_id INTEGER NOT NULL DEFAULT 0,
|
||||
code TEXT NOT NULL,
|
||||
label TEXT NOT NULL,
|
||||
UNIQUE (tenant_id, code)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS destajo_periods (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
period_start TEXT NOT NULL,
|
||||
period_end TEXT NOT NULL,
|
||||
week_id INTEGER NOT NULL REFERENCES payroll_weeks(id) ON DELETE CASCADE,
|
||||
tenant_id INTEGER NOT NULL DEFAULT 0,
|
||||
UNIQUE (tenant_id, period_end)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS destajo_jobs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id),
|
||||
worker_id INTEGER NOT NULL REFERENCES workers(id),
|
||||
concepto TEXT NOT NULL,
|
||||
unit_code TEXT NOT NULL,
|
||||
qty_total_estimated REAL NOT NULL,
|
||||
unit_price REAL NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'open' CHECK (status IN ('open', 'done')),
|
||||
tenant_id INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS destajo_cut_lines (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
period_id INTEGER NOT NULL REFERENCES destajo_periods(id) ON DELETE CASCADE,
|
||||
job_id INTEGER NOT NULL REFERENCES destajo_jobs(id) ON DELETE CASCADE,
|
||||
qty_planned REAL NOT NULL DEFAULT 0,
|
||||
qty_actual REAL NOT NULL DEFAULT 0,
|
||||
qty_extra REAL NOT NULL DEFAULT 0,
|
||||
UNIQUE (period_id, job_id)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS loan_payments (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
loan_id INTEGER NOT NULL REFERENCES loans(id) ON DELETE CASCADE,
|
||||
week_id INTEGER NOT NULL REFERENCES payroll_weeks(id),
|
||||
amount REAL NOT NULL,
|
||||
installment_n INTEGER NOT NULL DEFAULT 1,
|
||||
label TEXT,
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS budget_chapters (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
parent_id INTEGER REFERENCES budget_chapters(id) ON DELETE SET NULL,
|
||||
code TEXT NOT NULL DEFAULT '',
|
||||
name TEXT NOT NULL,
|
||||
wbs TEXT NOT NULL DEFAULT '',
|
||||
sort_order INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS budget_items (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
|
||||
chapter_id INTEGER REFERENCES budget_chapters(id) ON DELETE SET NULL,
|
||||
code TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL,
|
||||
unit TEXT NOT NULL DEFAULT '',
|
||||
quantity REAL NOT NULL DEFAULT 0,
|
||||
unit_price REAL NOT NULL DEFAULT 0,
|
||||
amount REAL NOT NULL DEFAULT 0,
|
||||
wbs TEXT NOT NULL DEFAULT '',
|
||||
sort_order INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
447
api/scripts/migrate-sqlite-to-postgres.ts
Normal file
447
api/scripts/migrate-sqlite-to-postgres.ts
Normal file
|
|
@ -0,0 +1,447 @@
|
|||
#!/usr/bin/env -S deno run --allow-ffi --allow-net --allow-read --allow-write --allow-env
|
||||
/**
|
||||
* PANELS · Fase 5 · ETL único SQLite -> Postgres.
|
||||
*
|
||||
* Migra los datos de las dos SQLite históricas (data/app.db, data/platform.db)
|
||||
* a las 2 bases Postgres del monolito modular (panels_platform,
|
||||
* panels_product con esquemas iam/core). Corre en 3 fases:
|
||||
*
|
||||
* 1. Pre-flight: detecta de antemano lo que puede reventar el load o
|
||||
* corromper datos en silencio (duplicados CURP/RFC case-insensitive,
|
||||
* FKs huérfanas, tenant_id sin tenant, fechas con formato inválido).
|
||||
* 2. Carga: una transacción POR BASE/ESQUENA (platform, iam, core),
|
||||
* preservando los ids originales (OVERRIDING SYSTEM VALUE) para no
|
||||
* romper referencias cruzadas entre tablas.
|
||||
* 3. Verificación: compara conteos de filas origen/destino y sumas de
|
||||
* columnas de dinero con una tolerancia explícita (REAL -> NUMERIC
|
||||
* puede mover centavos).
|
||||
*
|
||||
* Uso (correr DESDE api/, con las credenciales _owner del ambiente destino
|
||||
* en el entorno -- DATABASE_URL_PLATFORM_OWNER, DATABASE_URL_IAM_OWNER,
|
||||
* DATABASE_URL_CORE_OWNER; el rol _app normal no alcanza a propósito):
|
||||
* cd api
|
||||
* set -a && source ../.env.dev-local && set +a # o el .env real del ambiente
|
||||
* deno run --allow-ffi --allow-net --allow-read --allow-write --allow-env \
|
||||
* scripts/migrate-sqlite-to-postgres.ts \
|
||||
* --app-db=../data/app.db --platform-db=../data/platform.db
|
||||
*
|
||||
* Sale con código != 0 si el pre-flight encuentra problemas o si la
|
||||
* verificación posterior no cuadra -- diseñado para un pipeline de
|
||||
* corte con criterios go/no-go, no para "correr y ya".
|
||||
*/
|
||||
import { Database as SqliteDatabase } from "jsr:@db/sqlite@0.12";
|
||||
import { createPool } from "../pg.ts";
|
||||
|
||||
function arg(name: string, fallback: string): string {
|
||||
const prefix = `--${name}=`;
|
||||
const found = Deno.args.find((a) => a.startsWith(prefix));
|
||||
return found ? found.slice(prefix.length) : fallback;
|
||||
}
|
||||
|
||||
const APP_DB_PATH = arg("app-db", "../data/app.db");
|
||||
const PLATFORM_DB_PATH = arg("platform-db", "../data/platform.db");
|
||||
const DRY_RUN = Deno.args.includes("--dry-run");
|
||||
|
||||
const MONEY_TOLERANCE = 0.05; // pesos; ver nota de REAL -> NUMERIC en el plan
|
||||
|
||||
let exitCode = 0;
|
||||
function fail(msg: string) {
|
||||
console.error(`[FAIL] ${msg}`);
|
||||
exitCode = 1;
|
||||
}
|
||||
function warn(msg: string) {
|
||||
console.warn(`[WARN] ${msg}`);
|
||||
}
|
||||
function ok(msg: string) {
|
||||
console.log(`[OK] ${msg}`);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Conexiones
|
||||
// ---------------------------------------------------------------------------
|
||||
const appDb = new SqliteDatabase(APP_DB_PATH, { readonly: true });
|
||||
const platformDb = new SqliteDatabase(PLATFORM_DB_PATH, { readonly: true });
|
||||
|
||||
// El ETL necesita privilegios de owner (INSERT con OVERRIDING SYSTEM VALUE +
|
||||
// setval() de secuencias) -- el rol _app normal no alcanza a propósito
|
||||
// (least privilege). Cada rol _owner solo manda en SU esquema
|
||||
// (panels_iam_owner en iam, panels_core_owner en core, ver
|
||||
// db/provision/04-product-database.sql), así que hacen falta conexiones
|
||||
// separadas incluso dentro de panels_product -- no hay un "super owner"
|
||||
// que pueda escribir en ambos esquemas de una vez.
|
||||
function ownerUrl(name: string): string {
|
||||
const url = Deno.env.get(name) || "";
|
||||
if (!url) throw new Error(`Falta ${name} -- el ETL requiere credenciales _owner, no _app`);
|
||||
return url;
|
||||
}
|
||||
const platformPg = createPool(ownerUrl("DATABASE_URL_PLATFORM_OWNER"), { max: 3 });
|
||||
const iamPg = createPool(ownerUrl("DATABASE_URL_IAM_OWNER"), { max: 3 });
|
||||
const corePg = createPool(ownerUrl("DATABASE_URL_CORE_OWNER"), { max: 3 });
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fase 1: Pre-flight
|
||||
// ---------------------------------------------------------------------------
|
||||
async function preflight(): Promise<void> {
|
||||
console.log("\n=== Pre-flight ===");
|
||||
|
||||
// 1. Duplicados case-insensitive de CURP/RFC (SQLite los toleraba con
|
||||
// COLLATE NOCASE + índice único case-insensitive; CITEXT en Postgres
|
||||
// los rechazará igual, pero mejor detectarlo ANTES de la carga).
|
||||
const dupCurp = appDb.prepare(
|
||||
`SELECT LOWER(curp) AS c, COUNT(*) AS n FROM workers GROUP BY LOWER(curp) HAVING COUNT(*) > 1`,
|
||||
).all() as { c: string; n: number }[];
|
||||
if (dupCurp.length) fail(`${dupCurp.length} CURP duplicadas (case-insensitive): ${dupCurp.map((d) => d.c).join(", ")}`);
|
||||
else ok("Sin CURP duplicadas");
|
||||
|
||||
const dupRfc = appDb.prepare(
|
||||
`SELECT LOWER(rfc) AS c, COUNT(*) AS n FROM workers GROUP BY LOWER(rfc) HAVING COUNT(*) > 1`,
|
||||
).all() as { c: string; n: number }[];
|
||||
if (dupRfc.length) fail(`${dupRfc.length} RFC duplicados (case-insensitive): ${dupRfc.map((d) => d.c).join(", ")}`);
|
||||
else ok("Sin RFC duplicados");
|
||||
|
||||
// 2. FKs huérfanas (SQLite solo valida si PRAGMA foreign_keys estuvo ON
|
||||
// en cada escritura histórica -- puede haber huecos).
|
||||
const orphanChecks: { label: string; sql: string }[] = [
|
||||
{ label: "workers.company_id sin companies", sql: `SELECT COUNT(*) AS n FROM workers WHERE company_id IS NOT NULL AND company_id NOT IN (SELECT id FROM companies)` },
|
||||
{ label: "workers.risk_code sin risk_levels", sql: `SELECT COUNT(*) AS n FROM workers WHERE risk_code NOT IN (SELECT code FROM risk_levels)` },
|
||||
{ label: "documents.worker_id sin workers", sql: `SELECT COUNT(*) AS n FROM documents WHERE worker_id NOT IN (SELECT id FROM workers)` },
|
||||
{ label: "documents.uploaded_by sin users", sql: `SELECT COUNT(*) AS n FROM documents WHERE uploaded_by IS NOT NULL AND uploaded_by NOT IN (SELECT id FROM users)` },
|
||||
{ label: "assignments.worker_id sin workers", sql: `SELECT COUNT(*) AS n FROM assignments WHERE worker_id NOT IN (SELECT id FROM workers)` },
|
||||
{ label: "assignments.project_id sin projects", sql: `SELECT COUNT(*) AS n FROM assignments WHERE project_id NOT IN (SELECT id FROM projects)` },
|
||||
{ label: "projects.theme_id sin badge_themes", sql: `SELECT COUNT(*) AS n FROM projects WHERE theme_id NOT IN (SELECT id FROM badge_themes)` },
|
||||
{ label: "users.company_id sin companies", sql: `SELECT COUNT(*) AS n FROM users WHERE company_id IS NOT NULL AND company_id NOT IN (SELECT id FROM companies)` },
|
||||
];
|
||||
for (const check of orphanChecks) {
|
||||
const row = appDb.prepare(check.sql).get() as { n: number };
|
||||
if (row.n > 0) fail(`${row.n} filas: ${check.label}`);
|
||||
else ok(check.label.replace("sin", "OK ->"));
|
||||
}
|
||||
|
||||
// 3. tenant_id nulo o sin tenant real.
|
||||
const tenantIds = new Set(
|
||||
(platformDb.prepare("SELECT id FROM tenants").all() as { id: number }[]).map((t) => t.id),
|
||||
);
|
||||
for (const table of ["companies", "workers", "projects", "users"]) {
|
||||
const rows = appDb.prepare(`SELECT id, tenant_id FROM ${table}`).all() as { id: number; tenant_id: number | null }[];
|
||||
const bad = rows.filter((r) => r.tenant_id == null || !tenantIds.has(r.tenant_id));
|
||||
if (bad.length) fail(`${table}: ${bad.length} filas con tenant_id nulo o inexistente (ids: ${bad.slice(0, 10).map((r) => r.id).join(",")}${bad.length > 10 ? "..." : ""})`);
|
||||
else ok(`${table}.tenant_id: todas resuelven a un tenant real`);
|
||||
}
|
||||
|
||||
// 4. Fechas con formato inválido en columnas que pasan a DATE.
|
||||
const dateCols: { table: string; col: string }[] = [
|
||||
{ table: "workers", col: "imss_alta_at" },
|
||||
{ table: "workers", col: "imss_baja_at" },
|
||||
{ table: "workers", col: "last_rehire_at" },
|
||||
{ table: "projects", col: "start_date" },
|
||||
{ table: "projects", col: "end_date" },
|
||||
{ table: "documents", col: "issued_at" },
|
||||
{ table: "documents", col: "expires_at" },
|
||||
];
|
||||
for (const { table, col } of dateCols) {
|
||||
const rows = appDb.prepare(
|
||||
`SELECT id, ${col} AS v FROM ${table} WHERE ${col} IS NOT NULL AND ${col} != ''`,
|
||||
).all() as { id: number; v: string }[];
|
||||
const bad = rows.filter((r) => !/^\d{4}-\d{2}-\d{2}/.test(r.v));
|
||||
if (bad.length) fail(`${table}.${col}: ${bad.length} fechas con formato inválido (ej. id=${bad[0].id} -> "${bad[0].v}")`);
|
||||
}
|
||||
if (!dateCols.some(() => false)) ok("Formato de fechas revisado");
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fase 2: Carga
|
||||
// ---------------------------------------------------------------------------
|
||||
type Row = Record<string, unknown>;
|
||||
|
||||
function toBool(v: unknown): boolean {
|
||||
return v === 1 || v === true || v === "1";
|
||||
}
|
||||
|
||||
/** Copia una tabla completa preservando ids, con transform opcional por fila.
|
||||
* `identityColumn`: columna GENERATED ALWAYS AS IDENTITY a preservar via
|
||||
* OVERRIDING SYSTEM VALUE (null para catálogos con PK de texto/compuesta,
|
||||
* que no tienen identity y no la necesitan). `conflictColumns`: columna(s)
|
||||
* de conflicto para el ON CONFLICT ... DO NOTHING (idempotencia si se
|
||||
* corre el ETL dos veces). */
|
||||
async function copyTable(
|
||||
sqlite: SqliteDatabase,
|
||||
pg: ReturnType<typeof createPool>,
|
||||
schema: string,
|
||||
table: string,
|
||||
opts: {
|
||||
sqliteTable?: string;
|
||||
transform?: (row: Row) => Row | null;
|
||||
columns?: string[];
|
||||
identityColumn?: string | null;
|
||||
conflictColumns?: string[];
|
||||
} = {},
|
||||
): Promise<number> {
|
||||
const sqliteTable = opts.sqliteTable ?? table;
|
||||
const identityColumn = opts.identityColumn === undefined ? "id" : opts.identityColumn;
|
||||
const conflictColumns = opts.conflictColumns ?? (identityColumn ? [identityColumn] : []);
|
||||
const rows = sqlite.prepare(`SELECT * FROM ${sqliteTable}`).all() as Row[];
|
||||
let inserted = 0;
|
||||
for (const raw of rows) {
|
||||
const row = opts.transform ? opts.transform(raw) : raw;
|
||||
if (!row) continue; // transform puede filtrar filas (ej. no migrables)
|
||||
const cols = opts.columns ?? Object.keys(row);
|
||||
const values = cols.map((c) => row[c]);
|
||||
const placeholders = cols.map((_, i) => `$${i + 1}`).join(", ");
|
||||
const overriding = identityColumn ? "OVERRIDING SYSTEM VALUE" : "";
|
||||
const onConflict = conflictColumns.length
|
||||
? `ON CONFLICT (${conflictColumns.join(", ")}) DO NOTHING`
|
||||
: "";
|
||||
const text = `INSERT INTO ${schema}.${table} (${cols.join(", ")}) ${overriding}
|
||||
VALUES (${placeholders}) ${onConflict}`;
|
||||
if (!DRY_RUN) await pg.unsafe(text, values as never[]);
|
||||
inserted++;
|
||||
}
|
||||
console.log(` ${schema}.${table}: ${inserted}/${rows.length} filas`);
|
||||
return inserted;
|
||||
}
|
||||
|
||||
async function setSequence(pg: ReturnType<typeof createPool>, schema: string, table: string): Promise<void> {
|
||||
if (DRY_RUN) return;
|
||||
await pg.unsafe(
|
||||
`SELECT setval(pg_get_serial_sequence('${schema}.${table}', 'id'), COALESCE((SELECT MAX(id) FROM ${schema}.${table}), 1), (SELECT MAX(id) IS NOT NULL FROM ${schema}.${table}))`,
|
||||
);
|
||||
}
|
||||
|
||||
async function migratePlatform(): Promise<void> {
|
||||
console.log("\n=== Carga: panels_platform ===");
|
||||
await platformPg.begin(async (tx) => {
|
||||
await copyTable(platformDb, tx as never, "public", "tenants", {
|
||||
transform: (r) => ({ ...r, status: r.status === "inactivo" ? "suspendido" : r.status }),
|
||||
});
|
||||
await copyTable(platformDb, tx as never, "public", "platform_users");
|
||||
await copyTable(platformDb, tx as never, "public", "smtp_settings", {
|
||||
identityColumn: null, // id fijo = 1 (singleton), no es GENERATED
|
||||
conflictColumns: ["id"], // el baseline de Liquibase ya insertó la fila id=1 por defecto
|
||||
transform: (r) => ({ ...r, enabled: toBool(r.enabled) }),
|
||||
});
|
||||
});
|
||||
await setSequence(platformPg, "public", "tenants");
|
||||
await setSequence(platformPg, "public", "platform_users");
|
||||
}
|
||||
|
||||
async function migrateIam(): Promise<void> {
|
||||
console.log("\n=== Carga: panels_product.iam ===");
|
||||
await iamPg.begin(async (tx) => {
|
||||
await copyTable(appDb, tx as never, "iam", "users", {
|
||||
columns: [
|
||||
"id", "username", "password_hash", "display_name", "company_id", "tenant_id",
|
||||
"role_code", "must_change_password", "email", "created_at",
|
||||
],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
role_code: r.role === "tenant_admin" ? "tenant_admin" : "user",
|
||||
must_change_password: toBool(r.must_change_password),
|
||||
}),
|
||||
});
|
||||
});
|
||||
await setSequence(iamPg, "iam", "users");
|
||||
}
|
||||
|
||||
/** users.id -> display_name, para el snapshot de uploaded_by/created_by
|
||||
* (Fase 4: documents/badge_jobs ya no tienen FK viva hacia iam). */
|
||||
function userNameLookup(): Map<number, string> {
|
||||
const rows = appDb.prepare("SELECT id, display_name FROM users").all() as { id: number; display_name: string }[];
|
||||
return new Map(rows.map((r) => [r.id, r.display_name]));
|
||||
}
|
||||
|
||||
async function migrateCore(): Promise<void> {
|
||||
console.log("\n=== Carga: panels_product.core ===");
|
||||
const names = userNameLookup();
|
||||
|
||||
await corePg.begin(async (tx) => {
|
||||
const t = tx as never as ReturnType<typeof createPool>;
|
||||
// Orden por dependencias de FK. risk_levels/badge_themes son catálogos
|
||||
// con PK de texto (code/id), no tienen columna identity que preservar.
|
||||
await copyTable(appDb, t, "core", "risk_levels", { identityColumn: null, conflictColumns: ["code"] });
|
||||
await copyTable(appDb, t, "core", "badge_themes", { identityColumn: null, conflictColumns: ["id"] });
|
||||
await copyTable(appDb, t, "core", "companies");
|
||||
await copyTable(appDb, t, "core", "projects");
|
||||
await copyTable(appDb, t, "core", "workers", {
|
||||
transform: (r) => ({ ...r, needs_badge: toBool(r.needs_badge) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "assignments", {
|
||||
transform: (r) => ({ ...r, active: toBool(r.active) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "document_types", {
|
||||
identityColumn: null,
|
||||
conflictColumns: ["code"],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
required: toBool(r.required),
|
||||
requires_issued_at: toBool(r.requires_issued_at),
|
||||
requires_expires_at: toBool(r.requires_expires_at),
|
||||
}),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "documents", {
|
||||
columns: [
|
||||
"id", "worker_id", "type_code", "original_name", "mime", "size_bytes", "sha256", "iv",
|
||||
"storage_name", "is_current", "parse_status", "issued_at", "expires_at", "imss_company_id",
|
||||
"imss_alta_at", "uploaded_by_id", "uploaded_by_name", "uploaded_at",
|
||||
],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
is_current: toBool(r.is_current),
|
||||
uploaded_by_id: r.uploaded_by ?? null,
|
||||
uploaded_by_name: r.uploaded_by ? names.get(Number(r.uploaded_by)) ?? "" : "",
|
||||
}),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "project_document_types", {
|
||||
identityColumn: null,
|
||||
conflictColumns: ["code"],
|
||||
transform: (r) => ({ ...r, required: toBool(r.required) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "project_documents", {
|
||||
columns: [
|
||||
"id", "project_id", "type_code", "original_name", "mime", "size_bytes", "sha256", "iv",
|
||||
"storage_name", "is_current", "parse_status", "uploaded_by_id", "uploaded_by_name", "uploaded_at",
|
||||
],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
is_current: toBool(r.is_current),
|
||||
uploaded_by_id: r.uploaded_by ?? null,
|
||||
uploaded_by_name: r.uploaded_by ? names.get(Number(r.uploaded_by)) ?? "" : "",
|
||||
}),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "company_document_types", {
|
||||
identityColumn: null,
|
||||
conflictColumns: ["code"],
|
||||
transform: (r) => ({ ...r, required: toBool(r.required) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "company_documents", {
|
||||
columns: [
|
||||
"id", "company_id", "type_code", "original_name", "mime", "size_bytes", "sha256", "iv",
|
||||
"storage_name", "is_current", "parse_status", "uploaded_by_id", "uploaded_by_name", "uploaded_at",
|
||||
],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
is_current: toBool(r.is_current),
|
||||
uploaded_by_id: r.uploaded_by ?? null,
|
||||
uploaded_by_name: r.uploaded_by ? names.get(Number(r.uploaded_by)) ?? "" : "",
|
||||
}),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "badge_jobs", {
|
||||
columns: ["id", "project_id", "status", "pdf_path", "created_by_id", "created_by_name", "created_at"],
|
||||
transform: (r) => ({
|
||||
...r,
|
||||
created_by_id: r.created_by ?? null,
|
||||
created_by_name: r.created_by ? names.get(Number(r.created_by)) ?? "" : "",
|
||||
}),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "badge_job_people", {
|
||||
identityColumn: null,
|
||||
conflictColumns: ["job_id", "worker_id"],
|
||||
transform: (r) => ({ ...r, delivered: toBool(r.delivered) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "loans");
|
||||
await copyTable(appDb, t, "core", "attendance", {
|
||||
transform: (r) => ({ ...r, present: toBool(r.present) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "payroll_periods");
|
||||
await copyTable(appDb, t, "core", "payroll_lines");
|
||||
await copyTable(appDb, t, "core", "payroll_settings", {
|
||||
identityColumn: null,
|
||||
conflictColumns: ["tenant_id"],
|
||||
transform: (r) => ({ ...r, loan_commission_enabled: toBool(r.loan_commission_enabled) }),
|
||||
});
|
||||
await copyTable(appDb, t, "core", "payroll_weeks");
|
||||
await copyTable(appDb, t, "core", "payroll_sheets");
|
||||
await copyTable(appDb, t, "core", "payroll_week_lines");
|
||||
await copyTable(appDb, t, "core", "destajo_units");
|
||||
await copyTable(appDb, t, "core", "destajo_periods");
|
||||
await copyTable(appDb, t, "core", "destajo_jobs");
|
||||
await copyTable(appDb, t, "core", "destajo_cut_lines");
|
||||
await copyTable(appDb, t, "core", "loan_payments");
|
||||
await copyTable(appDb, t, "core", "budget_chapters");
|
||||
await copyTable(appDb, t, "core", "budget_items");
|
||||
});
|
||||
|
||||
for (
|
||||
const table of [
|
||||
"companies", "projects", "workers", "assignments", "documents", "project_documents",
|
||||
"company_documents", "badge_jobs", "loans", "attendance", "payroll_periods", "payroll_lines",
|
||||
"payroll_weeks", "payroll_sheets", "payroll_week_lines", "destajo_units", "destajo_periods",
|
||||
"destajo_jobs", "destajo_cut_lines", "loan_payments", "budget_chapters", "budget_items",
|
||||
]
|
||||
) {
|
||||
await setSequence(corePg, "core", table);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fase 3: Verificación
|
||||
// ---------------------------------------------------------------------------
|
||||
async function verifyCounts(): Promise<void> {
|
||||
console.log("\n=== Verificación: conteos de filas ===");
|
||||
const checks: { sqlite: SqliteDatabase; table: string; pg: ReturnType<typeof createPool>; schema: string }[] = [
|
||||
{ sqlite: platformDb, table: "tenants", pg: platformPg, schema: "public" },
|
||||
{ sqlite: platformDb, table: "platform_users", pg: platformPg, schema: "public" },
|
||||
{ sqlite: appDb, table: "users", pg: iamPg, schema: "iam" },
|
||||
{ sqlite: appDb, table: "companies", pg: corePg, schema: "core" },
|
||||
{ sqlite: appDb, table: "workers", pg: corePg, schema: "core" },
|
||||
{ sqlite: appDb, table: "projects", pg: corePg, schema: "core" },
|
||||
{ sqlite: appDb, table: "documents", pg: corePg, schema: "core" },
|
||||
{ sqlite: appDb, table: "loans", pg: corePg, schema: "core" },
|
||||
{ sqlite: appDb, table: "budget_items", pg: corePg, schema: "core" },
|
||||
];
|
||||
for (const c of checks) {
|
||||
const src = (c.sqlite.prepare(`SELECT COUNT(*) AS n FROM ${c.table}`).get() as { n: number }).n;
|
||||
const dstRows = await c.pg.unsafe(`SELECT COUNT(*)::int AS n FROM ${c.schema}.${c.table}`);
|
||||
const dst = (dstRows[0] as unknown as { n: number }).n;
|
||||
if (src !== dst) fail(`${c.schema}.${c.table}: origen=${src} destino=${dst}`);
|
||||
else ok(`${c.schema}.${c.table}: ${dst} filas en ambos lados`);
|
||||
}
|
||||
}
|
||||
|
||||
async function verifyMoney(): Promise<void> {
|
||||
console.log("\n=== Verificación: sumas de dinero (tolerancia $" + MONEY_TOLERANCE + ") ===");
|
||||
const checks: { label: string; sqliteSql: string; pgSql: string }[] = [
|
||||
{ label: "workers.daily_wage", sqliteSql: "SELECT COALESCE(SUM(daily_wage),0) AS n FROM workers", pgSql: "SELECT COALESCE(SUM(daily_wage),0)::float AS n FROM core.workers" },
|
||||
{ label: "loans.balance", sqliteSql: "SELECT COALESCE(SUM(balance),0) AS n FROM loans", pgSql: "SELECT COALESCE(SUM(balance),0)::float AS n FROM core.loans" },
|
||||
{ label: "budget_items.amount", sqliteSql: "SELECT COALESCE(SUM(amount),0) AS n FROM budget_items", pgSql: "SELECT COALESCE(SUM(amount),0)::float AS n FROM core.budget_items" },
|
||||
];
|
||||
for (const c of checks) {
|
||||
const src = (appDb.prepare(c.sqliteSql).get() as { n: number }).n;
|
||||
const dstRows = await corePg.unsafe(c.pgSql);
|
||||
const dst = (dstRows[0] as unknown as { n: number }).n;
|
||||
const diff = Math.abs(src - dst);
|
||||
if (diff > MONEY_TOLERANCE) fail(`${c.label}: origen=${src} destino=${dst} (diff=${diff.toFixed(4)} > tolerancia)`);
|
||||
else ok(`${c.label}: origen=${src} destino=${dst} (diff=${diff.toFixed(4)})`);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// main
|
||||
// ---------------------------------------------------------------------------
|
||||
try {
|
||||
await preflight();
|
||||
if (exitCode !== 0) {
|
||||
console.error("\nPre-flight encontró problemas -- corrígelos antes de cargar. Abortando (no se tocó Postgres).");
|
||||
Deno.exit(1);
|
||||
}
|
||||
if (DRY_RUN) {
|
||||
console.log("\n--dry-run: se detiene aquí (pre-flight OK, no se escribió nada).");
|
||||
} else {
|
||||
await migratePlatform();
|
||||
await migrateIam();
|
||||
await migrateCore();
|
||||
await verifyCounts();
|
||||
await verifyMoney();
|
||||
if (exitCode !== 0) {
|
||||
console.error("\nLa verificación posterior a la carga NO cuadra -- revisar antes de dar por buena la migración.");
|
||||
} else {
|
||||
console.log("\nMigración completa y verificada.");
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
appDb.close();
|
||||
platformDb.close();
|
||||
await platformPg.end({ timeout: 5 });
|
||||
await iamPg.end({ timeout: 5 });
|
||||
await corePg.end({ timeout: 5 });
|
||||
}
|
||||
Deno.exit(exitCode);
|
||||
16
db/README.md
16
db/README.md
|
|
@ -32,15 +32,21 @@ Requiere Java 17+ y las credenciales del rol `_owner` de cada módulo (nunca
|
|||
```bash
|
||||
# Local, después de correr db/provision/dev-local.sh:
|
||||
set -a && source .env.dev-local && set +a
|
||||
./db/update.sh all --context-filter=dev # dev: incluye datos de demo
|
||||
./db/update.sh all # staging/producción: solo esquema + catálogos
|
||||
./db/update.sh all --context-filter=dev # dev: incluye datos de demo
|
||||
./db/update.sh all --context-filter='!dev' # staging/producción: solo esquema + catálogos
|
||||
|
||||
# Un solo módulo
|
||||
./db/update.sh core
|
||||
./db/update.sh iam
|
||||
./db/update.sh platform
|
||||
./db/update.sh core --context-filter='!dev'
|
||||
./db/update.sh iam --context-filter='!dev'
|
||||
./db/update.sh platform --context-filter='!dev'
|
||||
```
|
||||
|
||||
**Importante**: Liquibase, si NO se le pasa `--context-filter`, corre TODOS
|
||||
los changesets sin importar su `context` -- incluidos los de `context="dev"`.
|
||||
No pasar el filtro en staging/producción NO es "modo seguro por defecto",
|
||||
es lo contrario: cargaría el tenant/empresas/proyecto de demostración. Por
|
||||
eso `--context-filter` es obligatorio siempre, nunca opcional.
|
||||
|
||||
`npm run db:migrate` corre `./db/update.sh all`. **Las migraciones ya NO
|
||||
corren automáticamente al arrancar la API** -- son un paso explícito de
|
||||
deploy (a diferencia del `runLiquibase()` que existía con SQLite).
|
||||
|
|
|
|||
89
db/RUNBOOK-corte.md
Normal file
89
db/RUNBOOK-corte.md
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
# Runbook de corte SQLite → Postgres (Fase 5)
|
||||
|
||||
Checklist operativo para migrar UN ambiente (dev, luego staging, luego
|
||||
producción -- nunca saltar directo a producción). Usa
|
||||
[`api/scripts/migrate-sqlite-to-postgres.ts`](../api/scripts/migrate-sqlite-to-postgres.ts).
|
||||
|
||||
## 0. Antes de empezar
|
||||
|
||||
- [ ] Postgres del ambiente aprovisionado (`db/provision/`) y migrado
|
||||
**sin** datos de demo: `./db/update.sh all --context-filter='!dev'`.
|
||||
- [ ] Backup fresco del volumen SQLite actual (`data/app.db`, `data/platform.db`)
|
||||
guardado aparte, fuera del volumen que se va a apagar.
|
||||
- [ ] Credenciales `_owner` de los 3 módulos disponibles en el entorno
|
||||
(`DATABASE_URL_PLATFORM_OWNER`, `DATABASE_URL_IAM_OWNER`, `DATABASE_URL_CORE_OWNER`).
|
||||
|
||||
## 1. Pre-flight (sin ventana de mantenimiento, se puede correr en caliente)
|
||||
|
||||
```bash
|
||||
cd api
|
||||
deno run --allow-ffi --allow-net --allow-read --allow-write --allow-env \
|
||||
scripts/migrate-sqlite-to-postgres.ts --dry-run \
|
||||
--app-db=../data/app.db --platform-db=../data/platform.db
|
||||
```
|
||||
|
||||
Si el pre-flight reporta `[FAIL]`, **no continuar** -- corregir los datos
|
||||
en SQLite (duplicados, huérfanas, tenant_id inválido, fechas mal
|
||||
formateadas) y repetir hasta que todo salga `[OK]`.
|
||||
|
||||
## 2. Ventana de mantenimiento (corte real)
|
||||
|
||||
Hoy no existe un "modo mantenimiento" en la app. Opciones, de menor a
|
||||
mayor invasividad:
|
||||
|
||||
- Parar el contenedor/proceso `api` (nadie puede escribir mientras está
|
||||
abajo -- los fronts mostrarán error de conexión).
|
||||
- Responder 503 temporal en nginx para `/v1/*` mientras se corre el ETL.
|
||||
|
||||
Elegir una, documentar la hora exacta de inicio.
|
||||
|
||||
## 3. Migración
|
||||
|
||||
```bash
|
||||
cd api
|
||||
set -a && source /ruta/al/.env.del.ambiente && set +a
|
||||
deno run --allow-ffi --allow-net --allow-read --allow-write --allow-env \
|
||||
scripts/migrate-sqlite-to-postgres.ts \
|
||||
--app-db=/ruta/a/app.db --platform-db=/ruta/a/platform.db
|
||||
```
|
||||
|
||||
El script hace: pre-flight → carga (una transacción por base/esquema,
|
||||
todo o nada) → `setval()` de secuencias → verificación de conteos y sumas
|
||||
de dinero. Si CUALQUIER paso falla, no queda un estado a medias en
|
||||
Postgres (la transacción de esa base se revierte completa), pero SQLite
|
||||
sigue siendo la fuente de verdad -- no se ha cortado nada todavía.
|
||||
|
||||
## 4. Criterios go/no-go
|
||||
|
||||
Antes de apuntar la app a Postgres y apagar SQLite:
|
||||
|
||||
- [ ] El script terminó con `Migración completa y verificada.` (exit code 0).
|
||||
- [ ] Conteos de filas origen=destino en **todas** las tablas listadas
|
||||
(no solo las 9 de ejemplo del script -- ampliar `verifyCounts()` si
|
||||
el ambiente tiene datos en tablas no cubiertas ahí).
|
||||
- [ ] Diferencia de sumas de dinero dentro de la tolerancia ($0.05) -- si
|
||||
no cuadra, decidir explícitamente si se acepta el redondeo
|
||||
REAL→NUMERIC o se investiga antes de continuar.
|
||||
- [ ] Spot-check manual de 2-3 registros conocidos (un trabajador, un
|
||||
preupuesto, un préstamo) comparando app vieja vs. Postgres.
|
||||
|
||||
Si algo no cuadra: **no cortar**. Volver a levantar la app contra SQLite
|
||||
(no se tocó), investigar, y repetir desde el paso 1 en otro intento.
|
||||
|
||||
## 5. Cutover
|
||||
|
||||
- [ ] Actualizar `DATABASE_URL_*`/`REDIS_URL_*` del servicio `api` a los
|
||||
valores del ambiente Postgres/Redis recién migrado.
|
||||
- [ ] Levantar `api` -- el fail-fast de arranque (`/v1/health`) debe
|
||||
responder `{"ok":true,...}` antes de reabrir tráfico.
|
||||
- [ ] Reabrir tráfico (quitar el 503/levantar el contenedor).
|
||||
- [ ] Login de prueba con un usuario real del ambiente.
|
||||
|
||||
## 6. Después del corte
|
||||
|
||||
- [ ] Conservar el volumen SQLite (`data/`) como respaldo frío por un
|
||||
período de retención definido (ej. 30 días) antes de borrarlo.
|
||||
- [ ] Correr `./db/provision/verify-isolation.sh` contra el ambiente para
|
||||
confirmar que el aislamiento de roles/RLS/ACLs sigue intacto.
|
||||
- [ ] Repetir todo el runbook en el siguiente ambiente (dev → staging →
|
||||
producción), nunca en paralelo.
|
||||
108
db/backups/README.md
Normal file
108
db/backups/README.md
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
# Backups y continuidad (Fase 6)
|
||||
|
||||
## 0. Validar antes de diseñar el resto
|
||||
|
||||
Coolify despliega Postgres como un recurso "managed" que en realidad es un
|
||||
contenedor `postgres:16` estándar con volumen persistente. **Hay que
|
||||
confirmar, para el ambiente real, si se puede:**
|
||||
|
||||
1. Montar un `postgresql.conf` custom (para `archive_mode`, `archive_command`, `wal_level`), y
|
||||
2. Ejecutar un sidecar/proceso adicional (`pgbackrest`) con acceso al mismo volumen de datos y a WAL.
|
||||
|
||||
- **Si sí se puede**: seguir esta guía con pgBackRest (PITR real).
|
||||
- **Si no se puede** (Coolify no expone esos hooks en el plan/versión
|
||||
usada): el respaldo queda limitado a los `pg_dump` programados que
|
||||
Coolify ya ofrece -- el RPO pasa a ser "la frecuencia del dump", no
|
||||
segundos. En ese caso, para producción, evaluar auto-hospedar Postgres
|
||||
(contenedor propio, fuera del recurso "managed" de Coolify) solo para
|
||||
tener control de WAL -- es la única forma de bajar el RPO por debajo de
|
||||
la frecuencia del dump.
|
||||
|
||||
Esta decisión condiciona todo lo demás; no asumir que WAL archiving va a
|
||||
funcionar sin probarlo primero contra el Coolify real del ambiente.
|
||||
|
||||
## 1. Qué se respalda y con qué política
|
||||
|
||||
| Qué | Herramienta | Frecuencia | Retención | RPO objetivo |
|
||||
|---|---|---|---|---|
|
||||
| `panels_platform` (prod) | pgBackRest (o `pg_dump` si no hay WAL) | full diario + diferencial c/6h + WAL continuo | 30 días | segundos (con WAL) / 24h (solo dump) |
|
||||
| `panels_product` (prod) | igual que arriba | igual | 30 días | igual |
|
||||
| `panels_platform`/`panels_product` (staging/dev) | `pg_dump` | full diario | 7 días | 24h |
|
||||
| Archivos (Contabo, Fase 4c) | versionado de objetos del bucket + backup del bucket | continuo (versionado) | igual que la BD | ver nota de coordinación abajo |
|
||||
| Redis | RDB snapshot | al reiniciar/periódico | ninguna (no es fuente de verdad) | N/A -- ver `db/backups/redis-persistence.md` |
|
||||
|
||||
**Respaldar SIEMPRE ambas bases.** Es fácil configurar el backup de
|
||||
`panels_product` (donde está "todo el negocio") y olvidar `panels_platform`
|
||||
(que tiene el registro de TODOS los tenants) -- el día del incidente ahí es
|
||||
cuando se descubre.
|
||||
|
||||
## 2. pgBackRest (si Coolify lo permite)
|
||||
|
||||
Ver [`pgbackrest-panels_platform.conf`](./pgbackrest-panels_platform.conf) y
|
||||
[`pgbackrest-panels_product.conf`](./pgbackrest-panels_product.conf) --
|
||||
plantillas, una `stanza` por base de datos (no por esquema: `panels_product`
|
||||
es una sola stanza aunque tenga los esquemas iam/core adentro).
|
||||
|
||||
Pasos (por base):
|
||||
|
||||
```bash
|
||||
# 1. postgresql.conf del contenedor
|
||||
wal_level = replica
|
||||
archive_mode = on
|
||||
archive_command = 'pgbackrest --stanza=panels_platform archive-push %p'
|
||||
archive_timeout = 60 # cap de RPO a 60s incluso en bases con poca escritura
|
||||
|
||||
# 2. Crear la stanza una vez
|
||||
pgbackrest --stanza=panels_platform --log-level-console=info stanza-create
|
||||
|
||||
# 3. Verificar que el archiving realmente funciona ANTES de confiar en él
|
||||
pgbackrest --stanza=panels_platform check
|
||||
|
||||
# 4. Backups programados (cron / scheduler de Coolify)
|
||||
pgbackrest --stanza=panels_platform --type=full backup # 1x/semana
|
||||
pgbackrest --stanza=panels_platform --type=diff backup # cada 6h
|
||||
```
|
||||
|
||||
Repetir con `--stanza=panels_product` y su propio `archive_command`.
|
||||
|
||||
### Point-in-time recovery
|
||||
|
||||
```bash
|
||||
pgbackrest --stanza=panels_platform --type=time \
|
||||
--target="2026-09-01 14:00:00-06" --target-action=promote restore
|
||||
```
|
||||
|
||||
## 3. Si NO hay WAL archiving (solo pg_dump)
|
||||
|
||||
```bash
|
||||
pg_dump --format=custom --file=panels_platform-$(date +%Y%m%d).dump "$DATABASE_URL_PLATFORM_OWNER"
|
||||
pg_dump --format=custom --file=panels_product-$(date +%Y%m%d).dump "$DATABASE_URL_CORE_OWNER"
|
||||
```
|
||||
|
||||
`--format=custom` permite restore paralelo (`pg_restore -j4`) y restore
|
||||
selectivo por tabla si algún día hace falta. Subir los `.dump` a un bucket
|
||||
S3-compatible (puede ser el mismo de Contabo, con prefijo/bucket distinto
|
||||
al de documentos) con retención por lifecycle policy del bucket.
|
||||
|
||||
## 4. Coordinación con el respaldo de archivos (Fase 4c)
|
||||
|
||||
Un restore de Postgres a un punto en el tiempo, sin restaurar los archivos
|
||||
de Contabo al MISMO punto, deja filas (`documents.storage_name`) apuntando
|
||||
a objetos que ya no existen o que cambiaron. Política:
|
||||
|
||||
- Activar versionado de objetos en el bucket de Contabo.
|
||||
- Si se hace un PITR de Postgres a una hora `T`, documentar que los
|
||||
archivos subidos/borrados después de `T` pueden quedar huérfanos o
|
||||
inconsistentes -- es una ventana de inconsistencia aceptada, no un bug
|
||||
a corregir en caliente durante el incidente.
|
||||
|
||||
## 5. Redis
|
||||
|
||||
Ver [`redis-persistence.md`](./redis-persistence.md). Resumen: solo RDB
|
||||
para evitar deslogueo masivo, sin retención de negocio, fuera del alcance
|
||||
del simulacro de restauración de abajo.
|
||||
|
||||
## 6. Simulacro de restauración (mensual, obligatorio)
|
||||
|
||||
Ver [`restore-drill-checklist.md`](./restore-drill-checklist.md). Un
|
||||
backup que nunca se probó restaurar no es un backup, es una esperanza.
|
||||
24
db/backups/pgbackrest-panels_platform.conf
Normal file
24
db/backups/pgbackrest-panels_platform.conf
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# Plantilla pgBackRest para panels_platform (control plane SaaS).
|
||||
# Copiar a /etc/pgbackrest/pgbackrest.conf (o incluir via `include` si ya
|
||||
# existe una stanza para panels_product en el mismo host/instancia) y
|
||||
# ajustar repo1-* según el backend real (aquí: S3-compatible, ej. Contabo
|
||||
# Object Storage -- puede ser el MISMO endpoint que Fase 4c con un bucket
|
||||
# o prefijo distinto para no mezclar backups de BD con documentos).
|
||||
|
||||
[global]
|
||||
repo1-type=s3
|
||||
repo1-s3-endpoint=CAMBIAR-ENDPOINT-S3
|
||||
repo1-s3-bucket=panels-backups
|
||||
repo1-s3-key=CAMBIAR-ACCESS-KEY
|
||||
repo1-s3-key-secret=CAMBIAR-SECRET-KEY
|
||||
repo1-s3-region=CAMBIAR-REGION
|
||||
repo1-path=/panels_platform
|
||||
repo1-retention-full=4
|
||||
repo1-retention-diff=14
|
||||
process-max=2
|
||||
compress-type=zst
|
||||
log-level-console=info
|
||||
|
||||
[panels_platform]
|
||||
pg1-path=/var/lib/postgresql/data
|
||||
pg1-port=5432
|
||||
21
db/backups/pgbackrest-panels_product.conf
Normal file
21
db/backups/pgbackrest-panels_product.conf
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
# Plantilla pgBackRest para panels_product (iam + core). Una stanza por
|
||||
# BASE DE DATOS, no por esquema -- panels_product es una sola stanza aunque
|
||||
# tenga iam/core adentro (ver db/README.md sobre el modelo de 2 bases).
|
||||
|
||||
[global]
|
||||
repo1-type=s3
|
||||
repo1-s3-endpoint=CAMBIAR-ENDPOINT-S3
|
||||
repo1-s3-bucket=panels-backups
|
||||
repo1-s3-key=CAMBIAR-ACCESS-KEY
|
||||
repo1-s3-key-secret=CAMBIAR-SECRET-KEY
|
||||
repo1-s3-region=CAMBIAR-REGION
|
||||
repo1-path=/panels_product
|
||||
repo1-retention-full=4
|
||||
repo1-retention-diff=14
|
||||
process-max=2
|
||||
compress-type=zst
|
||||
log-level-console=info
|
||||
|
||||
[panels_product]
|
||||
pg1-path=/var/lib/postgresql/data
|
||||
pg1-port=5432
|
||||
35
db/backups/redis-persistence.md
Normal file
35
db/backups/redis-persistence.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# Persistencia de Redis (mínima, no es backup de negocio)
|
||||
|
||||
Redis guarda sesiones (`iam:session:*`) y cache (`core:cache:*`, ver
|
||||
Fase 4e). Ninguno de los dos es fuente de verdad:
|
||||
|
||||
- Sesión perdida → el usuario hace login de nuevo. Molesto, no es pérdida
|
||||
de datos.
|
||||
- Cache perdido → la siguiente lectura recalcula desde Postgres.
|
||||
|
||||
Por eso Redis **no** entra en la política de backup con retención/PITR de
|
||||
las bases Postgres (Fase 6 del plan). Lo único que vale la pena es evitar
|
||||
que un reinicio del contenedor Redis deslogueé a **todos** los tenants a
|
||||
la vez -- eso se resuelve con RDB, no con un pipeline de backup.
|
||||
|
||||
## Configuración recomendada (solo producción)
|
||||
|
||||
```conf
|
||||
# redis.conf del recurso de Coolify (o el equivalente que exponga)
|
||||
save 900 1 # snapshot si hubo >=1 cambio en 15 min
|
||||
save 300 10 # snapshot si hubo >=10 cambios en 5 min
|
||||
save 60 10000 # snapshot si hubo >=10000 cambios en 1 min
|
||||
appendonly no # AOF no hace falta -- RDB alcanza para el objetivo (evitar
|
||||
# deslogueo masivo), y agrega complejidad/IO sin beneficio
|
||||
# dado que nada aquí es irremplazable.
|
||||
```
|
||||
|
||||
En dev/staging, ni siquiera hace falta esto -- perder las sesiones de
|
||||
desarrollo no tiene costo real.
|
||||
|
||||
## Qué NO hacer
|
||||
|
||||
- No configurar `pgbackrest`/backups con retención larga para Redis --
|
||||
sería tratar como "fuente de verdad" algo que por diseño no lo es.
|
||||
- No mezclar la política de respaldo de Redis con la de Postgres en la
|
||||
misma automatización -- son necesidades distintas (ver Fase 6 del plan).
|
||||
37
db/backups/restore-drill-checklist.md
Normal file
37
db/backups/restore-drill-checklist.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Simulacro de restauración (mensual, obligatorio)
|
||||
|
||||
Un backup que nunca se probó restaurar no es un backup. Correr esto en un
|
||||
ambiente descartable (nunca contra staging/producción reales).
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Levantar un Postgres nuevo y vacío (contenedor descartable).
|
||||
- [ ] Restaurar el backup más reciente de `panels_platform`:
|
||||
- pgBackRest: `pgbackrest --stanza=panels_platform restore`
|
||||
- o `pg_restore` si es dump lógico.
|
||||
- [ ] Restaurar el backup más reciente de `panels_product` (mismo método).
|
||||
- [ ] Correr [`db/provision/verify-isolation.sh`](../provision/verify-isolation.sh)
|
||||
contra el ambiente restaurado -- confirma que roles/RLS/permisos
|
||||
sobrevivieron el restore intactos, no solo los datos.
|
||||
- [ ] Spot-check de datos: comparar conteos de filas y 2-3 registros
|
||||
conocidos contra lo que se espera (usar el mismo enfoque que
|
||||
`verifyCounts()`/`verifyMoney()` del ETL, ver
|
||||
[`api/scripts/migrate-sqlite-to-postgres.ts`](../../api/scripts/migrate-sqlite-to-postgres.ts)
|
||||
como referencia de qué comparar).
|
||||
- [ ] Si hay WAL archiving: probar un restore a un punto en el tiempo
|
||||
específico (no solo "el último backup"), confirmar que aterriza en
|
||||
el estado esperado para ese instante.
|
||||
- [ ] Archivos (Contabo): confirmar que al menos una descarga de
|
||||
documento cifrado conocido sigue descifrando correctamente después
|
||||
del restore (prueba de que la clave `DOCS_KEY` y el bucket siguen
|
||||
consistentes).
|
||||
- [ ] Documentar cuánto tardó el restore de punta a punta -- es el RTO
|
||||
real, no el teórico.
|
||||
- [ ] Destruir el ambiente descartable al terminar.
|
||||
|
||||
## Cuándo escalar
|
||||
|
||||
Si cualquier paso falla (restore no completa, RLS no aplica, datos no
|
||||
cuadran, documento no descifra), **no esperar al siguiente simulacro** --
|
||||
es una señal de que el backup en producción probablemente tampoco sirve.
|
||||
Tratarlo como incidente, no como hallazgo de rutina.
|
||||
31
db/provision/docker-provision.sh
Normal file
31
db/provision/docker-provision.sh
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
#!/usr/bin/env sh
|
||||
# PANELS · Fase 0 · aprovisiona roles/bases/esquemas/ACLs contra un
|
||||
# Postgres/Redis alcanzables por red (uso: contenedor de docker-compose
|
||||
# para desarrollo local -- ver el servicio "provision" en
|
||||
# docker-compose.yml). Requiere psql y redis-cli en la imagen (usa la
|
||||
# imagen postgres:16-alpine + redis-cli instalado, o dos pasos separados).
|
||||
set -eu
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
: "${PGHOST:?}"; : "${PGPORT:=5432}"; : "${PGUSER:=postgres}"; : "${PGPASSWORD:?}"
|
||||
export PGPASSWORD
|
||||
|
||||
SUPERUSER_URL="postgresql://${PGUSER}@${PGHOST}:${PGPORT}/postgres"
|
||||
|
||||
echo "== 1/4: roles =="
|
||||
psql "$SUPERUSER_URL" -v ON_ERROR_STOP=1 \
|
||||
-v platform_owner_pw="${PLATFORM_OWNER_PASSWORD}" -v platform_app_pw="${PLATFORM_APP_PASSWORD}" \
|
||||
-v iam_owner_pw="${IAM_OWNER_PASSWORD}" -v iam_app_pw="${IAM_APP_PASSWORD}" \
|
||||
-v core_owner_pw="${CORE_OWNER_PASSWORD}" -v core_app_pw="${CORE_APP_PASSWORD}" \
|
||||
-f 01-roles.sql
|
||||
|
||||
echo "== 2/4: bases de datos =="
|
||||
psql "$SUPERUSER_URL" -v ON_ERROR_STOP=1 -f 02-databases.sql
|
||||
|
||||
echo "== 3/4: privilegios panels_platform =="
|
||||
psql "postgresql://${PGUSER}@${PGHOST}:${PGPORT}/panels_platform" -v ON_ERROR_STOP=1 -f 03-platform-database.sql
|
||||
|
||||
echo "== 4/4: esquemas + privilegios panels_product =="
|
||||
psql "postgresql://${PGUSER}@${PGHOST}:${PGPORT}/panels_product" -v ON_ERROR_STOP=1 -f 04-product-database.sql
|
||||
|
||||
echo "Roles/bases/esquemas listos."
|
||||
19
db/update.sh
19
db/update.sh
|
|
@ -13,6 +13,7 @@
|
|||
# Uso:
|
||||
# set -a && source .env.dev-local && set +a && ./db/update.sh all
|
||||
# ./db/update.sh core --context-filter=dev
|
||||
# ./db/update.sh all --context-filter='!dev' -- updateSQL # dry-run (CI)
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
"$ROOT/db/bootstrap-tools.sh"
|
||||
|
|
@ -21,7 +22,19 @@ PG_JDBC="$ROOT/db/tools/postgresql-jdbc.jar"
|
|||
|
||||
TARGET="${1:-all}"
|
||||
shift || true
|
||||
EXTRA_ARGS=("$@")
|
||||
|
||||
LB_COMMAND="update"
|
||||
EXTRA_ARGS=()
|
||||
while [[ $# -gt 0 ]]; do
|
||||
if [[ "$1" == "--" ]]; then
|
||||
shift
|
||||
LB_COMMAND="${1:-update}"
|
||||
shift || true
|
||||
else
|
||||
EXTRA_ARGS+=("$1")
|
||||
shift
|
||||
fi
|
||||
done
|
||||
|
||||
# postgresql://user:pass@host:port/db -> jdbc:postgresql://host:port/db + user/pass
|
||||
parse_url() {
|
||||
|
|
@ -43,7 +56,7 @@ run_one() {
|
|||
exit 1
|
||||
fi
|
||||
parse_url "$url"
|
||||
echo "==> Liquibase update: $name ($PARSED_JDBC)"
|
||||
echo "==> Liquibase $LB_COMMAND: $name ($PARSED_JDBC)"
|
||||
(
|
||||
cd "$ROOT/db/$name"
|
||||
"$LIQUIBASE" \
|
||||
|
|
@ -52,7 +65,7 @@ run_one() {
|
|||
--url="$PARSED_JDBC" \
|
||||
--username="$PARSED_USER" \
|
||||
--password="$PARSED_PASS" \
|
||||
update "${EXTRA_ARGS[@]}"
|
||||
"$LB_COMMAND" "${EXTRA_ARGS[@]}"
|
||||
)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,10 +1,85 @@
|
|||
# Coolify / local — 3 services, SQLite en volumen panel-data
|
||||
# Coolify / local — Postgres + Redis (ver plan de migración: monolito
|
||||
# modular panels_platform / panels_product con esquemas iam+core).
|
||||
# En Coolify, postgres/redis normalmente son recursos gestionados aparte
|
||||
# (ver docs/coolify.md); este compose incluye contenedores propios para
|
||||
# que `docker compose up` funcione standalone en desarrollo local.
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_PASSWORD: ${POSTGRES_SUPERUSER_PASSWORD:?define POSTGRES_SUPERUSER_PASSWORD}
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
expose:
|
||||
- "5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- "6379"
|
||||
healthcheck:
|
||||
test: ["CMD", "redis-cli", "ping"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
|
||||
# Jobs de un solo uso (nunca en producción -- ver db/provision/README.md):
|
||||
# 1) crea roles/bases/esquemas/ACLs, 2) aplica Liquibase. `api` espera a
|
||||
# que ambos terminen bien antes de arrancar.
|
||||
provision:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.provision
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
redis:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
PGHOST: postgres
|
||||
PGPASSWORD: ${POSTGRES_SUPERUSER_PASSWORD:?}
|
||||
PLATFORM_OWNER_PASSWORD: ${PLATFORM_OWNER_PASSWORD:?}
|
||||
PLATFORM_APP_PASSWORD: ${PLATFORM_APP_PASSWORD:?}
|
||||
IAM_OWNER_PASSWORD: ${IAM_OWNER_PASSWORD:?}
|
||||
IAM_APP_PASSWORD: ${IAM_APP_PASSWORD:?}
|
||||
CORE_OWNER_PASSWORD: ${CORE_OWNER_PASSWORD:?}
|
||||
CORE_APP_PASSWORD: ${CORE_APP_PASSWORD:?}
|
||||
REDIS_ADMIN_URL: redis://redis:6379
|
||||
IAM_REDIS_PASSWORD: ${IAM_REDIS_PASSWORD:?}
|
||||
CORE_REDIS_PASSWORD: ${CORE_REDIS_PASSWORD:?}
|
||||
command:
|
||||
- -c
|
||||
- "./db/provision/docker-provision.sh && ./db/provision/05-redis-acl.sh"
|
||||
|
||||
migrate:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.migrate
|
||||
depends_on:
|
||||
provision:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
DATABASE_URL_PLATFORM_OWNER: postgresql://panels_platform_owner:${PLATFORM_OWNER_PASSWORD:?}@postgres:5432/panels_platform
|
||||
DATABASE_URL_IAM_OWNER: postgresql://panels_iam_owner:${IAM_OWNER_PASSWORD:?}@postgres:5432/panels_product
|
||||
DATABASE_URL_CORE_OWNER: postgresql://panels_core_owner:${CORE_OWNER_PASSWORD:?}@postgres:5432/panels_product
|
||||
|
||||
api:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.api
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
redis:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
PORT: ${PORT:-8000}
|
||||
SESSION_SECRET: ${SESSION_SECRET}
|
||||
|
|
@ -20,6 +95,23 @@ services:
|
|||
SMTP_USER: ${SMTP_USER:-}
|
||||
SMTP_PASS: ${SMTP_PASS:-}
|
||||
SMTP_FROM: ${SMTP_FROM:-}
|
||||
# Postgres: panels_platform (base separada) + panels_product (iam/core)
|
||||
DATABASE_URL_PLATFORM: postgresql://panels_platform_app:${PLATFORM_APP_PASSWORD:?}@postgres:5432/panels_platform
|
||||
DATABASE_URL_IAM: postgresql://panels_iam_app:${IAM_APP_PASSWORD:?}@postgres:5432/panels_product
|
||||
DATABASE_URL_CORE: postgresql://panels_core_app:${CORE_APP_PASSWORD:?}@postgres:5432/panels_product
|
||||
DATABASE_URL_IAM_OWNER: postgresql://panels_iam_owner:${IAM_OWNER_PASSWORD:?}@postgres:5432/panels_product
|
||||
DATABASE_URL_CORE_OWNER: postgresql://panels_core_owner:${CORE_OWNER_PASSWORD:?}@postgres:5432/panels_product
|
||||
# Redis: ACLs por módulo (ver db/provision/05-redis-acl.sh)
|
||||
REDIS_URL_IAM: redis://panels_iam_redis:${IAM_REDIS_PASSWORD:?}@redis:6379
|
||||
REDIS_URL_CORE: redis://panels_core_redis:${CORE_REDIS_PASSWORD:?}@redis:6379
|
||||
# Contabo Object Storage (Fase 4c) -- opcional; sin esto cae a disco
|
||||
# local bajo el volumen panel-data (no recomendado en producción,
|
||||
# ver plan: un volumen local no escala horizontalmente).
|
||||
S3_ENDPOINT: ${S3_ENDPOINT:-}
|
||||
S3_BUCKET: ${S3_BUCKET:-}
|
||||
S3_REGION: ${S3_REGION:-}
|
||||
S3_ACCESS_KEY_ID: ${S3_ACCESS_KEY_ID:-}
|
||||
S3_SECRET_ACCESS_KEY: ${S3_SECRET_ACCESS_KEY:-}
|
||||
volumes:
|
||||
- panel-data:/app/data
|
||||
expose:
|
||||
|
|
@ -35,7 +127,7 @@ services:
|
|||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 45s
|
||||
start_period: 15s
|
||||
|
||||
web-panel:
|
||||
build:
|
||||
|
|
@ -64,4 +156,5 @@ services:
|
|||
- "80"
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
panel-data:
|
||||
|
|
|
|||
|
|
@ -1,24 +1,29 @@
|
|||
# Deploy PANELS en Coolify (Docker Compose)
|
||||
# Deploy PANELS en Coolify (Docker Compose + Postgres + Redis)
|
||||
|
||||
## Resumen
|
||||
|
||||
| Qué | Cuánto |
|
||||
|-----|--------|
|
||||
| Bases gestionadas Coolify (Postgres/MySQL) | **0** |
|
||||
| Archivos SQLite en volumen | **2** (`app.db`, `platform.db`) |
|
||||
| Contenedores | **3** (`api`, `web-panel`, `web-saas`) |
|
||||
| Volumen persistente | **1** → `/app/data` en `api` |
|
||||
| Postgres gestionado en Coolify | **1 instancia** por ambiente, con **2 bases**: `panels_platform`, `panels_product` (esquemas `iam`/`core`) |
|
||||
| Redis gestionado en Coolify | **1 instancia** por ambiente, con **2 usuarios ACL** (`panels_iam_redis`, `panels_core_redis`) |
|
||||
| Object storage | Contabo Object Storage (S3-compatible) para expedientes/PDFs/logos cifrados |
|
||||
| Contenedores de la app | **3** (`api`, `web-panel`, `web-saas`) |
|
||||
| Volumen persistente | **1** → `/app/data` en `api` (solo fallback local si no hay Contabo configurado -- no usar así en producción) |
|
||||
|
||||
Los fronts (Alpine + nginx) hacen proxy de `/v1` al servicio `api`, así las cookies de sesión van same-origin.
|
||||
|
||||
Ver el plan de migración para el detalle de arquitectura: por qué
|
||||
`panels_platform` es una base separada, por qué `iam`/`core` son esquemas
|
||||
distintos dentro de `panels_product`, y las reglas del monolito modular.
|
||||
|
||||
## Datos sensibles (qué NO va al git)
|
||||
|
||||
| Ítem | Estado |
|
||||
|------|--------|
|
||||
| `.env` | gitignored — no commitear |
|
||||
| `data/` (`*.db`, expedientes, pdfs) | gitignored |
|
||||
| SMTP password en SaaS | vive en `platform.db` (volumen) — proteger backups |
|
||||
| Defaults de desarrollo en código | solo fallbacks locales; Coolify **exige** secrets |
|
||||
| `data/` (fallback local de archivos, si no hay Contabo) | gitignored |
|
||||
| SMTP password en SaaS | vive en `panels_platform.smtp_settings` — proteger backups |
|
||||
| Defaults de desarrollo en código | `SESSION_SECRET`/`DOCS_KEY` solo tienen fallback si `DENO_ENV` no es `production` -- fuera de eso, la app falla al arrancar si faltan |
|
||||
|
||||
## Variables de entorno (servicio `api`)
|
||||
|
||||
|
|
@ -26,15 +31,23 @@ Los fronts (Alpine + nginx) hacen proxy de `/v1` al servicio `api`, así las coo
|
|||
|
||||
| Variable | Formato | Uso |
|
||||
|----------|---------|-----|
|
||||
| `SESSION_SECRET` | string largo aleatorio | Firma cookie de sesión |
|
||||
| `DOCS_KEY` | **64** caracteres hex (32 bytes) | Cifrado de documentos |
|
||||
| `SEED_PASSWORD` | string | Password inicial de `admin` SaaS (solo al crear / migrar seed) |
|
||||
| `SESSION_SECRET` | string largo aleatorio | Firma interna (no ya la cookie -- la sesión vive en Redis, ver Fase 4) |
|
||||
| `DOCS_KEY` | **64** caracteres hex (32 bytes) | Cifrado AES-GCM de documentos |
|
||||
| `SEED_PASSWORD` | string | Password inicial usado por `api/scripts/bootstrap-admin.ts` |
|
||||
| `PANEL_LOGIN_URL` | URL absoluta | Link en correos de acceso |
|
||||
| `DATABASE_URL_PLATFORM` | `postgresql://panels_platform_app:...@host:5432/panels_platform` | Runtime, rol `_app` |
|
||||
| `DATABASE_URL_IAM` | `postgresql://panels_iam_app:...@host:5432/panels_product` | Runtime, rol `_app`, esquema `iam` |
|
||||
| `DATABASE_URL_CORE` | `postgresql://panels_core_app:...@host:5432/panels_product` | Runtime, rol `_app`, esquema `core` |
|
||||
| `DATABASE_URL_IAM_OWNER` | igual, rol `_owner` | Solo para el lookup de login por username (bypassa RLS a propósito, ver `api/iam_db.ts`) |
|
||||
| `DATABASE_URL_CORE_OWNER` | igual, rol `_owner` | Solo para resoluciones administrativas puntuales (ver `api/db.ts#getCoreDb`) |
|
||||
| `REDIS_URL_IAM` | `redis://panels_iam_redis:...@host:6379` | Sesiones |
|
||||
| `REDIS_URL_CORE` | `redis://panels_core_redis:...@host:6379` | Cache |
|
||||
|
||||
Generar:
|
||||
Generar secretos:
|
||||
|
||||
```bash
|
||||
openssl rand -hex 32 # SESSION_SECRET o DOCS_KEY
|
||||
openssl rand -hex 24 # passwords de roles Postgres/Redis
|
||||
```
|
||||
|
||||
### Recomendadas (HTTPS / Coolify)
|
||||
|
|
@ -49,50 +62,64 @@ openssl rand -hex 32 # SESSION_SECRET o DOCS_KEY
|
|||
|
||||
| Variable | Uso |
|
||||
|----------|-----|
|
||||
| `API_KEY` | Auth alternativa por header `X-API-Key` (vacío = desactivado) |
|
||||
| `API_KEY` | Auth alternativa por header `X-API-Key` **+ `X-Tenant-Id` obligatorio** (ya no ve todos los tenants, ver revisión de seguridad) |
|
||||
| `VCARD_BASE` | Prefijo QR/vCard gafetes |
|
||||
| `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` / `SMTP_FROM` | Correo por env (alternativa al panel `/smtp`) |
|
||||
| `S3_ENDPOINT` / `S3_BUCKET` / `S3_REGION` / `S3_ACCESS_KEY_ID` / `S3_SECRET_ACCESS_KEY` | Contabo Object Storage (Fase 4c). Sin esto, cae a disco local -- **no recomendado en producción** |
|
||||
|
||||
`web-panel` y `web-saas` **no** necesitan variables de entorno en runtime (estáticos + proxy nginx).
|
||||
|
||||
## Imágenes
|
||||
|
||||
- **api:** Deno 2.9 + OpenJDK 21 JRE + Liquibase (Debian; no Alpine por FFI SQLite)
|
||||
- **web-panel / web-saas:** build Node Alpine → **nginx Alpine**
|
||||
- **api:** Deno 2.9, sin FFI ni JRE (Fase 2/7 del plan de migración).
|
||||
- **migrate:** JRE 21 + Liquibase -- imagen aparte, de un solo uso, NO sirve tráfico (ver `Dockerfile.migrate`).
|
||||
- **web-panel / web-saas:** build Node Alpine → **nginx Alpine**.
|
||||
|
||||
Archivos: `Dockerfile.api`, `Dockerfile.web-panel`, `Dockerfile.web-saas`, `docker-compose.yml`, `web-panel/nginx.conf`, `web-saas/nginx.conf`.
|
||||
Archivos: `Dockerfile.api`, `Dockerfile.migrate`, `Dockerfile.provision`, `Dockerfile.web-panel`, `Dockerfile.web-saas`, `docker-compose.yml`, `web-panel/nginx.conf`, `web-saas/nginx.conf`.
|
||||
|
||||
En Coolify el servicio de PANELS se llama **`web-panel`** (FQDN ej. `panels.mrdev.mx`). La carpeta `web/` queda libre para una website futura.
|
||||
|
||||
## Paso a paso en Coolify
|
||||
|
||||
1. **Push** este repo (sin `.env` ni `data/`).
|
||||
2. Coolify → **New Resource → Docker Compose** → `docker-compose.yml`.
|
||||
3. **Persistent storage:** volumen `panel-data` → `/app/data` en `api`.
|
||||
4. **Dominios:** `web-panel` → panels; `web-saas` → saas; `api` sin FQDN (proxy `/v1`).
|
||||
5. Cargar en Coolify las **obligatorias** + `COOKIE_SECURE=true`.
|
||||
6. Deploy (Liquibase crea/migra las 2 SQLite en el volumen).
|
||||
7. Verificar `https://app…/v1/health`, login SaaS `admin` / tu `SEED_PASSWORD`.
|
||||
8. SMTP en `/smtp` o por `SMTP_*`.
|
||||
1. **Provisionar Postgres y Redis** como recursos gestionados de Coolify para el ambiente (uno de cada, no por módulo).
|
||||
2. **Aprovisionar roles/esquemas/ACLs**: correr los scripts de [`db/provision/`](../db/provision/README.md) contra ese Postgres/Redis (una vez, desde tu máquina o un job manual -- Coolify no lo hace por ti).
|
||||
3. **Aplicar Liquibase** (paso explícito, NO ocurre al arrancar la app): `./db/update.sh all --context-filter='!dev'` con las credenciales `_owner`. En staging/producción, **nunca** olvidar el `--context-filter` -- sin él, Liquibase corre TAMBIÉN los changesets de demo (`context=dev`).
|
||||
4. **Bootstrap del primer admin**: `deno run ... api/scripts/bootstrap-admin.ts platform` y `... tenant --tenant-id=... --company-code=...` (ver `db/README.md`).
|
||||
5. **Push** este repo (sin `.env` ni `data/`).
|
||||
6. Coolify → **New Resource → Docker Compose** → `docker-compose.yml` (solo para `api`/`web-panel`/`web-saas` -- si Postgres/Redis ya son recursos gestionados aparte, quitar esos servicios del compose antes de desplegar, o apuntar sus variables a los recursos gestionados en vez de los contenedores locales del compose).
|
||||
7. **Persistent storage:** volumen `panel-data` → `/app/data` en `api` (solo fallback de archivos si no hay Contabo).
|
||||
8. **Dominios:** `web-panel` → panels; `web-saas` → saas; `api` sin FQDN (proxy `/v1`).
|
||||
9. Cargar en Coolify todas las variables **obligatorias** de la tabla de arriba + `COOKIE_SECURE=true`.
|
||||
10. Deploy.
|
||||
11. Verificar `https://app…/v1/health` → debe responder `{"ok":true,"core":true,"platform":true,"redis":{"iam":true,"core":true}}`. Si algo es `false`, la app ni siquiera debería haber arrancado (fail-fast, Fase 7).
|
||||
12. Login SaaS `admin` / tu `SEED_PASSWORD`.
|
||||
13. SMTP en `/smtp` o por `SMTP_*`.
|
||||
|
||||
## Local
|
||||
## Local (todo en docker-compose, incluyendo Postgres/Redis propios)
|
||||
|
||||
```bash
|
||||
cp .env.example .env # rellenar obligatorias
|
||||
cp .env.example .env # rellenar TODAS las variables (incluye los passwords de roles Postgres/Redis)
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Sin `SESSION_SECRET` / `DOCS_KEY` / `SEED_PASSWORD` / `PANEL_LOGIN_URL`, compose **no arranca**.
|
||||
El compose local incluye: `provision` (roles/esquemas/ACLs, un solo uso) →
|
||||
`migrate` (Liquibase con datos de demo, un solo uso) → `api`/`web-panel`/`web-saas`.
|
||||
Para producción, `provision`/`migrate` NO se corren así -- ver
|
||||
`db/provision/README.md` y `db/RUNBOOK-corte.md`.
|
||||
|
||||
Sin las variables obligatorias, compose **no arranca** (`:?` en docker-compose.yml).
|
||||
|
||||
## Checklist ops
|
||||
|
||||
- [ ] `.env` y `data/` fuera del git
|
||||
- [ ] Secrets distintos a los de desarrollo
|
||||
- [ ] Backup del volumen `panel-data` (incluye DBs + SMTP guardado)
|
||||
- [ ] Secrets distintos a los de desarrollo (incluye los 6 roles Postgres + 2 Redis)
|
||||
- [ ] Backup de `panels_platform` Y `panels_product` (ver `db/backups/`) -- no solo una
|
||||
- [ ] Backup del bucket de Contabo (documentos cifrados)
|
||||
- [ ] `COOKIE_SECURE=true` en HTTPS
|
||||
- [ ] Healthcheck API OK
|
||||
- [ ] Healthcheck API OK (`/v1/health` con las 5 conexiones en `true`)
|
||||
- [ ] `--context-filter` explícito en cada corrida de Liquibase contra staging/producción
|
||||
|
||||
## Qué no hace falta
|
||||
|
||||
- Postgres/MySQL/MariaDB en Coolify
|
||||
- Contenedor monolítico Node+Deno+Java
|
||||
- Contenedor monolítico Node+Deno+Java sirviendo tráfico (Liquibase vive en su propia imagen, de un solo uso)
|
||||
- FFI de SQLite (ya no hay SQLite en ningún ambiente)
|
||||
|
|
|
|||
Loading…
Reference in a new issue