rede5/gohorsejobs
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix(seeder): use fully qualified table names for location seeding

Browse source 
docs: update DATABASE.md and DEVOPS.md for local environment setup
This commit is contained in:
Yamamoto 2026-01-03 16:55:45 -03:00
parent c9fbcc22bb
commit e59e15dd35
3 changed files with 187 additions and 227 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

73
docs/DATABASE.md
View file

@ -2,14 +2,81 @@
Complete database documentation for the GoHorseJobs platform.
> **Last Updated:** 2024-12-26
> **Database:** PostgreSQL 16+
> **Last Updated:** 2026-01-03
> **Database:** PostgreSQL 16+ (Local `postgres-main` container)
> **Connection:** Internal `gohorsejobs_dev` database via `web_proxy` network
> **ID Strategy:** UUID v7 for core tables, SERIAL for reference tables
> **Migrations:** 30 SQL files in `backend/migrations/`
---
## 📊 Entity Relationship Diagram
## <20>️ Development Environment Structure
The development environment (`apolo` server) uses a **Local Containerized Strategy** to ensure isolation and speed.
### 🏗️ Topology
```mermaid
graph TD
subgraph VPS ["Apolo Server (VPS)"]
subgraph Net ["Docker Network: web_proxy"]
PG[("postgres-main")]
BE["Backend API"]
BO["Backoffice"]
SE["Seeder API"]
end
Traefik["Traefik Proxy"]
end
Traefik --> BE
Traefik --> BO
Traefik --> SE
BE -- "internal:5432" --> PG
BO -- "internal:5432" --> PG
SE -- "internal:5432" --> PG
style PG fill:#336791,stroke:#fff,stroke-width:2px,color:#fff
```
### 🔌 Connection Details
All services connect to the database via the internal Docker network.
| Parameter | Value | Notes |
|-----------|-------|-------|
| **Host** | `postgres-main` | Internal Container Hostname |
| **Port** | `5432` | Internal Port |
| **Database** | `gohorsejobs_dev` | Dedicated Dev DB (Isolated from `main_db`) |
| **User** | `yuki` | Owner of public schema |
| **Network** | `web_proxy` | Shared Bridge Network |
| **SSL Mode** | `disable` | Internal traffic is unencrypted |
### 🚀 Access & Management
Since the database runs inside a container and is not exposed to the public internet, use the following methods for access:
**1. CLI Access (via SSH)**
```bash
# Connect to PostgreSQL shell
ssh root@apolo 'podman exec -it postgres-main psql -U yuki -d gohorsejobs_dev'
```
**2. Run Migrations**
Migrations are applied using the Backend service or manually piped:
```bash
# Manual Pipe (from local machine)
cat backend/migrations/*.sql | ssh root@apolo 'podman exec -i postgres-main psql -U yuki -d gohorsejobs_dev'
```
**3. Seeding Data**
Trigger the Seeder API (running locally) to populate data:
```bash
curl -X POST https://seeder.gohorsejobs.com/seed
```
## <20>📊 Entity Relationship Diagram
```mermaid
erDiagram

335
docs/DEVOPS.md
View file

@ -1,262 +1,155 @@
# DevOps - GoHorseJobs
# DevOps - GoHorseJobs (Development Environment)
Infraestrutura, CI/CD e deploy do projeto GoHorseJobs.
Infraestrutura, CI/CD e deploy do projeto GoHorseJobs no servidor `apolo`.
> **Last Updated:** 2024-12-26
> **Last Updated:** 2026-01-03
> **Server:** Apolo VPS
> **Tech Stack:** Podman, Systemd (Quadlet), Traefik, PostgreSQL (Local)
---
## 📁 Estrutura
## 🏗️ Architecture Diagram
```
.
├── .drone.yml # Pipeline CI/CD (Drone)
├── k8s/
│ ├── dev/ # Kubernetes - Desenvolvimento
│ ├── hml/ # Kubernetes - Homologação
│ └── prd/ # Kubernetes - Produção
│
├── backend/
│ ├── Dockerfile # Go API (multi-stage, ~73MB)
│ └── .env.example
│
├── backoffice/
│ ├── Dockerfile # NestJS (Fastify)
│ └── .env.example
│
├── frontend/
│ ├── Dockerfile # Next.js
│ └── .env.example
│
├── seeder-api/ # Node.js seeder
└── docs/ # Documentation
Simplified view of the container hierarchy, networking, and storage.
```mermaid
graph TD
subgraph Host ["Apolo VPS (Host)"]
subgraph FS ["File System (/mnt/data)"]
EnvBE["/gohorsejobs/backend/.env"]
EnvBO["/gohorsejobs/backoffice/.env"]
EnvSE["/gohorsejobs/seeder-api/.env"]
DBData[("postgres-general")]
end
subgraph Net ["Network: web_proxy"]
Traefik("Traefik")
subgraph App ["Application Containers"]
BE["Backend API (:8521)"]
BO["Backoffice (:3001)"]
SE["Seeder API (:8080)"]
FE["Frontend (:3000)"]
end
PG[("postgres-main (:5432)")]
end
end
%% Ingress
Internet((Internet)) --> Traefik
%% Routing
Traefik -- "dev.gohorsejobs.com" --> FE
Traefik -- "api-tmp.gohorsejobs.com" --> BE
Traefik -- "b-tmp.gohorsejobs.com" --> BO
Traefik -- "seeder.gohorsejobs.com" --> SE
%% Config Mounts
EnvBE -.-> BE
EnvBO -.-> BO
EnvSE -.-> SE
%% Data Persistence
PG -.-> DBData
%% Database Connections
BE --> PG
BO --> PG
SE --> PG
style PG fill:#336791,stroke:#fff,color:#fff
style Traefik fill:#f5a623,stroke:#fff,color:#fff
```
---
## 🌍 Ambientes
## 💾 Storage & Persistence (`/mnt/data`)
| Ambiente | Branch | Namespace | Réplicas | URL |
|----------|--------|-----------|----------|-----|
| **DEV** | `dev` | `gohorsejobsdev` | 1 | gohorsejobs-dev.appwrite.network |
| **HML** | `hml` | `gohorsejobshml` | 2 | - |
| **PRD** | `main` | `gohorsejobs` | 3 | gohorsejobs.com |
All persistent data and configuration files are stored in `/mnt/data` on the host.
| Host Path | Container Path | Purpose | Type |
|-----------|----------------|---------|------|
| `/mnt/data/gohorsejobs/backend/.env` | (Injected Env) | **Backend Config:** Secrets, DB URL, Port settings. | File |
| `/mnt/data/gohorsejobs/backoffice/.env` | (Injected Env) | **Backoffice Config:** Secrets, DB URL. | File |
| `/mnt/data/gohorsejobs/seeder-api/.env` | (Injected Env) | **Seeder Config:** Secrets, DB URL. | File |
| `/mnt/data/postgres-general` | `/var/lib/postgresql/data` | **Database Storage:** Main storage for `postgres-main` container. Contains `gohorsejobs_dev` DB. | Directory |
> **Backup Note:** To backup the environment, ensure `/mnt/data/gohorsejobs` and `/mnt/data/postgres-general` are included in snapshots.
---
## 🔄 Pipeline CI/CD (Drone)
## 🌍 Service Maps & Networking
### Fluxo
### 🚦 Traefik Routing
Services are exposed via Traefik labels defined in the Quadlet `.container` files.
```
dev branch → build → push (Harbor) → deploy (K8s dev)
↓
hml branch → build → push (Harbor) → deploy (K8s hml)
↓
main branch → build → push (Harbor) → deploy (K8s prd)
```
| Domain | Service | Internal Port | Host Port (Debug) |
|--------|---------|---------------|-------------------|
| `dev.gohorsejobs.com` | `gohorsejobs-frontend-dev` | `3000` | `8523` |
| `api-tmp.gohorsejobs.com` | `gohorsejobs-backend-dev` | `8521` | `8521` |
| `b-tmp.gohorsejobs.com` | `gohorsejobs-backoffice-dev` | `3001` | - |
| `seeder.gohorsejobs.com` | `gohorsejobs-seeder-dev` | `8080` | `8522` |
### Stages
1. **build-and-push** - Docker build + push to Harbor
2. **export-envs-to-k8s** - Create/update secrets
3. **deploy** - Apply manifests + rollout restart
### 🛑 Security
- **Backend/Seeder/Frontend** expose ports to the Host (`85xx`) for debugging/direct access if needed.
- **Backoffice** is *only* accessible via Traefik (internal network).
- **PostgreSQL** is *only* accessible internally via `web_proxy` network (no host port binding).
---
## 🔐 Secrets (Drone CI)
### Registry
| Secret | Description |
|--------|-------------|
| `HARBOR_USERNAME` | Harbor username |
| `HARBOR_PASSWORD` | Harbor password |
### Database
| Secret | Description |
|--------|-------------|
| `DATABASE_URL` | PostgreSQL connection string |
| `DB_SSLMODE` | `require` or `disable` |
### Application
| Secret | Description |
|--------|-------------|
| `JWT_SECRET` | JWT secret (min 32 chars) |
| `PASSWORD_PEPPER` | Password pepper |
| `CORS_ORIGINS` | Allowed origins |
### External Services
| Secret | Description |
|--------|-------------|
| `RSA_PRIVATE_KEY_BASE64` | RSA key for credentials |
| `STRIPE_SECRET_KEY` | Stripe API key |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook secret |
| `FIREBASE_SERVICE_ACCOUNT` | Firebase Admin SDK JSON |
### Storage (S3/R2)
| Secret | Description |
|--------|-------------|
| `AWS_ACCESS_KEY_ID` | Access Key |
| `AWS_SECRET_ACCESS_KEY` | Secret Key |
| `AWS_ENDPOINT` | S3-compatible endpoint |
| `S3_BUCKET` | Bucket name |
### LavinMQ
| Secret | Description |
|--------|-------------|
| `AMQP_URL` | LavinMQ connection URL |
---
## ☸️ Kubernetes
### Create Namespaces
## 🛠️ Operational Guide
### 1. View & Manage Configs
Configurations are **not** inside containers. Edit them on the host:
```bash
kubectl create namespace gohorsejobsdev
kubectl create namespace gohorsejobshml
kubectl create namespace gohorsejobs
# Edit Backend Config
vim /mnt/data/gohorsejobs/backend/.env
# Apply changes
systemctl restart gohorsejobs-backend-dev
```
### Registry Secret
### 2. Full Environment Restart
To restart all GoHorseJobs related services (excluding Database):
```bash
kubectl create secret docker-registry harbor-registry \
--docker-server=in.gohorsejobs.com \
--docker-username=<user> \
--docker-password=<pass> \
-n gohorsejobsdev
systemctl restart gohorsejobs-backend-dev gohorsejobs-backoffice-dev gohorsejobs-seeder-dev gohorsejobs-frontend-dev
```
### Deploy Manual
### 3. Database Access
Access the local database directly via the `postgres-main` container:
```bash
# DEV
kubectl apply -f k8s/dev/ -n gohorsejobsdev
# Check pods
kubectl get pods -n gohorsejobsdev
# Check logs
kubectl logs -f deployment/gohorse-backend -n gohorsejobsdev
# Rollout restart
kubectl rollout restart deployment/gohorse-backend -n gohorsejobsdev
# Internal Connection
docker exec -it postgres-main psql -U yuki -d gohorsejobs_dev
```
---
## 🐳 Docker
## 🚀 Deployment Pipeline (Manual)
### Build Local
Current workflow uses **Local Build** -> **Forgejo Registry** -> **Server Pull**.
### 1. Build & Push (Local Machine)
```bash
# Backend
cd backend && docker build -t gohorse-backend .
# Login
podman login forgejo-gru.rede5.com.br
# Backoffice
cd backoffice && docker build -t gohorse-backoffice .
# Frontend
cd frontend && docker build -t gohorse-frontend .
```
### Docker Compose (Dev)
```yaml
version: '3.8'
services:
backend:
build: ./backend
ports:
- "8521:8521"
env_file:
- ./backend/.env
backoffice:
build: ./backoffice
ports:
- "3001:3001"
env_file:
- ./backoffice/.env
frontend:
build: ./frontend
ports:
- "3000:3000"
env_file:
- ./frontend/.env
```
---
## 🗄️ Database
### Connection
```bash
# PostgreSQL hosted
Host: db-60059.dc-sp-1.absamcloud.com
Port: 26868
SSL: require
```
### Databases
| Environment | Database |
|-------------|----------|
| DEV | `gohorsejobs_dev` |
| HML | `gohorsejobs_hml` |
| PRD | `gohorsejobs` |
### Migrations
```bash
# Build
cd backend
go run ./cmd/manual_migrate
podman build -t forgejo-gru.rede5.com.br/rede5/gohorsejobs-backend:latest .
# Push
podman push forgejo-gru.rede5.com.br/rede5/gohorsejobs-backend:latest
```
### Seeder
### 2. Deploy (On Apolo Server)
```bash
cd seeder-api
npm install
npm run seed
ssh root@apolo
# Pull new image
podman pull forgejo-gru.rede5.com.br/rede5/gohorsejobs-backend:latest
# Restart service (Systemd handles container recreation)
systemctl restart gohorsejobs-backend-dev
```
---
## 👤 Test Users
> **Nota:** O SuperAdmin foi atualizado via migration `032_update_superadmin_lol.sql`.
### SuperAdmin
- **Login:** `lol`
- **Email:** `lol@gohorsejobs.com`
- **Nome:** Dr. Horse Expert
- **Password:** *trocar no primeiro acesso* (status `force_change_password`)
### Company Admin
| Login | Password |
|-------|----------|
| `takeshi_yamamoto` | `Takeshi@2025` |
### Candidate
| Login | Password |
|-------|----------|
| `paulo_santos` | `User@2025` |
---
## 📋 Deploy Checklist
- [ ] Create K8s namespace
- [ ] Create `harbor-registry` secret
- [ ] Add Drone CI secrets
- [ ] Create database
- [ ] Run migrations
- [ ] Run seeder (optional)
- [ ] Push to branch
- [ ] Verify pipeline logs
- [ ] Test `/health` endpoint
- [ ] Test `/docs` endpoint

6
seeder-api/src/seeders/location-loader.js
View file

@ -13,9 +13,9 @@ const SQL_DIR = join(__dirname, '..', '..', 'sql');
* Table name mapping from SQL dumps to our schema
*/
const TABLE_MAPPING = {
'public.regions': 'continents',
'regions': 'continents',
'public.subregions': 'subregions',
'public.regions': 'public.continents',
'regions': 'public.continents',
'public.subregions': 'public.subregions',
};
/**