# Backend Deployment & Operations Guide

## Quick Start

### Development
```bash
cd backend
cp .env.example .env.local
npm install
npm run db:migrate
npm run dev
```

Server: http://localhost:5000

### Production
```bash
export NODE_ENV=production
npm install
npm run db:migrate
npm run prod
```

## Pre-Deployment Checklist

### Security
- [ ] All environment variables set (check .env.example)
- [ ] JWT secrets are strong and random (32+ chars)
- [ ] Database password is strong
- [ ] HTTPS/TLS certificate obtained
- [ ] CORS_ORIGIN set to production domain
- [ ] FRONTEND_URL set to production domain
- [ ] API_URL configured on frontend
- [ ] Rate limits adjusted for production scale
- [ ] Helmet.js CSP configured for production domains

### Database
- [ ] PostgreSQL 13+ installed
- [ ] Database backups configured
- [ ] Migrations applied successfully
- [ ] Indexes verified
- [ ] Connection pooling optimized
- [ ] SSL enabled for DB connections
- [ ] Automated backups scheduled

### Application
- [ ] All dependencies installed
- [ ] Tests passing
- [ ] Error logging configured (Sentry)
- [ ] Performance monitoring configured (DataDog)
- [ ] PM2 ecosystem.config.js created
- [ ] Process restart script configured
- [ ] Log rotation configured
- [ ] Health checks set up

### Monitoring
- [ ] Database monitoring enabled
- [ ] Runtime metrics collecting
- [ ] Alerting configured
- [ ] Log aggregation set up
- [ ] Error tracking active
- [ ] Uptime monitoring enabled

## Deployment Methods

### Method 1: VPS with PM2

**Setup:**
```bash
# Install PM2 globally
npm install -g pm2

# Create ecosystem.config.js
cat > ecosystem.config.js << EOF
module.exports = {
  apps: [{
    name: 'serdark-backend',
    script: './src/server.js',
    env: {
      NODE_ENV: 'development'
    },
    env_production: {
      NODE_ENV: 'production'
    },
    instances: 'max',
    exec_mode: 'cluster',
    error_file: './logs/pm2-error.log',
    out_file: './logs/pm2-out.log',
    log_date_format: 'YYYY-MM-DD HH:mm:ss Z'
  }]
};
EOF

# Start application
pm2 start ecosystem.config.js --env production

# Setup auto-restart
pm2 startup
pm2 save

# View logs
pm2 logs serdark-backend

# Monitor
pm2 monit
```

**Nginx reverse proxy:**
```nginx
upstream serdark_backend {
    server localhost:5000;
    server localhost:5001;
    server localhost:5002;
}

server {
    listen 80;
    server_name api.serdark.digital;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name api.serdark.digital;

    ssl_certificate /etc/letsencrypt/live/api.serdark.digital/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.serdark.digital/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 10m;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "DENY" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    location / {
        proxy_pass http://serdark_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
        
        # Timeouts
        proxy_connect_timeout 60s;
        proxy_send_timeout 60s;
        proxy_read_timeout 60s;
    }

    # Health check
    location /health {
        proxy_pass http://serdark_backend;
        access_log off;
    }
}
```

### Method 2: Docker & Docker Compose

**Dockerfile:**
```dockerfile
FROM node:18-alpine

WORKDIR /app

# Install dependencies
COPY package*.json ./
RUN npm ci --only=production

# Copy source
COPY src ./src
COPY database ./database

# Expose port
EXPOSE 5000

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
  CMD node -e "require('http').get('http://localhost:5000/health', (r) => {if (r.statusCode !== 200) throw new Error(r.statusCode)})"

# Start app
CMD ["node", "src/server.js"]
```

**docker-compose.yml:**
```yaml
version: '3.8'

services:
  backend:
    build: ./backend
    ports:
      - "5000:5000"
    environment:
      NODE_ENV: production
      DB_HOST: db
      DB_PORT: 5432
      DB_NAME: serdark_db
      DB_USER: serdark_user
      DB_PASSWORD: ${DB_PASSWORD}
      JWT_SECRET: ${JWT_SECRET}
      JWT_REFRESH_SECRET: ${JWT_REFRESH_SECRET}
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_DB: serdark_db
      POSTGRES_USER: serdark_user
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U serdark_user"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    restart: unless-stopped

volumes:
  postgres_data:
```

**Deploy:**
```bash
docker-compose up -d
docker-compose logs -f backend
```

### Method 3: Kubernetes

**Deployment manifest:**
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: serdark-backend
  labels:
    app: serdark-backend
spec:
  replicas: 3
  selector:
    matchLabels:
      app: serdark-backend
  template:
    metadata:
      labels:
        app: serdark-backend
    spec:
      containers:
      - name: backend
        image: registry.example.com/serdark-backend:1.0.0
        ports:
        - containerPort: 5000
        envFrom:
        - configMapRef:
            name: backend-config
        - secretRef:
            name: backend-secrets
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 5000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /ready
            port: 5000
          initialDelaySeconds: 10
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: serdark-backend-service
spec:
  selector:
    app: serdark-backend
  ports:
  - port: 80
    targetPort: 5000
  type: LoadBalancer
```

## Monitoring & Logging

### Log Aggregation

**With ELK Stack:**
```bash
# Send logs to ElasticSearch
npm install --save winston-elasticsearch

# Configure in logger.js
// Add Elasticsearch transport
```

**With Datadog:**
```javascript
// Add Datadog tracer
import tracer from 'dd-trace';
tracer.init();

// Already configured via DDG_* env vars
```

### Alerting Rules

```yaml
# Prometheus alerting rules
groups:
- name: serdark-backend
  rules:
  - alert: HighErrorRate
    expr: rate(errors_total[5m]) > 0.05
    for: 5m
    annotations:
      summary: "High error rate detected"
  
  - alert: DatabaseConnectionFailed
    expr: db_connection_errors_total > 0
    for: 1m
    annotations:
      summary: "Database connection failed"
  
  - alert: HighRateLimitViolations
    expr: rate_limit_exceeded_total[5m] > 100
    for: 5m
    annotations:
      summary: "High rate limit violations"
```

### Performance Optimization

```javascript
// Enable clustering
import cluster from 'cluster';
import { cpuCount } from 'os';

if (cluster.isMaster) {
  for (let i = 0; i < cpuCount(); i++) {
    cluster.fork();
  }
  cluster.on('exit', (worker) => {
    console.log(`Worker ${worker.process.pid} died`);
    cluster.fork();
  });
} else {
  startServer();
}
```

## Scaling Strategy

### Horizontal Scaling
```
┌─────────────────────────────────────┐
│      Load Balancer (Nginx)          │
└──────────────┬──────────────────────┘
               │
       ┌───────┼───────┐
       │       │       │
   Backend1 Backend2 Backend3
       │       │       │
       └───────┼───────┘
               │
        ┌──────┴──────┐
        │   Database  │
        │ PostgreSQL  │
        └─────────────┘
```

### Vertical Scaling
- Increase container memory/CPU limits
- Optimize database indexes
- Enable query caching with Redis
- Implement response compression

### Database Optimization
```sql
-- Create indexes for frequently queried fields
CREATE INDEX idx_users_email ON users(email);
CREATE INDEX idx_sessions_user_id ON sessions(user_id);
CREATE INDEX idx_audit_logs_user_id ON audit_logs(user_id);
CREATE INDEX idx_audit_logs_created_at ON audit_logs(created_at);

-- Analyze query performance
EXPLAIN ANALYZE SELECT * FROM users WHERE email = 'test@example.com';

-- Vacuum and analyze
VACUUM ANALYZE;
```

## Backup & Recovery

### Database Backup

```bash
# Full backup
pg_dump -U serdark_user -d serdark_db > backup_$(date +%Y%m%d_%H%M%S).sql

# With compression
pg_dump -U serdark_user -d serdark_db | gzip > backup_$(date +%Y%m%d).sql.gz

# Automated backup script
#!/bin/bash
BACKUP_DIR="/backups/postgres"
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR
pg_dump -U serdark_user -d serdark_db | gzip > $BACKUP_DIR/backup_$DATE.sql.gz

# Keep only last 30 days
find $BACKUP_DIR -name "backup_*.sql.gz" -mtime +30 -delete
```

### Recovery

```bash
# Restore from backup
gunzip < backup_20240130.sql.gz | psql -U serdark_user -d serdark_db
```

## Troubleshooting

### High Memory Usage
```bash
# Check Node.js memory
node --max-old-space-size=2048 src/server.js

# Monitor with PM2
pm2 monit
```

### Database Connection Pool Issues
```javascript
// Adjust pool settings in .env
DB_POOL_MIN=5
DB_POOL_MAX=20
```

### Slow Queries
```sql
-- Find slow queries
SELECT query, mean_time, calls 
FROM pg_stat_statements 
ORDER BY mean_time DESC LIMIT 10;

-- Enable query logging
SET log_min_duration_statement = 1000; -- 1 second
```

## Emergency Procedures

### Database Down
1. Check PostgreSQL status: `systemctl status postgresql`
2. Check database logs: `/var/log/postgresql/`
3. Try restart: `systemctl restart postgresql`
4. Restore from backup if needed

### High CPU Usage
1. Check running processes: `pm2 monit`
2. Check database queries: `SELECT * FROM pg_stat_activity`
3. Kill stuck processes: `pm2 kill` then restart

### Security Incident
1. Check audit logs: `SELECT * FROM audit_logs WHERE severity = 'critical'`
2. Revoke compromised tokens: `UPDATE sessions SET expires_at = NOW()`
3. Reset passwords: `UPDATE users SET password_hash = NULL`
4. Report and escalate

## Support & Documentation

- [Backend Setup](./BACKEND_SETUP.md)
- [API Documentation](./API_DOCUMENTATION.md)
- [Integration Guide](./INTEGRATION_GUIDE.md)
- [Security Documentation](./SECURITY.md) (Frontend)