flea/CONFIGURATION.md

# Configuration Guide

This document describes how to configure the NOCR microservices system for different deployment scenarios.

## Table of Contents

1. [Configuration System Overview](#configuration-system-overview)
2. [Deployment Scenarios](#deployment-scenarios)
   - [Local Development (VS Code)](#local-development-vs-code)
   - [Docker Compose](#docker-compose)
   - [Kubernetes](#kubernetes)
3. [Configuration Priority](#configuration-priority)
4. [Service-Specific Settings](#service-specific-settings)
5. [Debug Mode](#debug-mode)
6. [Troubleshooting](#troubleshooting)

---

## Configuration System Overview

The system uses ASP.NET Core's layered configuration approach with the following sources (listed from lowest to highest priority):

1. `appsettings.json` - Base configuration (committed to git)
2. `appsettings.{Environment}.json` - Environment-specific settings (some committed, some not)
3. User Secrets - Development-only secrets (stored locally, not in git)
4. **Environment Variables** - **Highest priority** (overrides everything)

### Important Notes

- **Never commit secrets to git**
- Use `.example` files as templates
- Environment Variables always win in priority
- Each service has its own configuration requirements

---

## Deployment Scenarios

### Local Development (VS Code)

For debugging services individually in VS Code:

#### Step 1: Create Environment-Specific Configuration

For each service you want to run, create `appsettings.Development.json` from the example:

```bash
# Telegram Listener
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.Development.json.example appsettings.Development.json

# Text Matcher
cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.Development.json.example appsettings.Development.json

# Users
cd users/src/Nocr.Users.Host
cp appsettings.Development.json.example appsettings.Development.json

# Telegram Client
cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.Development.json.example appsettings.Development.json
```

#### Step 2: Fill in Your Values

Edit each `appsettings.Development.json` file and replace placeholder values with your actual credentials.

**Example for Telegram Listener:**
```json
{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://admin:admin@localhost:5672/"
  },
  "WTelegramClientOptions": {
    "ApiId": "22101230",
    "ApiHash": "c72f884d8eb84cb7134a14362bff060b",
    "PhoneNumber": "79167310711"
  }
}
```

#### Step 3: Alternative - Use Environment Variables

Instead of creating `appsettings.Development.json`, you can use environment variables in `.vscode/launch.json`:

```json
{
  "name": "Launch Service",
  "type": "coreclr",
  "request": "launch",
  "env": {
    "ASPNETCORE_ENVIRONMENT": "Development",
    "RebusRabbitMqOptions__ConnectionString": "amqp://admin:admin@localhost:5672/",
    "WTelegramClientOptions__ApiId": "22101230",
    "WTelegramClientOptions__ApiHash": "c72f884d8eb84cb7134a14362bff060b",
    "WTelegramClientOptions__PhoneNumber": "79167310711"
  }
}
```

**Note:** Use double underscores `__` to represent nested configuration sections in environment variables.

#### Step 4: Start Infrastructure

You need RabbitMQ and databases running:

```bash
# Start only infrastructure services
docker-compose up nocr-rabbitmq nocr-text-matcher-db nocr-users-db -d
```

#### Step 5: Debug in VS Code

Press F5 or use the Run and Debug panel to start your service.

---

### Docker Compose

For running the entire system locally with Docker:

#### Step 1: Create .nocr.env File

```bash
cd /path/to/project/root
cp .nocr.env.example .nocr.env
```

#### Step 2: Edit .nocr.env

Open `.nocr.env` and fill in your actual credentials:

```bash
# Telegram Listener - get from https://my.telegram.org/apps
WTelegramClientOptions__ApiId=22101230
WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b
WTelegramClientOptions__PhoneNumber=79167310711

# Telegram Client Bot - get from @BotFather
TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz

# Optional: Enable debug logging
# NOCR_DEBUG_MODE=true
```

#### Step 3: Create DockerCompose Configuration Files

Each service needs `appsettings.DockerCompose.json`:

```bash
# For each service
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd users/src/Nocr.Users.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json
```

**Note:** These files contain Docker network hostnames (e.g., `nocr-rabbitmq:5672` instead of `localhost:5672`).

#### Step 4: Start All Services

```bash
docker-compose up
```

Or build and start:

```bash
docker-compose up --build
```

#### Step 5: Verify Services are Running

- Telegram Listener: http://localhost:5040/health
- Text Matcher: http://localhost:5041/health
- Users: http://localhost:5042/health
- Telegram Client: http://localhost:5050/health
- RabbitMQ Management: http://localhost:15672 (admin/admin)

---

### Kubernetes

For production deployment on Kubernetes:

#### Step 1: Create Kubernetes Secrets

**Do NOT use `.nocr.env` file in K8s.** Instead, create Secrets:

```bash
# Create namespace
kubectl create namespace nocr

# Create secrets for Telegram Listener
kubectl create secret generic telegram-listener-secrets \
  --from-literal=WTelegramClientOptions__ApiId=22101230 \
  --from-literal=WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b \
  --from-literal=WTelegramClientOptions__PhoneNumber=79167310711 \
  -n nocr

# Create secrets for Telegram Client
kubectl create secret generic telegram-client-secrets \
  --from-literal=TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz \
  -n nocr
```

#### Step 2: Reference Secrets in Deployment

Example Kubernetes deployment manifest:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: telegram-listener
  namespace: nocr
spec:
  template:
    spec:
      containers:
      - name: telegram-listener
        image: nocr-telegram-listener:latest
        env:
        - name: ASPNETCORE_ENVIRONMENT
          value: "Production"
        envFrom:
        - secretRef:
            name: telegram-listener-secrets
```

#### Step 3: Create appsettings.Production.json

Create production config files with K8s service names:

```json
{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://admin:admin@rabbitmq-service:5672/"
  }
}
```

**Note:** Secrets from environment variables will override these values.

#### Step 4: Deploy

```bash
kubectl apply -f deployment/
```

---

## Configuration Priority

Understanding priority is crucial when troubleshooting configuration issues.

### Priority Order (Lowest → Highest)

1. ⬇️ `appsettings.json` (base)
2. ⬆️ `appsettings.{Environment}.json` (e.g., Development, DockerCompose, Production)
3. ⬆️ User Secrets (Development only)
4. ⬆️⬆️ **Environment Variables (ALWAYS WINS)**

### Example Scenario

If you have:

- `appsettings.json`: `"ConnectionString": ""`
- `appsettings.DockerCompose.json`: `"ConnectionString": "amqp://admin:admin@nocr-rabbitmq:5672/"`
- Environment Variable: `RebusRabbitMqOptions__ConnectionString=amqp://admin:admin@localhost:5672/`

**Result:** Environment variable wins! Connection will use `localhost:5672`.

This is why we removed `appsettings.protected.json` - it was overriding Docker Compose settings incorrectly.

---

## Service-Specific Settings

### Telegram Listener

**Required Configuration:**

```json
{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  },
  "WTelegramClientOptions": {
    "ApiId": "YOUR_API_ID",
    "ApiHash": "YOUR_API_HASH",
    "PhoneNumber": "YOUR_PHONE_NUMBER"
  }
}
```

**How to obtain Telegram credentials:**
1. Visit https://my.telegram.org/apps
2. Log in with your phone number
3. Create a new application
4. Copy `api_id` and `api_hash`

---

### Text Matcher

**Required Configuration:**

```json
{
  "ConnectionStrings": {
    "TextMatcherContext": "server=host;port=3306;database=nocr_text_matcher;uid=root;pwd=password"
  },
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  }
}
```

**Ports by environment:**
- Development: `localhost:3316`
- Docker Compose: `nocr-text-matcher-db:3306`
- Kubernetes: `text-matcher-db-service:3306`

---

### Users

**Required Configuration:**

```json
{
  "ConnectionStrings": {
    "UsersContext": "server=host;port=3306;database=nocr_users;uid=root;pwd=password"
  }
}
```

**Ports by environment:**
- Development: `localhost:3326`
- Docker Compose: `nocr-users-db:3306`
- Kubernetes: `users-db-service:3306`

---

### Telegram Client

**Required Configuration:**

```json
{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  },
  "UsersRestEaseOptions": {
    "BasePath": "http://users-service:8080"
  },
  "TextMatcherRestEaseOptions": {
    "BasePath": "http://text-matcher-service:8080"
  },
  "TelegramBotOptions": {
    "Token": "YOUR_BOT_TOKEN"
  }
}
```

**How to obtain Bot Token:**
1. Open Telegram and find @BotFather
2. Send `/newbot` command
3. Follow instructions to create your bot
4. Copy the token provided

---

## Debug Mode

Enable debug mode to print masked configuration values on startup.

### How to Enable

Set environment variable:

```bash
export NOCR_DEBUG_MODE=true
```

Or in `.nocr.env`:

```bash
NOCR_DEBUG_MODE=true
```

Or in `docker-compose.yml`:

```yaml
environment:
  NOCR_DEBUG_MODE: "true"
```

### Example Output

When debug mode is enabled, you'll see masked configuration on startup:

```
=== [NOCR_DEBUG] Configuration Values ===
[NOCR_DEBUG] RebusRabbitMqOptions:
[NOCR_DEBUG]   ConnectionString: amqp://admin:***@nocr-rabbitmq:5672/
[NOCR_DEBUG]   InputQueueName: nocr.telegram.listener.queue
[NOCR_DEBUG]   DirectExchangeName: nocr.direct
[NOCR_DEBUG]   TopicsExchangeName: nocr.topics
[NOCR_DEBUG] WTelegramClientOptions:
[NOCR_DEBUG]   ApiId: 22101230
[NOCR_DEBUG]   ApiHash: c7...0b
[NOCR_DEBUG]   PhoneNumber: 79...11
=== [NOCR_DEBUG] End Configuration ===
```

**Security:** Passwords and secrets are automatically masked to protect sensitive data.

---

## Troubleshooting

### Problem: Service can't connect to RabbitMQ

**Symptoms:**
```
Connection refused to nocr-rabbitmq:5672
```

**Solution:**
1. Check if RabbitMQ is running: `docker-compose ps nocr-rabbitmq`
2. Enable debug mode: `NOCR_DEBUG_MODE=true`
3. Verify ConnectionString in debug output
4. For Docker Compose, ensure hostname is `nocr-rabbitmq:5672`, NOT `localhost:5672`
5. Check if environment variable is overriding appsettings

---

### Problem: Service can't connect to database

**Symptoms:**
```
Unable to connect to any of the specified MySQL hosts
```

**Solution:**
1. Check database is running: `docker-compose ps nocr-text-matcher-db`
2. Enable debug mode to see masked connection string
3. Verify hostname in connection string:
   - Docker Compose: `nocr-text-matcher-db:3306`
   - Local dev: `localhost:3316`
4. Wait for database healthcheck to pass (may take 10-30 seconds on first start)

---

### Problem: Configuration values not applied

**Symptoms:**
- Settings in `appsettings.Development.json` are ignored
- Service uses wrong configuration

**Solution:**
1. Check `ASPNETCORE_ENVIRONMENT` is set correctly:
   - VS Code: `Development`
   - Docker Compose: `DockerCompose`
   - K8s: `Production`
2. Enable debug mode to see which values are loaded
3. Check for environment variables overriding your settings
4. Remember: **Environment Variables always win!**

---

### Problem: Secrets exposed in logs

**Solution:**
- Debug mode automatically masks sensitive values
- Never log configuration in production without masking
- Review debug output format in `Startup.cs`:
  - Passwords: `amqp://user:***@host`
  - Secrets: `ab...yz` (first 2 + last 2 chars only)

---

### Problem: Missing .nocr.env file

**Symptoms:**
```
docker-compose up fails with missing environment variables
```

**Solution:**
1. Copy example file: `cp .nocr.env.example .nocr.env`
2. Fill in your actual values
3. Make sure `.nocr.env` is in the project root (same level as `docker-compose.yml`)

---

## Quick Reference

### Environment Variable Naming Convention

ASP.NET Core uses double underscores `__` to represent nested JSON structure:

```bash
# JSON: { "Section": { "Key": "Value" } }
# Environment Variable:
Section__Key=Value

# Example:
WTelegramClientOptions__ApiId=12345
RebusRabbitMqOptions__ConnectionString=amqp://localhost
```

### File Locations

```
flea/                                   # Project root
├── .nocr.env                          # Your secrets (gitignored)
├── .nocr.env.example                  # Template (committed)
├── docker-compose.yml                 # References .nocr.env
│
├── telegram-listener/
│   └── src/Nocr.TelegramListener.Host/
│       ├── appsettings.json                        # Base (committed)
│       ├── appsettings.Development.json            # Local dev (gitignored)
│       ├── appsettings.Development.json.example    # Template (committed)
│       ├── appsettings.DockerCompose.json          # Docker (gitignored)
│       └── appsettings.DockerCompose.json.example  # Template (committed)
│
└── [same structure for text-matcher, users, telegram-client]
```

---

## Security Best Practices

1. ✅ **Never commit secrets** - Use `.example` files as templates
2. ✅ **Use environment variables** for sensitive data in production
3. ✅ **Use K8s Secrets** for Kubernetes deployments
4. ✅ **Enable debug mode** only when troubleshooting
5. ✅ **Rotate credentials** regularly
6. ✅ **Use User Secrets** for local development (optional)
7. ❌ **Don't commit** `.nocr.env` or `appsettings.*.json` (except `.example` files)
8. ❌ **Don't use** `appsettings.protected.json` (removed from code)

---

## Additional Resources

- [ASP.NET Core Configuration Docs](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/)
- [Docker Compose Environment Variables](https://docs.docker.com/compose/environment-variables/)
- [Kubernetes Secrets](https://kubernetes.io/docs/concepts/configuration/secret/)
- [Telegram API Documentation](https://core.telegram.org/api)
- [Telegram Bot API](https://core.telegram.org/bots/api)