flea/CONFIGURATION.md

Configuration Guide

This document describes how to configure the NOCR microservices system for different deployment scenarios.

Table of Contents

1. Configuration System Overview
2. Deployment Scenarios
   • Local Development (VS Code)
   • Docker Compose
   • Kubernetes
3. Configuration Priority
4. Service-Specific Settings
5. Debug Mode
6. Troubleshooting

---

Configuration System Overview

The system uses ASP.NET Core's layered configuration approach with the following sources (listed from lowest to highest priority):

1. appsettings.json - Base configuration (committed to git)
2. appsettings.{Environment}.json - Environment-specific settings (some committed, some not)
3. User Secrets - Development-only secrets (stored locally, not in git)
4. Environment Variables - Highest priority (overrides everything)

Important Notes

• Never commit secrets to git
• Use .example files as templates
• Environment Variables always win in priority
• Each service has its own configuration requirements

---

Deployment Scenarios

Local Development (VS Code)

For debugging services individually in VS Code:

Step 1: Create Environment-Specific Configuration

For each service you want to run, create appsettings.Development.json from the example:

# Telegram Listener
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.Development.json.example appsettings.Development.json

# Text Matcher
cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.Development.json.example appsettings.Development.json

# Users
cd users/src/Nocr.Users.Host
cp appsettings.Development.json.example appsettings.Development.json

# Telegram Client
cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.Development.json.example appsettings.Development.json

Step 2: Fill in Your Values

Edit each appsettings.Development.json file and replace placeholder values with your actual credentials.

Example for Telegram Listener:

{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://admin:admin@localhost:5672/"
  },
  "WTelegramClientOptions": {
    "ApiId": "22101230",
    "ApiHash": "c72f884d8eb84cb7134a14362bff060b",
    "PhoneNumber": "79167310711"
  }
}

Step 3: Alternative - Use Environment Variables

Instead of creating appsettings.Development.json, you can use environment variables in .vscode/launch.json:

{
  "name": "Launch Service",
  "type": "coreclr",
  "request": "launch",
  "env": {
    "ASPNETCORE_ENVIRONMENT": "Development",
    "RebusRabbitMqOptions__ConnectionString": "amqp://admin:admin@localhost:5672/",
    "WTelegramClientOptions__ApiId": "22101230",
    "WTelegramClientOptions__ApiHash": "c72f884d8eb84cb7134a14362bff060b",
    "WTelegramClientOptions__PhoneNumber": "79167310711"
  }
}

Note: Use double underscores __ to represent nested configuration sections in environment variables.

Step 4: Start Infrastructure

You need RabbitMQ and databases running:

# Start only infrastructure services
docker-compose up nocr-rabbitmq nocr-text-matcher-db nocr-users-db -d

Step 5: Debug in VS Code

Press F5 or use the Run and Debug panel to start your service.

---

Docker Compose

For running the entire system locally with Docker:

Step 1: Create .nocr.env File

cd /path/to/project/root
cp .nocr.env.example .nocr.env

Step 2: Edit .nocr.env

Open .nocr.env and fill in your actual credentials:

# Telegram Listener - get from https://my.telegram.org/apps
WTelegramClientOptions__ApiId=22101230
WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b
WTelegramClientOptions__PhoneNumber=79167310711

# Telegram Client Bot - get from @BotFather
TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz

# Optional: Enable debug logging
# NOCR_DEBUG_MODE=true

Step 3: Create DockerCompose Configuration Files

Each service needs appsettings.DockerCompose.json:

# For each service
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd users/src/Nocr.Users.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json

Note: These files contain Docker network hostnames (e.g., nocr-rabbitmq:5672 instead of localhost:5672).

Step 4: Start All Services

docker-compose up

Or build and start:

docker-compose up --build

Step 5: Verify Services are Running

• Telegram Listener: http://localhost:5040/health
• Text Matcher: http://localhost:5041/health
• Users: http://localhost:5042/health
• Telegram Client: http://localhost:5050/health
• RabbitMQ Management: http://localhost:15672 (admin/admin)

---

Kubernetes

For production deployment on Kubernetes:

Step 1: Create Kubernetes Secrets

Do NOT use .nocr.env file in K8s. Instead, create Secrets:

# Create namespace
kubectl create namespace nocr

# Create secrets for Telegram Listener
kubectl create secret generic telegram-listener-secrets \
  --from-literal=WTelegramClientOptions__ApiId=22101230 \
  --from-literal=WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b \
  --from-literal=WTelegramClientOptions__PhoneNumber=79167310711 \
  -n nocr

# Create secrets for Telegram Client
kubectl create secret generic telegram-client-secrets \
  --from-literal=TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz \
  -n nocr

Step 2: Reference Secrets in Deployment

Example Kubernetes deployment manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: telegram-listener
  namespace: nocr
spec:
  template:
    spec:
      containers:
      - name: telegram-listener
        image: nocr-telegram-listener:latest
        env:
        - name: ASPNETCORE_ENVIRONMENT
          value: "Production"
        envFrom:
        - secretRef:
            name: telegram-listener-secrets

Step 3: Create appsettings.Production.json

Create production config files with K8s service names:

{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://admin:admin@rabbitmq-service:5672/"
  }
}

Note: Secrets from environment variables will override these values.

Step 4: Deploy

kubectl apply -f deployment/

---

Configuration Priority

Understanding priority is crucial when troubleshooting configuration issues.

Priority Order (Lowest → Highest)

1. ⬇️ appsettings.json (base)
2. ⬆️ appsettings.{Environment}.json (e.g., Development, DockerCompose, Production)
3. ⬆️ User Secrets (Development only)
4. ⬆️⬆️ Environment Variables (ALWAYS WINS)

Example Scenario

If you have:

• appsettings.json: "ConnectionString": ""
• appsettings.DockerCompose.json: "ConnectionString": "amqp://admin:admin@nocr-rabbitmq:5672/"
• Environment Variable: RebusRabbitMqOptions__ConnectionString=amqp://admin:admin@localhost:5672/

Result: Environment variable wins! Connection will use localhost:5672.

This is why we removed appsettings.protected.json - it was overriding Docker Compose settings incorrectly.

---

Service-Specific Settings

Telegram Listener

Required Configuration:

{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  },
  "WTelegramClientOptions": {
    "ApiId": "YOUR_API_ID",
    "ApiHash": "YOUR_API_HASH",
    "PhoneNumber": "YOUR_PHONE_NUMBER"
  }
}

How to obtain Telegram credentials:

1. Visit https://my.telegram.org/apps
2. Log in with your phone number
3. Create a new application
4. Copy api_id and api_hash

---

Text Matcher

Required Configuration:

{
  "ConnectionStrings": {
    "TextMatcherContext": "server=host;port=3306;database=nocr_text_matcher;uid=root;pwd=password"
  },
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  }
}

Ports by environment:

• Development: localhost:3316
• Docker Compose: nocr-text-matcher-db:3306
• Kubernetes: text-matcher-db-service:3306

---

Users

Required Configuration:

{
  "ConnectionStrings": {
    "UsersContext": "server=host;port=3306;database=nocr_users;uid=root;pwd=password"
  }
}

Ports by environment:

• Development: localhost:3326
• Docker Compose: nocr-users-db:3306
• Kubernetes: users-db-service:3306

---

Telegram Client

Required Configuration:

{
  "RebusRabbitMqOptions": {
    "ConnectionString": "amqp://user:pass@host:port/"
  },
  "UsersRestEaseOptions": {
    "BasePath": "http://users-service:8080"
  },
  "TextMatcherRestEaseOptions": {
    "BasePath": "http://text-matcher-service:8080"
  },
  "TelegramBotOptions": {
    "Token": "YOUR_BOT_TOKEN"
  }
}

How to obtain Bot Token:

1. Open Telegram and find @BotFather
2. Send /newbot command
3. Follow instructions to create your bot
4. Copy the token provided

---

Debug Mode

Enable debug mode to print masked configuration values on startup.

How to Enable

Set environment variable:

export NOCR_DEBUG_MODE=true

Or in .nocr.env:

NOCR_DEBUG_MODE=true

Or in docker-compose.yml:

environment:
  NOCR_DEBUG_MODE: "true"

Example Output

When debug mode is enabled, you'll see masked configuration on startup:

=== [NOCR_DEBUG] Configuration Values ===
[NOCR_DEBUG] RebusRabbitMqOptions:
[NOCR_DEBUG]   ConnectionString: amqp://admin:***@nocr-rabbitmq:5672/
[NOCR_DEBUG]   InputQueueName: nocr.telegram.listener.queue
[NOCR_DEBUG]   DirectExchangeName: nocr.direct
[NOCR_DEBUG]   TopicsExchangeName: nocr.topics
[NOCR_DEBUG] WTelegramClientOptions:
[NOCR_DEBUG]   ApiId: 22101230
[NOCR_DEBUG]   ApiHash: c7...0b
[NOCR_DEBUG]   PhoneNumber: 79...11
=== [NOCR_DEBUG] End Configuration ===

Security: Passwords and secrets are automatically masked to protect sensitive data.

---

Troubleshooting

Problem: Service can't connect to RabbitMQ

Symptoms:

Connection refused to nocr-rabbitmq:5672

Solution:

1. Check if RabbitMQ is running: docker-compose ps nocr-rabbitmq
2. Enable debug mode: NOCR_DEBUG_MODE=true
3. Verify ConnectionString in debug output
4. For Docker Compose, ensure hostname is nocr-rabbitmq:5672, NOT localhost:5672
5. Check if environment variable is overriding appsettings

---

Problem: Service can't connect to database

Symptoms:

Unable to connect to any of the specified MySQL hosts

Solution:

1. Check database is running: docker-compose ps nocr-text-matcher-db
2. Enable debug mode to see masked connection string
3. Verify hostname in connection string:
   • Docker Compose: nocr-text-matcher-db:3306
   • Local dev: localhost:3316
4. Wait for database healthcheck to pass (may take 10-30 seconds on first start)

---

Problem: Configuration values not applied

Symptoms:

• Settings in appsettings.Development.json are ignored
• Service uses wrong configuration

Solution:

1. Check ASPNETCORE_ENVIRONMENT is set correctly:
   • VS Code: Development
   • Docker Compose: DockerCompose
   • K8s: Production
2. Enable debug mode to see which values are loaded
3. Check for environment variables overriding your settings
4. Remember: Environment Variables always win!

---

Problem: Secrets exposed in logs

Solution:

• Debug mode automatically masks sensitive values
• Never log configuration in production without masking
• Review debug output format in Startup.cs:
  • Passwords: amqp://user:***@host
  • Secrets: ab...yz (first 2 + last 2 chars only)

---

Problem: Missing .nocr.env file

Symptoms:

docker-compose up fails with missing environment variables

Solution:

1. Copy example file: cp .nocr.env.example .nocr.env
2. Fill in your actual values
3. Make sure .nocr.env is in the project root (same level as docker-compose.yml)

---

Quick Reference

Environment Variable Naming Convention

ASP.NET Core uses double underscores __ to represent nested JSON structure:

# JSON: { "Section": { "Key": "Value" } }
# Environment Variable:
Section__Key=Value

# Example:
WTelegramClientOptions__ApiId=12345
RebusRabbitMqOptions__ConnectionString=amqp://localhost

File Locations

flea/                                   # Project root
├── .nocr.env                          # Your secrets (gitignored)
├── .nocr.env.example                  # Template (committed)
├── docker-compose.yml                 # References .nocr.env
│
├── telegram-listener/
│   └── src/Nocr.TelegramListener.Host/
│       ├── appsettings.json                        # Base (committed)
│       ├── appsettings.Development.json            # Local dev (gitignored)
│       ├── appsettings.Development.json.example    # Template (committed)
│       ├── appsettings.DockerCompose.json          # Docker (gitignored)
│       └── appsettings.DockerCompose.json.example  # Template (committed)
│
└── [same structure for text-matcher, users, telegram-client]

---

Security Best Practices

1. ✅ Never commit secrets - Use .example files as templates
2. ✅ Use environment variables for sensitive data in production
3. ✅ Use K8s Secrets for Kubernetes deployments
4. ✅ Enable debug mode only when troubleshooting
5. ✅ Rotate credentials regularly
6. ✅ Use User Secrets for local development (optional)
7. ❌ Don't commit .nocr.env or appsettings.*.json (except .example files)
8. ❌ Don't use appsettings.protected.json (removed from code)

---

Additional Resources

• ASP.NET Core Configuration Docs
• Docker Compose Environment Variables
• Kubernetes Secrets
• Telegram API Documentation
• Telegram Bot API