Compare commits

...

10 Commits
0.7.41 ... main

Author SHA1 Message Date
ruberoid
77d1585ed5 docs: Update CLAUDE.md with prepare-build.sh requirement
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-10-28 00:33:25 +04:00
ruberoid
e03eae219d chore: Update submodule references after Dockerfile fix
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-28 00:33:07 +04:00
ruberoid
35bb116bfd fix: Remove additional_contexts and revert Kaniko changes
- Removed additional_contexts from docker-compose.yml (not supported by Kaniko)
- Reverted Kaniko image from v1.23.2 back to :debug
- Removed --build-context flags (not supported)
- Added prepare-build.sh script to copy nuget.config before builds
2025-10-28 00:30:17 +04:00
ruberoid
012d81bd66 fix: Update Kaniko to v1.23.2 to support --build-context flag
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
The previous fix added --build-context flag but the old Kaniko version
(executor:debug) doesn't support it. Updated to v1.23.2-debug which
includes support for --build-context flag (added in v1.9.0).
2025-10-28 00:16:39 +04:00
ruberoid
5531a42ed7 fix: Add --build-context=rootconfig flag to Kaniko builds
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
The recent docker-compose.yml change added additional_contexts with 'rootconfig'
reference. Dockerfiles now use 'COPY --from=rootconfig nuget.config' which works
with Docker Compose but breaks Kaniko builds.

Added --build-context=rootconfig=. flag to all Kaniko executor commands to provide
the named build context that Dockerfiles expect. This matches Docker Compose
behavior and fixes the 'rootconfig' image pull error.
2025-10-28 00:04:59 +04:00
ruberoid
36e574ed3a Merge branch 'main' of https://gitea.musk.fun/nocr/flea
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-27 23:57:09 +04:00
ruberoid
7172fe3659 fixed nuget.config copy to additional dockerfile context.
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2025-10-27 23:53:13 +04:00
ruberoid
9a7de46061 Новая модель, новые каноны контекста!
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-27 23:09:02 +04:00
ruberoid
97798df417 docs: Add detailed CI/CD workflow examples to CLAUDE.md
All checks were successful
continuous-integration/drone/push Build is passing
- Add 'How to trigger' sections with exact git commands for each pipeline
- Include practical examples: feature testing, contract publishing, releases
- Add 'Common CI/CD Workflows' section with real-world scenarios
- Document emergency rollback procedure
- Clarify contract-only service names (telegram_listener vs telegram-listener)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 12:44:23 +04:00
ruberoid
029319e7c1 docs: Add NuGet Package Management section to CLAUDE.md
- Document Central Package Management (CPM) with Package Source Mapping
- Explain package source configuration (nuget.org vs musk private feed)
- Describe how nuget.config is used across different environments
- Add troubleshooting context for NU1507 warning resolution

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 12:40:43 +04:00
6 changed files with 415 additions and 42 deletions

433
CLAUDE.md
View File

@ -27,28 +27,39 @@ The services follow a message bus pattern using RabbitMQ for async communication
**Note:** WTelegram sends both `UpdateNewChannelMessage` and `UpdateEditChannelMessage` for the same message. TelegramListener publishes separate events to avoid duplicate notifications downstream.
Each service follows Clean Architecture with separate projects for:
- Host (API/entry point)
- AppServices (business logic)
- Core (shared utilities)
- Api.Contracts (REST API contracts)
- Async.Api.Contracts (event contracts)
- Persistence (database layer, where applicable)
- Migrator (database migrations, where applicable)
- **Host** (API/entry point) - ASP.NET Core application, controllers, startup configuration
- **AppServices** (business logic) - Use cases, business logic, command/query handlers
- **Core** (shared utilities) - Domain entities, interfaces, shared logic
- **Api.Contracts** (REST API contracts) - DTOs, request/response models for REST endpoints (published as NuGet)
- **Async.Api.Contracts** (event contracts) - Event DTOs for RabbitMQ messaging (published as NuGet)
- **Persistence** (database layer) - EF Core DbContext, repositories, data access (text-matcher, users only)
- **Migrator** (database migrations) - EF Core migrations, migration scripts (text-matcher, users only)
**Project Structure Examples:**
- telegram-listener: Host, AppServices, Core, Async.Api.Contracts (no database)
- telegram-client: Host, AppServices, Core (no database, no published contracts)
- text-matcher: Host, AppServices, Core, Api.Contracts, Async.Api.Contracts, Persistence, Migrator
- users: Host, AppServices, Core, Api.Contracts, Persistence, Migrator
## Development Commands
### Running the System
```bash
# IMPORTANT: Before building with Docker Compose, prepare the build environment
./prepare-build.sh
# Start all services with Docker Compose
docker-compose up
# Start individual services for development
# Start individual services for development (no preparation needed)
cd telegram-client && dotnet run --project src/Nocr.TelegramClient.Host
cd telegram-listener && dotnet run --project src/Nocr.TelegramListener.Host
cd telegram-listener && dotnet run --project src/Nocr.TelegramListener.Host
cd text-matcher && dotnet run --project src/Nocr.TextMatcher.Host
cd users && dotnet run --project src/Nocr.Users.Host
```
**Note:** The `prepare-build.sh` script copies `nuget.config` to all submodule roots, which is required for Docker builds. This step is automatic in CI/CD but must be run manually for local Docker Compose builds.
### Building
```bash
# Build individual services
@ -60,8 +71,17 @@ dotnet build src/Nocr.<ServiceName>.Host
### Testing
```bash
# Run unit tests (only text-matcher has tests currently)
# Run all tests in text-matcher
cd text-matcher && dotnet test
# Run tests for a specific project
cd text-matcher && dotnet test tests/Nocr.TextMatcher.AppServices.UnitTests/Nocr.TextMatcher.AppServices.UnitTests.csproj
# Run tests with verbose output
cd text-matcher && dotnet test --verbosity detailed
# Run tests with code coverage (if configured)
cd text-matcher && dotnet test --collect:"XPlat Code Coverage"
```
### Database Migrations
@ -75,6 +95,29 @@ cd users && ./src/Nocr.Users.Migrator/AddMigration.sh MyMigrationName
# Apply migrations (handled automatically by migrator containers in docker-compose)
```
### Working with Git Submodules
```bash
# Initialize submodules after cloning the repository
git submodule update --init --recursive
# Update all submodules to their latest commits on main
./update-submodules.sh
# Commit and push changes to all submodules at once
./commit-all.sh "Your commit message"
# Update a single submodule
cd telegram-listener
git pull origin main
cd ..
git add telegram-listener
git commit -m "Update telegram-listener submodule"
# Check status of all submodules
git submodule status
```
## Configuration
**📖 See [CONFIGURATION.md](CONFIGURATION.md) for detailed configuration guide.**
@ -133,13 +176,47 @@ When running with docker-compose:
## Key Technologies
- .NET 8
- ASP.NET Core Web APIs
- Entity Framework Core with MariaDB
- WTelegramClient for Telegram API
- Rebus for message bus (RabbitMQ)
- Docker & Docker Compose for containerization
- Drone CI/CD on Kubernetes
- **.NET 8** - All services built on .NET 8
- **ASP.NET Core Web APIs** - REST endpoints and hosting
- **Entity Framework Core with MariaDB** - ORM for text-matcher and users databases
- **WTelegramClient** - MTProto API client for telegram-listener
- **Telegram.Bot** - Bot API client for telegram-client
- **Rebus** - Message bus abstraction over RabbitMQ
- **RabbitMQ** - Message broker for async event-driven communication
- **Docker & Docker Compose** - Local development and containerization
- **Kaniko** - Container image building in CI/CD
- **Drone CI** - CI/CD pipeline on Kubernetes
## NuGet Package Management
The project uses **Central Package Management (CPM)** with **Package Source Mapping** to manage NuGet dependencies:
### Package Sources
- **nuget.org**: All public packages (Microsoft.*, Serilog.*, etc.)
- **musk** (private): Internal `Nocr.*` contract packages
### Configuration
The `nuget.config` file in the project root defines package source mapping:
```xml
<packageSourceMapping>
<packageSource key="musk">
<package pattern="Nocr.*" />
</packageSource>
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
</packageSourceMapping>
```
### How It Works
1. **Local Development**: Copy `nuget.config` to each submodule root when needed
2. **CI/CD**: Drone automatically copies `nuget.config` to `/root/.nuget/NuGet/NuGet.Config`
3. **Docker Builds**: Kaniko copies `nuget.config` to submodule root before building
This eliminates NuGet warning NU1507 and ensures consistent package resolution across all environments.
## CI/CD Pipeline
@ -148,39 +225,240 @@ When running with docker-compose:
The project uses Drone CI with 5 specialized pipelines:
### 1. Feature Validation (`feature/*`, `fix/*` branches)
- Triggered on push to feature/fix branches
**Purpose**: Test feature branches before merging to main
**How to trigger**:
```bash
# Create feature branch and push
git checkout -b feature/my-new-feature
# ... make changes ...
git add .
git commit -m "Add new feature"
git push origin feature/my-new-feature
```
**What happens**:
- Runs build + tests with Testcontainers support
- Provides fast feedback before merge
- Duration: ~3-5 minutes
### 2. Main Validation (`main` branch)
- Triggered on push to main
**Purpose**: Ensure main branch stays healthy after merge
**How to trigger**:
```bash
# Merge feature to main
git checkout main
git merge feature/my-new-feature
git push origin main
```
**What happens**:
- Same checks as feature validation
- Ensures main branch stays healthy
- Ensures main branch builds and tests pass
- Duration: ~3-5 minutes
### 3. Contracts-Only Publish (tags with `contracts_only:`)
- Triggered by tag + commit message containing `contracts_only:<service>`
- Publishes NuGet contracts without building Docker images
- Fast iteration on contract changes
### 3. Contracts-Only Publish
**Purpose**: Publish NuGet contracts without building Docker images (fast iteration)
**How to trigger**:
```bash
# Update contracts in a submodule
cd telegram-listener
# ... update Async.Api.Contracts ...
git add . && git commit -m "Add MessageEdited event"
git push
# From parent repo, tag with contracts_only marker
cd ..
git add telegram-listener
git commit -m "contracts_only:telegram_listener - Add MessageEdited event"
git tag 0.7.41
git push && git push --tags
```
**What happens**:
- Publishes only the specified service's NuGet contracts
- Skips Docker image builds
- Duration: ~2 minutes
- Example: `git commit -m "contracts_only:telegram_listener - Add MessageEdited event"`
### 4. Full Release (clean tags)
- Triggered by tag without special markers
- Complete release cycle:
1. Publish all NuGet contracts (parallel)
2. Build all Docker images with Kaniko (parallel, after contracts)
3. Deploy to Kubernetes (automatic for `v*` tags)
**Services**: `telegram_listener`, `text_matcher`, `users`
### 4. Full Release
**Purpose**: Complete release - contracts + images + deployment
**How to trigger**:
```bash
# Make your changes, commit to main
git add .
git commit -m "Implement new feature"
git push
# Tag for full release (no special markers in commit message)
git tag v1.3.0
git push --tags
```
**What happens**:
1. Publishes all NuGet contracts (parallel)
2. Builds all Docker images with Kaniko (parallel, after contracts)
3. Deploys to Kubernetes (automatic for `v*` tags)
- Duration: ~8-10 minutes
- Example: `git tag v1.3.0`
### 5. Deploy-Only (tags with `deploy_only:`)
- Triggered by tag + commit message containing `deploy_only:`
- Skips building, only deploys existing images
- Useful for rollbacks and hotfixes
### 5. Deploy-Only
**Purpose**: Deploy existing images without rebuilding (rollbacks, hotfixes)
**How to trigger**:
```bash
# Deploy already-built images (e.g., rollback to previous version)
git commit --allow-empty -m "deploy_only: Rollback to v1.2.9"
git tag deploy-v1.2.9
git push && git push --tags
# Or deploy latest images
git commit --allow-empty -m "deploy_only: Deploy latest"
git tag deploy-latest
git push && git push --tags
```
**What happens**:
- Skips building entirely
- Deploys specified tag's images (or `latest`)
- Duration: ~1 minute
- Example: `git commit --allow-empty -m "deploy_only: Deploy v1.2.9"`
## Development Workflows
### Making Changes to a Single Service
When working on a specific service (e.g., telegram-listener):
```bash
# 1. Navigate to the submodule
cd telegram-listener
# 2. Create a feature branch in the submodule
git checkout -b feature/new-feature
# 3. Make your changes to the code
# ... edit files ...
# 4. Test locally (if tests exist)
dotnet test
# 5. Commit and push in the submodule
git add .
git commit -m "Add new feature"
git push origin feature/new-feature
# 6. Return to parent repo and update submodule reference
cd ..
git add telegram-listener
git commit -m "Update telegram-listener: Add new feature"
git push origin main
```
### Updating Contract Packages
When you change event contracts (Async.Api.Contracts) or REST contracts (Api.Contracts):
```bash
# 1. Make changes to contracts in the submodule
cd text-matcher
# ... edit Nocr.TextMatcher.Async.Api.Contracts ...
git add . && git commit -m "Add new event contract"
git push origin main
# 2. Return to parent repo and publish contracts only
cd ..
git add text-matcher
git commit -m "contracts_only:text_matcher - Add new event contract"
git tag 0.8.6
git push && git push --tags
# 3. Other services can now reference the new contract version
# Update their .csproj or Directory.Packages.props to use version 0.8.6
```
### Working Across Multiple Services
When implementing a feature that spans multiple services:
```bash
# 1. Update each submodule in sequence
cd telegram-listener
git checkout -b feature/cross-service-feature
# ... make changes ...
git commit -m "Part 1: Update listener"
git push origin feature/cross-service-feature
cd ../text-matcher
git checkout -b feature/cross-service-feature
# ... make changes ...
git commit -m "Part 2: Update matcher"
git push origin feature/cross-service-feature
# 2. Update parent repo to reference all changes
cd ..
git add telegram-listener text-matcher
git commit -m "Implement cross-service feature"
git push origin main
# 3. Tag for full release
git tag v1.5.0
git push --tags
```
## Common CI/CD Workflows
### Test a Feature Branch
```bash
git checkout -b feature/add-logging
# ... make changes ...
git add . && git commit -m "Add structured logging"
git push origin feature/add-logging
# ✅ Drone runs feature-validation pipeline
```
### Publish Contracts After API Changes
```bash
cd text-matcher
# Update Nocr.TextMatcher.Async.Api.Contracts
git add . && git commit -m "Add UserDeleted event"
git push
cd ..
git add text-matcher
git commit -m "contracts_only:text_matcher - Add UserDeleted event"
git tag 0.8.5
git push && git push --tags
# ✅ Drone publishes text-matcher contracts to NuGet
```
### Full Release Workflow
```bash
# Work on main branch
git add .
git commit -m "Implement payment processing"
git push
# ✅ Drone runs main-validation
# Tag for release
git tag v2.0.0
git push --tags
# ✅ Drone runs full-release pipeline:
# 1. Publishes all contracts
# 2. Builds all Docker images
# 3. Deploys to Kubernetes
```
### Emergency Rollback
```bash
# Rollback to last known good version
git commit --allow-empty -m "deploy_only: Emergency rollback to v1.9.5"
git tag rollback-v1.9.5
git push && git push --tags
# ✅ Drone deploys v1.9.5 images immediately (~1 minute)
```
### Deployment Scripts
@ -195,4 +473,83 @@ Located in `_deploy/scripts/`:
- **Parallel execution** for independent operations
- **Proper dependency order**: Contracts → Images → Deploy
- **Kaniko layer caching** for faster Docker builds
- **Docker-in-Docker** support for Testcontainers in tests
- **Docker-in-Docker** support for Testcontainers in tests
## Troubleshooting
### Submodule Issues
**Problem:** Submodule directories are empty after cloning
```bash
# Solution: Initialize submodules
git submodule update --init --recursive
```
**Problem:** Submodule is in detached HEAD state
```bash
# Solution: Check out the main branch in the submodule
cd <submodule-name>
git checkout main
git pull origin main
cd ..
```
**Problem:** Changes in submodule not reflected in parent repo
```bash
# Solution: Update submodule reference in parent
cd .. # Return to parent repo
git add <submodule-name>
git commit -m "Update <submodule-name> reference"
```
### Build Issues
**Problem:** NuGet package restore fails with NU1507 warning
```bash
# Solution: Copy nuget.config to submodule root
cp nuget.config telegram-listener/
cd telegram-listener && dotnet restore
```
**Problem:** Cannot find Nocr.* contract packages
```bash
# Solution: Ensure you have access to the private NuGet feed
# Check nuget.config has the correct source configuration
# Verify credentials for the "musk" package source
```
### Docker Compose Issues
**Problem:** Service cannot connect to RabbitMQ
```bash
# Solution: Use the Docker network hostname, not localhost
# In docker-compose environment: nocr-rabbitmq:5672
# In local development: localhost:5672
```
**Problem:** Database connection fails
```bash
# Solution: Wait for database health checks to pass
# Check docker-compose logs for database container
docker-compose logs nocr-text-matcher-db
# Database takes 10-30 seconds to be ready on first start
```
### CI/CD Issues
**Problem:** Pipeline doesn't trigger on tag
```bash
# Solution: Ensure tag matches expected patterns
# Feature validation: feature/*, fix/*, issues/*
# Main validation: main branch
# Contracts: Tag with commit message containing "contracts_only:<service>"
# Full release: Any tag without special markers
# Deploy-only: Tag with commit message containing "deploy_only:"
```
**Problem:** Docker image build fails in Kaniko
```bash
# Solution: Check if nuget.config is correctly copied
# Verify _deploy/deploy.sh contains nuget.config copy step
# Check Drone logs for nuget.config presence in build context
```

16
prepare-build.sh Executable file
View File

@ -0,0 +1,16 @@
#!/bin/bash
# Prepares the build environment by copying nuget.config to all submodules
# This is required for both local docker-compose builds and CI/CD
set -e
echo "📦 Copying nuget.config to all submodules..."
for submodule in telegram-client telegram-listener text-matcher users; do
if [ -d "$submodule" ]; then
cp nuget.config "$submodule/"
echo "✓ Copied to $submodule/"
fi
done
echo "✅ Build preparation complete!"

@ -1 +1 @@
Subproject commit 904a58bc138242a6dff1b02b26536643583fe3e0
Subproject commit e8ee01cbe809867305ff02611155cddffccbb401

@ -1 +1 @@
Subproject commit 80ac8138c857123c658c2649f079a9cc756ed60f
Subproject commit 9ad504753550414edd96ef4194a3e0ee7e604e28

@ -1 +1 @@
Subproject commit 1a3ffada92b0521c4c4e7ae8364aab151c65c377
Subproject commit 1a55061e4b1b125d1b911f479fc830fb94cd93cc

2
users

@ -1 +1 @@
Subproject commit 4cb1a005116bbf9cb6dbff90c99ff21c98dedfd1
Subproject commit 30ae0c043ca2f8958223d6fb343b8b3e3561d727