Compare commits

...

149 Commits

Author SHA1 Message Date
ruberoid
77d1585ed5 docs: Update CLAUDE.md with prepare-build.sh requirement
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-10-28 00:33:25 +04:00
ruberoid
e03eae219d chore: Update submodule references after Dockerfile fix
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-28 00:33:07 +04:00
ruberoid
35bb116bfd fix: Remove additional_contexts and revert Kaniko changes
- Removed additional_contexts from docker-compose.yml (not supported by Kaniko)
- Reverted Kaniko image from v1.23.2 back to :debug
- Removed --build-context flags (not supported)
- Added prepare-build.sh script to copy nuget.config before builds
2025-10-28 00:30:17 +04:00
ruberoid
012d81bd66 fix: Update Kaniko to v1.23.2 to support --build-context flag
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
The previous fix added --build-context flag but the old Kaniko version
(executor:debug) doesn't support it. Updated to v1.23.2-debug which
includes support for --build-context flag (added in v1.9.0).
2025-10-28 00:16:39 +04:00
ruberoid
5531a42ed7 fix: Add --build-context=rootconfig flag to Kaniko builds
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
The recent docker-compose.yml change added additional_contexts with 'rootconfig'
reference. Dockerfiles now use 'COPY --from=rootconfig nuget.config' which works
with Docker Compose but breaks Kaniko builds.

Added --build-context=rootconfig=. flag to all Kaniko executor commands to provide
the named build context that Dockerfiles expect. This matches Docker Compose
behavior and fixes the 'rootconfig' image pull error.
2025-10-28 00:04:59 +04:00
ruberoid
36e574ed3a Merge branch 'main' of https://gitea.musk.fun/nocr/flea
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-27 23:57:09 +04:00
ruberoid
7172fe3659 fixed nuget.config copy to additional dockerfile context.
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2025-10-27 23:53:13 +04:00
ruberoid
9a7de46061 Новая модель, новые каноны контекста!
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-27 23:09:02 +04:00
ruberoid
97798df417 docs: Add detailed CI/CD workflow examples to CLAUDE.md
All checks were successful
continuous-integration/drone/push Build is passing
- Add 'How to trigger' sections with exact git commands for each pipeline
- Include practical examples: feature testing, contract publishing, releases
- Add 'Common CI/CD Workflows' section with real-world scenarios
- Document emergency rollback procedure
- Clarify contract-only service names (telegram_listener vs telegram-listener)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 12:44:23 +04:00
ruberoid
029319e7c1 docs: Add NuGet Package Management section to CLAUDE.md
- Document Central Package Management (CPM) with Package Source Mapping
- Explain package source configuration (nuget.org vs musk private feed)
- Describe how nuget.config is used across different environments
- Add troubleshooting context for NU1507 warning resolution

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 12:40:43 +04:00
ruberoid
b8ba3df310 copy nuget.config while deploy to every submodule.
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-10-17 12:26:39 +04:00
ruberoid
e196a79174 Removed echoed package sourcing to nuget.config file attached.
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2025-10-17 12:11:58 +04:00
ruberoid
5407479361 deploy_only: Deploy latest
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-17 11:05:06 +04:00
ruberoid
af5d46e4e8 fixed for only latest usages to deploy. 2025-10-17 11:04:58 +04:00
ruberoid
718a4ff1c9 deploy_only: Deploy latest
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-17 11:02:06 +04:00
ruberoid
5e6d3e968d fixed for only latest usages to deploy. 2025-10-17 11:01:57 +04:00
ruberoid
82ade98af3 deploy_only: 0.7.35
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-17 10:59:05 +04:00
ruberoid
1be50c3eeb deploy_only: 0.7.35
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-17 10:56:40 +04:00
ruberoid
8a3c6b50e3 deploy_only: fixed deploy to k8s. added secret kube.
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build was killed
2025-10-17 10:49:47 +04:00
ruberoid
ce5540185b Returned back dockerfiles source as echo commands.
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is failing
2025-10-16 23:35:48 +04:00
ruberoid
47aaf30de1 Returned back curl to app images.
Some checks reported errors
continuous-integration/drone/push Build was killed
continuous-integration/drone/tag Build is failing
2025-10-16 23:12:22 +04:00
ruberoid
4ec038887c Optimized Dockerfiles in submodules. 2025-10-16 23:04:49 +04:00
ruberoid
b8a3814deb Made full release sequental vs parallel.
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build was killed
2025-10-16 22:50:54 +04:00
ruberoid
6ddf2ff0b5 Modified .dockerignore files for more optimizations 2025-10-16 22:47:48 +04:00
ruberoid
182178ebe1 removed branch main for trigger pipeline MAIN.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-16 22:13:24 +04:00
ruberoid
15519f2377 Обновил документацию. 2025-10-16 22:11:06 +04:00
ruberoid
32b0253981 drone fix 2. validated through python3 yaml.
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-10-16 21:56:40 +04:00
ruberoid
e9d2bcdb7a drone fix
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 21:53:11 +04:00
ruberoid
a9c846d793 fix 9
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 18:17:26 +04:00
ruberoid
59b5226d47 fix 8
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-16 18:12:45 +04:00
ruberoid
e0c7f8f38a fix drone 6
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 18:09:37 +04:00
ruberoid
7b3571cd5b .drone 4 fix
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 18:08:47 +04:00
ruberoid
f6fe2eb54e three more fix drone
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-16 18:07:27 +04:00
ruberoid
45c78ed008 more fix drone
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 18:07:00 +04:00
ruberoid
a2d01c84a9 merged .drone
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 18:06:12 +04:00
ruberoid
223e58c8d2 4/5 cicd build stages. 2025-10-16 18:00:23 +04:00
604ac1f507 issues/7 (#8)
Co-authored-by: ruberoid <dcharushnikov@gmail.com>
Reviewed-on: #8
2025-10-16 13:58:17 +00:00
ruberoid
cc7a5d3285 Just first build cicd left temporaryly.
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-16 17:56:07 +04:00
ruberoid
18d8f59673 Доабвил скрипт автокоммита субмодулей. #7
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 17:52:31 +04:00
ruberoid
a280ffc9db release moregit add .drone.yml 2025-10-16 17:48:53 +04:00
ruberoid
89aa4a8d28 releasegit add .drone.yml
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-16 17:47:19 +04:00
ruberoid
169acd2181 release
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-16 17:38:25 +04:00
ruberoid
1ed92a4ab9 contracts_only:telegram_listener contracts_only:users contracts_only:text_matcher
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-16 17:31:23 +04:00
ruberoid
257d3a95ed contracts_only:telegram_listener contracts_only:users contracts_only:text_matcher
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-16 17:26:54 +04:00
ruberoid
e262ac7a27 tagged block added.
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2025-10-16 17:20:23 +04:00
ruberoid
7103e4dc6d added 2nd cicd main block.
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-16 17:18:17 +04:00
ruberoid
289ddaaea7 main commit fix. 2025-10-16 17:17:24 +04:00
b2a4b8112d Merge pull request 'issues/5' (#6) from issues/5 into main
Reviewed-on: #6
2025-10-16 13:14:22 +00:00
ruberoid
793266b816 fix deps
All checks were successful
continuous-integration/drone/push Build is passing
2025-10-16 16:54:32 +04:00
ruberoid
0c467239b9 fix deps
Some checks failed
continuous-integration/drone/push Build is failing
2025-10-16 16:46:31 +04:00
ruberoid
0f9ee08779 build fix.
Some checks failed
continuous-integration/drone/push Build is failing
2025-10-16 16:09:51 +04:00
ruberoid
b0882f62d2 Fixed build
Some checks failed
continuous-integration/drone/push Build is failing
2025-10-16 15:59:16 +04:00
ruberoid
af9d023b20 cicd fix.
Some checks failed
continuous-integration/drone/push Build is failing
2025-10-16 15:49:07 +04:00
ruberoid
3f0f4d2c05 cicd fix.
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 15:46:51 +04:00
ruberoid
f274646e1e cicd fix.
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 15:45:34 +04:00
ruberoid
c1a043c6eb Попробую протестировать первый сборочный блок.
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2025-10-16 15:43:57 +04:00
ruberoid
40301b263e contracts_only:text_matcher. - Теперь надо триггернуть text-matcher.
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-16 15:19:17 +04:00
ruberoid
8d574f0c89 contracts_only:telegram_listener. - Переписал CI таким образом, чтобы триггерились отдельные этапы.
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-16 14:37:50 +04:00
ruberoid
aabf3546b4 Изменил контракт listener с учетом того, что теперь публикуется новый отдельный тип обновленного сообещния. 2025-10-16 14:30:18 +04:00
ruberoid
90431f7c72 parallel kaniko build.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-15 10:17:49 +04:00
ruberoid
5f711f4229 rewrite to total kaniko build.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-10-14 20:31:44 +04:00
ruberoid
d43885fb63 Kaniko
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-14 19:15:41 +04:00
ruberoid
cdd31d3c5b Test: replace plugins/docker with docker:dind for telegram-listener
Some checks failed
continuous-integration/drone/tag Build is failing
2025-10-14 19:04:56 +04:00
ruberoid
7b7a19d212 returned back Nuget as dockerfile. 2025-10-14 18:46:32 +04:00
ruberoid
d7d551d66c no cache build for all stages.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-14 17:28:02 +04:00
ruberoid
da53ef5727 Added some debug info while dry run.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-14 17:17:32 +04:00
ruberoid
4938092b5c Simplify all Dockerfiles: use dotnet nuget API instead of manual config
Some checks reported errors
continuous-integration/drone/tag Build was killed
Updated all service submodules with simplified NuGet configuration:
- Replaced 20+ lines of echo commands with single API call
- No API key needed for public package read access
- Verified gitea.musk.fun/api/packages/nocr is publicly accessible
- Much cleaner and more maintainable Dockerfiles

This should resolve the hanging dotnet restore issue in Drone CI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 15:00:41 +04:00
ruberoid
4bd1572f53 Add diagnostics to Drone CI and fix Dockerfiles in all services
Some checks reported errors
continuous-integration/drone/tag Build was killed
- Added debug-network step to diagnose DNS, SSL, and connectivity issues
- Added debug-dotnet-restore step to test NuGet restore in isolation
- Updated all service submodules with fixed Dockerfiles:
  * CA certificates now update before any network operations
  * Added verbose logging to dotnet restore
  * Optimized apt-get with --no-install-recommends and cache cleanup

This should resolve the NU1301 SSL certificate errors in Drone CI builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 14:46:01 +04:00
ruberoid
8edcce23ae fixed ca update for docker images,
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-14 14:27:06 +04:00
ruberoid
89507cc87b Разобрались с минимальной версией пакета для обновленного API матчера.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-10-14 13:52:03 +04:00
ruberoid
31a4a06bec Update CLAUDE.md with submodule structure and message flow
- Documented git submodule architecture
- Explained Drone CI/CD and tag-based versioning
- Updated message flow to include TextSubscriptionUpdated events
- Clarified text-matcher's role in storing match history

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-14 13:24:35 +04:00
ruberoid
c84dfe56e9 i dunno 2025-07-31 11:23:10 +04:00
Ruberoid
51b632637b enriched and shortened telegram-listener logs. 2025-07-22 19:40:21 +03:00
Ruberoid
e51c8562dc added 403 common bot response handling and k8s metrics.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-22 10:53:46 +03:00
Ruberoid
aaaebf1f34 Badge fix. 2025-07-22 00:26:36 +03:00
Ruberoid
df09db6461 Log messages translated into english and some punctuation fixes.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-22 00:10:32 +03:00
Ruberoid
46720bbfe8 removed unused deps.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-21 19:33:26 +03:00
Ruberoid
4f96f38d3b removed nuget priority restore warnings.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-21 17:46:47 +03:00
Ruberoid
b28948b85e added changes. 2025-07-21 17:15:00 +03:00
Ruberoid
23ffce6465 fixed client module for start.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-21 14:59:40 +03:00
Ruberoid
22ad80cdf2 change all services exposed ports to prevent from crossing with the defaults ones. 2025-07-21 14:04:40 +03:00
Ruberoid
e7bf96f081 added healthchecks & changed dev dockercompose mariadb instances ports. 2025-07-21 13:51:14 +03:00
Ruberoid
37cca43282 added claude generated readme. 2025-07-21 11:13:45 +03:00
Ruberoid
9f9f7dfbcc added to config telegram notify channels as a secret. 2025-07-19 16:04:54 +03:00
Ruberoid aka tverpen
640172d8ac fixed deployment. 2025-07-19 12:32:19 +00:00
Ruberoid
343109a727 Fixed deployment for telegram-client service. 2025-07-19 15:23:28 +03:00
Ruberoid
30587eddd0 fixed migrator for text-matcher.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-19 15:16:30 +03:00
Ruberoid
5b1dee0ab3 return to drone/docker #05.
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-19 14:20:44 +03:00
Ruberoid
35a977efd9 return to drone/docker #04.
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-19 14:08:21 +03:00
Ruberoid
fcf6f26f40 return to drone/docker #03.
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-19 13:54:06 +03:00
Ruberoid
f7d8ad0fd2 return to drone/docker #02.
Some checks reported errors
continuous-integration/drone/push Build was killed
2025-07-19 13:46:21 +03:00
Ruberoid
3d85488716 return to drone/docker #01. 2025-07-19 13:45:17 +03:00
Ruberoid
b56bf27460 return to drone/docker #00. 2025-07-19 13:44:44 +03:00
Ruberoid
405e0aec57 removed main branch push immediate build
All checks were successful
continuous-integration/drone/tag Build is passing
2025-07-18 20:02:25 +03:00
Ruberoid
11ad2653fd Fixed all warnings.
Some checks failed
continuous-integration/drone/push Build is failing
2025-07-18 19:58:19 +03:00
Ruberoid
1bdf5efb74 fixed all configs,
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-18 17:23:48 +03:00
Ruberoid
61a70f5b7f added docker-compose info into readme. 2025-07-18 15:03:34 +03:00
Ruberoid
adc7bb95d2 added docker compose env variables file template. 2025-07-18 15:00:10 +03:00
Ruberoid
4d067b1188 Slight telegram client improvements.
Some checks reported errors
continuous-integration/drone/push Build was killed
continuous-integration/drone/tag Build is passing
2025-07-15 00:27:23 +03:00
Ruberoid
b6c2668254 Added correct env var for array initialization.
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-14 20:55:31 +03:00
Ruberoid
6ac42fbdd3 added parameter for nabling admin message notification.
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-14 17:00:21 +03:00
Ruberoid
7531dd93df added env variables for correct services start.
All checks were successful
continuous-integration/drone/push Build is passing
2025-07-14 16:01:29 +03:00
Ruberoid
79322cc9dc Merge branch 'main' of https://gitea.musk.fun/nocr/flea 2025-07-14 14:46:53 +03:00
Ruberoid
5842743219 added envs ignore for docker compose. 2025-07-14 14:45:45 +03:00
ruberoid
373ae695dd finally read drone man, but claude sux.
Some checks reported errors
continuous-integration/drone/tag Build was killed
continuous-integration/drone/push Build is passing
2025-06-03 22:43:54 +04:00
ruberoid
afb39f4f59 fixed yaml/ 2025-06-03 22:37:45 +04:00
ruberoid
3af947eddf Added some sensitive readme data. 2025-06-03 22:30:55 +04:00
ruberoid
62325d3378 added semantic ref for trigger to build. 2025-06-03 22:22:04 +04:00
ruberoid
676e6dc51d restricted build only for tags + branch main 2025-06-03 22:00:52 +04:00
ruberoid
c5becb802b fixed build and push for the text_matcher|users conflict naming.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-06-03 21:20:20 +04:00
ruberoid
414f804380 fix time limit compare issue.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-14 09:59:27 +04:00
ruberoid
50edb867df new approach 00.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-13 21:23:32 +04:00
ruberoid
e0e5f84bda TL update
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-13 20:49:26 +04:00
ruberoid
046c242529 Changed telegram listener to update manager.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-13 18:36:24 +04:00
ruberoid
54bd0c9426 Actual version of WTelegram listener 3.7.1 -> 4.3.4
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-11 18:04:53 +04:00
ruberoid
92a36e6005 added correct badge. 2025-05-11 16:17:02 +04:00
ruberoid
516f27d14a trigger for build changed.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2025-05-11 15:09:37 +04:00
ruberoid
3cb4428d14 added debug step and printenv. 2025-05-11 15:07:31 +04:00
ruberoid
fa279f6c56 added liveness and rediness probes for telegram-listener.
Some checks failed
continuous-integration/drone/push Build is failing
2025-05-11 14:05:44 +04:00
ruberoid
b70b947095 slight debug changes for listener.
Some checks failed
continuous-integration/drone/push Build is failing
2025-05-11 13:58:35 +04:00
ruberoid
0d5ee88669 return back to dron kaniko plugin.
Some checks reported errors
continuous-integration/drone/push Build is passing
continuous-integration/drone Build was killed
2024-12-15 13:54:11 +04:00
ruberoid
da483cc10c fix 01
Some checks failed
continuous-integration/drone/push Build is failing
2024-12-15 13:48:16 +04:00
ruberoid
1387cc107b added banzai plugin for drone kaniko
Some checks failed
continuous-integration/drone/push Build is failing
2024-12-15 13:46:24 +04:00
ruberoid
71af26aee5 Graph build disabled.
All checks were successful
continuous-integration/drone Build is passing
2024-12-06 15:03:51 +00:00
ruberoid
ab181a2c58 Update .drone.yml 2024-12-06 14:32:46 +00:00
ruberoid
6cd56139a0 Update .drone.yml 2024-12-06 13:37:42 +00:00
ruberoid
17e96163a9 New day new hopes. 2024-12-06 11:08:33 +00:00
ruberoid
aad947c0c4 Update .drone.yml 2024-12-05 17:54:45 +00:00
ruberoid
383e1d1841 Update .drone.yml 2024-12-05 17:53:37 +00:00
ruberoid
5dbca76b02 trying /kaniko directory for the mount path of nfs storage 2024-12-05 17:48:57 +00:00
ruberoid
6097c30428 Update .drone.yml 2024-12-05 17:30:49 +00:00
ruberoid
4419672373 Update .drone.yml 2024-12-05 16:52:58 +00:00
ruberoid
46d36c36e2 Update .drone.yml 2024-12-05 16:26:37 +00:00
ruberoid
596dc94f35 Update .drone.yml 2024-12-05 16:23:54 +00:00
Ruberoid (ghostcat)
b259675378 fixes 2024-12-05 20:19:47 +04:00
Ruberoid (ghostcat)
d0a3e0b7b0 test 001. kaniko build. 2024-12-05 20:12:04 +04:00
Ruberoid (ghostcat)
fe8042546f creds fix 2024-12-05 16:53:37 +04:00
Ruberoid (ghostcat)
6f91076772 temporary disabled ci steps graph. 2024-12-05 16:31:58 +04:00
Ruberoid
0c537b1324 Fixed drone ci/cd parallelism. 2024-12-05 15:32:13 +04:00
Ruberoid
28cab7a378 Fixed drone ci/cd parallelism. 2024-12-05 15:30:28 +04:00
Ruberoid
dea08d76f5 Added drone ci/cd parallelism. 2024-12-05 14:59:54 +04:00
Ruberoid
4d014435e8 Made cache volume whole ci/cd wide. Also changed mounted directories. 2024-12-05 14:23:21 +04:00
Ruberoid
41f871ecf2 added cache volumes for docker and pure image builds. Also removed temporary trigger for the build. 2024-12-05 13:56:03 +04:00
ruberoid
82d98ae358 Update README.md 2024-12-05 08:45:33 +00:00
Ruberoid (ghostcat)
7f1a63fbd2 Fixed all project dependencies and reset to 0.0.1 version. 2024-12-05 00:53:26 +04:00
Ruberoid (ghostcat)
59391fa6b1 light improvements for drone cicd.
Some checks reported errors
continuous-integration/drone/tag Build was killed
2024-12-04 17:45:53 +04:00
Ruberoid (ghostcat)
5209a2dbfa changed readme. 2024-12-03 20:47:20 +04:00
Ruberoid (ghostcat)
c514175d8d fixed badge status. 2024-12-03 20:45:40 +04:00
Ruberoid
1e0e6633e0 submodules changes for correct ci 2024-08-04 11:53:49 +03:00
21 changed files with 3586 additions and 224 deletions

View File

@ -1,139 +1,663 @@
---
kind: pipeline
type: kubernetes
name: Nocr
clone:
disable: true
steps:
- name: clone
image: alpine/git
settings:
tags: true
commands:
- pwd
- git clone https://gitea.musk.fun/nocr/local-env
- cd local-env
- git submodule update --init --recursive
- name: nocr telegram listener build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/telegram_listener
registry: hub.musk.fun
context: ./local-env/telegram-listener
dockerfile: ./local-env/telegram-listener/src/Nocr.TelegramListener.Host/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
- name: nocr telegram listener nuget package make & push
image: mcr.microsoft.com/dotnet/sdk:8.0
environment:
VERSION: ${DRONE_TAG}
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- dotnet nuget add source --name musk https://gitea.musk.fun/api/packages/nocr/nuget/index.json
- dotnet pack ./local-env/telegram-listener/Nocr.TelegramListener.sln -o ./local-env/telegram-listener/bin
- dotnet nuget push ./local-env/telegram-listener/bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
- name: nocr text matcher build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/text_matcher
registry: hub.musk.fun
context: ./local-env/text-matcher
dockerfile: ./local-env/text-matcher/src/Nocr.TextMatcher.Host/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
- name: nocr text matcher nuget package make & push
image: mcr.microsoft.com/dotnet/sdk:8.0
environment:
VERSION: ${DRONE_TAG}
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- dotnet nuget add source --name musk https://gitea.musk.fun/api/packages/nocr/nuget/index.json
- dotnet pack ./local-env/text-matcher/Nocr.TextMatcher.sln -o ./local-env/text-matcher/bin
- dotnet nuget push ./local-env/text-matcher/bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
- name: nocr users build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/users
registry: hub.musk.fun
context: ./local-env/users
dockerfile: ./local-env/users/src/Nocr.Users.Host/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
- name: nocr users nuget package make & push
image: mcr.microsoft.com/dotnet/sdk:8.0
environment:
VERSION: ${DRONE_TAG}
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- dotnet nuget add source --name musk https://gitea.musk.fun/api/packages/nocr/nuget/index.json
- dotnet pack ./local-env/users/Nocr.Users.sln -o ./local-env/users/bin
- dotnet nuget push ./local-env/users/bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
- name: nocr telegram client build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/telegram_client
registry: hub.musk.fun
context: ./local-env/telegram-client
dockerfile: ./local-env/telegram-client/src/Nocr.TelegramClient.Host/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
- name: nocr text matcher migrator build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/text_matcher_migrator
registry: hub.musk.fun
context: ./local-env/text-matcher/
dockerfile: ./local-env/text-matcher/src/Nocr.TextMatcher.Migrator/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
- name: nocr users migrator build & push
image: plugins/docker
privileged: true
settings:
repo: hub.musk.fun/k8s/nocr/users_migrator
registry: hub.musk.fun
context: ./local-env/users/
dockerfile: ./local-env/users/src/Nocr.Users.Migrator/Dockerfile
tags: ["${DRONE_COMMIT_SHA:0:7}", "latest"]
username:
from_secret: hub_username
password:
from_secret: hub_password
trigger:
event:
- tag
---
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 🧪 Pipeline 1: Feature Branch Validation
# Trigger: Push to feature/* or fix/* branches
# Purpose: Fast feedback for developers - build and test only
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
kind: pipeline
type: kubernetes
name: feature-validation
metadata:
namespace: musk-drone
trigger:
ref:
- refs/heads/feature/*
- refs/heads/issues/*
- refs/heads/fix/*
event:
- push
clone:
disable: true
# Service container for Testcontainers
services:
- name: docker
image: docker:27-dind
privileged: true
volumes:
- name: dockersock
path: /var/run
volumes:
- name: dockersock
temp: {}
- name: nuget-cache
temp: {}
steps:
- name: clone
image: alpine/git
commands:
- git clone https://gitea.musk.fun/nocr/flea
- cd flea
- git checkout $DRONE_COMMIT
- git submodule update --init --recursive
- name: dotnet-restore
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
commands:
- cd flea
- mkdir -p /root/.nuget/NuGet
- cp nuget.config /root/.nuget/NuGet/NuGet.Config
- echo "🔄 Restoring all projects..."
- dotnet restore telegram-listener/Nocr.TelegramListener.sln
- dotnet restore telegram-client/Nocr.TelegramClient.sln
- dotnet restore text-matcher/Nocr.TextMatcher.sln
- dotnet restore users/Nocr.Users.sln
depends_on:
- clone
- name: dotnet-build
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
commands:
- cd flea
- echo "🔨 Building all projects..."
- dotnet build telegram-listener/Nocr.TelegramListener.sln --no-restore -c Debug
- dotnet build telegram-client/Nocr.TelegramClient.sln --no-restore -c Debug
- dotnet build text-matcher/Nocr.TextMatcher.sln --no-restore -c Debug
- dotnet build users/Nocr.Users.sln --no-restore -c Debug
depends_on:
- dotnet-restore
- name: dotnet-test
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
- name: dockersock
path: /var/run
environment:
DOCKER_HOST: unix:///var/run/docker.sock
commands:
- cd flea
- echo "🧪 Running tests..."
- dotnet test text-matcher/Nocr.TextMatcher.sln --no-build -c Debug --logger trx --results-directory ./test-results
- echo "✅ Tests completed"
depends_on:
- dotnet-build
- docker
---
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 📝 Pipeline 2: Main Branch Validation
# Trigger: Push to main branch
# Purpose: Validate main branch after merge
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
kind: pipeline
type: kubernetes
name: main-validation
metadata:
namespace: musk-drone
trigger:
ref:
- refs/heads/main
event:
- push
clone:
disable: true
services:
- name: docker
image: docker:27-dind
privileged: true
volumes:
- name: dockersock
path: /var/run
volumes:
- name: dockersock
temp: {}
- name: nuget-cache
temp: {}
steps:
- name: clone
image: alpine/git
commands:
- git clone https://gitea.musk.fun/nocr/flea
- cd flea
- git checkout $DRONE_COMMIT
- git submodule update --init --recursive
- name: dotnet-restore
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
commands:
- cd flea
- mkdir -p /root/.nuget/NuGet
- cp nuget.config /root/.nuget/NuGet/NuGet.Config
- dotnet restore telegram-listener/Nocr.TelegramListener.sln
- dotnet restore telegram-client/Nocr.TelegramClient.sln
- dotnet restore text-matcher/Nocr.TextMatcher.sln
- dotnet restore users/Nocr.Users.sln
depends_on:
- clone
- name: dotnet-build
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
commands:
- cd flea
- dotnet build telegram-listener/Nocr.TelegramListener.sln --no-restore -c Release
- dotnet build telegram-client/Nocr.TelegramClient.sln --no-restore -c Release
- dotnet build text-matcher/Nocr.TextMatcher.sln --no-restore -c Release
- dotnet build users/Nocr.Users.sln --no-restore -c Release
depends_on:
- dotnet-restore
- name: dotnet-test
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
- name: dockersock
path: /var/run
environment:
DOCKER_HOST: unix:///var/run/docker.sock
commands:
- cd flea
- dotnet test text-matcher/Nocr.TextMatcher.sln --no-build -c Release --logger trx --results-directory ./test-results
depends_on:
- dotnet-build
- docker
---
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 📦 Pipeline 3: Contracts-Only Publish
# Trigger: Tag with commit message containing "contracts_only:<service>"
# Purpose: Fast publish of contract packages without building images
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
kind: pipeline
type: kubernetes
name: contracts-publish
metadata:
namespace: musk-drone
trigger:
ref:
- refs/tags/*
event:
- tag
clone:
disable: true
volumes:
- name: nuget-cache
temp: {}
steps:
- name: clone
image: alpine/git
commands:
- git clone https://gitea.musk.fun/nocr/flea
- cd flea
- git checkout $DRONE_TAG
- git submodule update --init --recursive
- name: check-trigger
image: alpine/git
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
echo "Commit message - $COMMIT_MSG"
if echo "$COMMIT_MSG" | grep -q "contracts_only:"; then
echo "✅ contracts_only detected, proceeding..."
exit 0
else
echo "⏭️ No contracts_only marker, skipping..."
exit 78
fi
depends_on:
- clone
- name: telegram-listener-contracts
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
if echo "$COMMIT_MSG" | grep -q "contracts_only:telegram_listener"; then
echo "📦 Publishing telegram-listener contracts..."
mkdir -p /root/.nuget/NuGet
cp nuget.config /root/.nuget/NuGet/NuGet.Config
dotnet pack telegram-listener/Nocr.TelegramListener.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
else
echo "⏭️ Skipping telegram-listener contracts"
fi
depends_on:
- check-trigger
- name: text-matcher-contracts
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
if echo "$COMMIT_MSG" | grep -q "contracts_only:text_matcher"; then
echo "📦 Publishing text-matcher contracts..."
mkdir -p /root/.nuget/NuGet
cp nuget.config /root/.nuget/NuGet/NuGet.Config
dotnet pack text-matcher/Nocr.TextMatcher.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
else
echo "⏭️ Skipping text-matcher contracts"
fi
depends_on:
- check-trigger
- name: users-contracts
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
if echo "$COMMIT_MSG" | grep -q "contracts_only:users"; then
echo "📦 Publishing users contracts..."
mkdir -p /root/.nuget/NuGet
cp nuget.config /root/.nuget/NuGet/NuGet.Config
dotnet pack users/Nocr.Users.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
else
echo "⏭️ Skipping users contracts"
fi
depends_on:
- check-trigger
---
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 🚀 Pipeline 4: Full Release
# Trigger: Tag on main branch WITHOUT "contracts_only" or "deploy_only"
# Purpose: Full release cycle - contracts → images → optional deploy
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
kind: pipeline
type: kubernetes
name: full-release
metadata:
namespace: musk-drone
trigger:
ref:
- refs/tags/*
event:
- tag
clone:
disable: true
volumes:
- name: nuget-cache
temp: {}
steps:
- name: clone
image: alpine/git
commands:
- git clone https://gitea.musk.fun/nocr/flea
- cd flea
- git checkout $DRONE_TAG
- git submodule update --init --recursive
- name: check-trigger
image: alpine/git
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
echo "Commit message - $COMMIT_MSG"
if echo "$COMMIT_MSG" | grep -qE "contracts_only:|deploy_only:"; then
echo "⏭️ contracts_only or deploy_only detected, skipping full release..."
exit 78
else
echo "✅ Full release triggered"
exit 0
fi
depends_on:
- clone
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# STAGE 1: Publish NuGet Contracts (sequental)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
- name: telegram-listener-nuget
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- cd flea
- mkdir -p /root/.nuget/NuGet
- cp nuget.config /root/.nuget/NuGet/NuGet.Config
- dotnet pack telegram-listener/Nocr.TelegramListener.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
- dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
depends_on:
- check-trigger
- name: text-matcher-nuget
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- cd flea
- mkdir -p /root/.nuget/NuGet
- cp nuget.config /root/.nuget/NuGet/NuGet.Config
- dotnet pack text-matcher/Nocr.TextMatcher.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
- dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
depends_on:
- check-trigger
- telegram-listener-nuget
- name: users-nuget
image: mcr.microsoft.com/dotnet/sdk:8.0
volumes:
- name: nuget-cache
path: /root/.nuget/packages
environment:
NUGETAPIKEY:
from_secret: nuget_musk_api_key
commands:
- cd flea
- mkdir -p /root/.nuget/NuGet
- cp nuget.config /root/.nuget/NuGet/NuGet.Config
- dotnet pack users/Nocr.Users.sln -o ./bin -p:PackageVersion=${DRONE_TAG}
- dotnet nuget push ./bin/*Contract*.nupkg --api-key $NUGETAPIKEY --source musk --skip-duplicate
depends_on:
- check-trigger
- text-matcher-nuget
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# STAGE 2: Build Docker Images with Kaniko (sequential)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
- name: telegram-listener-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config telegram-listener/
- cd telegram-listener
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.TelegramListener.Host/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/telegram_listener:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/telegram_listener:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/telegram_listener:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-telegram-listener \
--compressed-caching=true
depends_on:
- users-nuget
- name: telegram-client-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config telegram-client/
- cd telegram-client
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.TelegramClient.Host/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/telegram_client:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/telegram_client:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/telegram_client:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-telegram-client \
--compressed-caching=true
depends_on:
- telegram-listener-build-push
- name: text-matcher-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config text-matcher/
- cd text-matcher
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.TextMatcher.Host/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/text_matcher:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/text_matcher:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/text_matcher:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-text-matcher \
--compressed-caching=true
depends_on:
- telegram-client-build-push
- name: text-matcher-migrator-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config text-matcher/
- cd text-matcher
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.TextMatcher.Migrator/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/text_matcher_migrator:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/text_matcher_migrator:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/text_matcher_migrator:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-text-matcher-migrator \
--compressed-caching=true
depends_on:
- text-matcher-build-push
- name: users-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config users/
- cd users
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.Users.Host/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/users:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/users:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/users:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-users \
--compressed-caching=true
depends_on:
- text-matcher-migrator-build-push
- name: users-migrator-build-push
image: gcr.io/kaniko-project/executor:debug
environment:
HUB_USERNAME:
from_secret: hub_username
HUB_PASSWORD:
from_secret: hub_password
commands:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"hub.musk.fun\":{\"username\":\"$HUB_USERNAME\",\"password\":\"$HUB_PASSWORD\"}}}" > /kaniko/.docker/config.json
- cd flea
- cp nuget.config users/
- cd users
- |
/kaniko/executor \
--context=. \
--dockerfile=src/Nocr.Users.Migrator/Dockerfile \
--destination=hub.musk.fun/k8s/nocr/users_migrator:${DRONE_COMMIT_SHA:0:7} \
--destination=hub.musk.fun/k8s/nocr/users_migrator:${DRONE_TAG} \
--destination=hub.musk.fun/k8s/nocr/users_migrator:latest \
--cache=true \
--cache-repo=hub.musk.fun/k8s/cache/nocr-users-migrator \
--compressed-caching=true
depends_on:
- users-build-push
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# STAGE 3: Deploy to Kubernetes (optional, based on tag pattern)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
- name: deploy-to-k8s
image: bitnami/kubectl:latest
environment:
KUBECONFIG_CONTENT:
from_secret: kubeconfig
commands:
- mkdir -p $HOME/.kube
- echo "$KUBECONFIG_CONTENT" > $HOME/.kube/config
- chmod 600 $HOME/.kube/config
- cd flea/_deploy/scripts
- sh deploy.sh "${DRONE_TAG}" "${DRONE_COMMIT_SHA:0:7}"
depends_on:
- users-migrator-build-push
when:
ref:
- refs/tags/*
---
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 🚀 Pipeline 5: Deploy-Only
# Trigger: Tag with commit message containing "deploy_only:"
# Purpose: Fast deploy of already-built images
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
kind: pipeline
type: kubernetes
name: deploy-only
metadata:
namespace: musk-drone
trigger:
ref:
- refs/tags/*
event:
- tag
clone:
disable: true
steps:
- name: clone
image: alpine/git
commands:
- git clone https://gitea.musk.fun/nocr/flea
- cd flea
- git checkout $DRONE_TAG
- git submodule update --init --recursive
- name: check-trigger
image: alpine/git
commands:
- |
cd flea
COMMIT_MSG=$(git log -1 --pretty=%B)
echo "Commit message - $COMMIT_MSG"
if echo "$COMMIT_MSG" | grep -q "deploy_only:"; then
echo "✅ deploy_only detected, proceeding..."
exit 0
else
echo "⏭️ No deploy_only marker, skipping..."
exit 78
fi
depends_on:
- clone
- name: deploy
image: bitnami/kubectl:latest
environment:
KUBECONFIG_CONTENT:
from_secret: kubeconfig
commands:
- mkdir -p $HOME/.kube
- echo "$KUBECONFIG_CONTENT" > $HOME/.kube/config
- chmod 600 $HOME/.kube/config
- cd flea/_deploy/scripts
- sh deploy.sh "${DRONE_TAG}" "" "latest"
depends_on:
- check-trigger

6
.gitignore vendored
View File

@ -3,5 +3,9 @@
!docker-compose.yml
!README.md
.secrets/
.env
.nocr.env
!.nocr.env.example
.mono/
WTelegram.session
WTelegram.session
.claude/

43
.nocr.env.example Normal file
View File

@ -0,0 +1,43 @@
# ===========================================
# Environment Variables for Docker Compose
# ===========================================
# This file provides example environment variables for running the system with Docker Compose.
#
# USAGE:
# 1. Copy this file to .nocr.env in the same directory
# 2. Fill in your actual values (removing placeholder text)
# 3. Run: docker-compose up
#
# SECURITY NOTE:
# The .nocr.env file is gitignored and should NEVER be committed to version control.
# For K8s deployments, use Secrets instead of .nocr.env files.
# ===========================================
# -----------------
# Telegram Listener
# -----------------
# WTelegram API credentials - obtain from https://my.telegram.org/apps
# Your Telegram API ID (numeric)
WTelegramClientOptions__ApiId=YOUR_API_ID_HERE
# Your Telegram API Hash (alphanumeric string)
WTelegramClientOptions__ApiHash=YOUR_API_HASH_HERE
# Your phone number with country code (e.g., 79167310711)
WTelegramClientOptions__PhoneNumber=YOUR_PHONE_NUMBER_HERE
# -----------------
# Telegram Client (Bot)
# -----------------
# Bot token from @BotFather on Telegram
# Telegram Bot API Token
TelegramBotOptions__Token=YOUR_BOT_TOKEN_HERE
# -----------------
# Optional: Debug Mode
# -----------------
# Set to "true" to enable configuration debug logging on service startup
# This will print masked configuration values to console for troubleshooting
# NOCR_DEBUG_MODE=true

555
CLAUDE.md Normal file
View File

@ -0,0 +1,555 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is a microservices-based Telegram bot system for text monitoring and notifications. The system consists of 4 main services that communicate via RabbitMQ:
- **telegram-listener**: Scans open Telegram channels and chats for messages
- **telegram-client**: Bot interface for user interactions
- **text-matcher**: Matches incoming messages against user subscriptions
- **users**: Manages users and their preferences
**Important:** This repository is the parent project containing all services as **git submodules**. Each service (telegram-listener, telegram-client, text-matcher, users) is a separate repository added as a submodule. Git tags are created at the parent repository level, which triggers the Drone CI/CD pipeline to automatically build and publish NuGet packages with versions based on the git tag.
## Architecture
The services follow a message bus pattern using RabbitMQ for async communication:
- TelegramListener publishes:
- **MessageReceived** events (for new messages)
- **MessageEdited** events (for edited messages)
- TextMatcher subscribes to MessageReceived and MessageEdited, stores match history in database, and publishes:
- TextSubscriptionMatched events (for first-time matches)
- TextSubscriptionUpdated events (for updates to previously matched messages)
- TelegramClient subscribes to both TextSubscriptionMatched and TextSubscriptionUpdated events and notifies users
**Note:** WTelegram sends both `UpdateNewChannelMessage` and `UpdateEditChannelMessage` for the same message. TelegramListener publishes separate events to avoid duplicate notifications downstream.
Each service follows Clean Architecture with separate projects for:
- **Host** (API/entry point) - ASP.NET Core application, controllers, startup configuration
- **AppServices** (business logic) - Use cases, business logic, command/query handlers
- **Core** (shared utilities) - Domain entities, interfaces, shared logic
- **Api.Contracts** (REST API contracts) - DTOs, request/response models for REST endpoints (published as NuGet)
- **Async.Api.Contracts** (event contracts) - Event DTOs for RabbitMQ messaging (published as NuGet)
- **Persistence** (database layer) - EF Core DbContext, repositories, data access (text-matcher, users only)
- **Migrator** (database migrations) - EF Core migrations, migration scripts (text-matcher, users only)
**Project Structure Examples:**
- telegram-listener: Host, AppServices, Core, Async.Api.Contracts (no database)
- telegram-client: Host, AppServices, Core (no database, no published contracts)
- text-matcher: Host, AppServices, Core, Api.Contracts, Async.Api.Contracts, Persistence, Migrator
- users: Host, AppServices, Core, Api.Contracts, Persistence, Migrator
## Development Commands
### Running the System
```bash
# IMPORTANT: Before building with Docker Compose, prepare the build environment
./prepare-build.sh
# Start all services with Docker Compose
docker-compose up
# Start individual services for development (no preparation needed)
cd telegram-client && dotnet run --project src/Nocr.TelegramClient.Host
cd telegram-listener && dotnet run --project src/Nocr.TelegramListener.Host
cd text-matcher && dotnet run --project src/Nocr.TextMatcher.Host
cd users && dotnet run --project src/Nocr.Users.Host
```
**Note:** The `prepare-build.sh` script copies `nuget.config` to all submodule roots, which is required for Docker builds. This step is automatic in CI/CD but must be run manually for local Docker Compose builds.
### Building
```bash
# Build individual services
cd <service-name> && dotnet build
# Build specific projects
dotnet build src/Nocr.<ServiceName>.Host
```
### Testing
```bash
# Run all tests in text-matcher
cd text-matcher && dotnet test
# Run tests for a specific project
cd text-matcher && dotnet test tests/Nocr.TextMatcher.AppServices.UnitTests/Nocr.TextMatcher.AppServices.UnitTests.csproj
# Run tests with verbose output
cd text-matcher && dotnet test --verbosity detailed
# Run tests with code coverage (if configured)
cd text-matcher && dotnet test --collect:"XPlat Code Coverage"
```
### Database Migrations
For text-matcher and users services:
```bash
# Add new migration (from service root directory)
cd text-matcher && ./src/Nocr.TextMatcher.Migrator/AddMigration.sh MyMigrationName
cd users && ./src/Nocr.Users.Migrator/AddMigration.sh MyMigrationName
# Apply migrations (handled automatically by migrator containers in docker-compose)
```
### Working with Git Submodules
```bash
# Initialize submodules after cloning the repository
git submodule update --init --recursive
# Update all submodules to their latest commits on main
./update-submodules.sh
# Commit and push changes to all submodules at once
./commit-all.sh "Your commit message"
# Update a single submodule
cd telegram-listener
git pull origin main
cd ..
git add telegram-listener
git commit -m "Update telegram-listener submodule"
# Check status of all submodules
git submodule status
```
## Configuration
**📖 See [CONFIGURATION.md](CONFIGURATION.md) for detailed configuration guide.**
### Quick Start
The system uses ASP.NET Core's layered configuration with **Environment Variables having highest priority**.
**Configuration priority (lowest → highest):**
1. `appsettings.json` (base settings, committed)
2. `appsettings.{Environment}.json` (environment-specific, some committed via `.example` files)
3. User Secrets (Development only)
4. **Environment Variables (ALWAYS WINS)**
### Three Deployment Modes
1. **VS Code (Local Development)**
- Copy `appsettings.Development.json.example``appsettings.Development.json` in each service
- Or use environment variables in `.vscode/launch.json`
- Start infrastructure: `docker-compose up nocr-rabbitmq nocr-text-matcher-db nocr-users-db -d`
2. **Docker Compose (Local Full Stack)**
- Copy `.nocr.env.example``.nocr.env` in project root
- Fill in your Telegram API credentials and Bot token
- Run: `docker-compose up`
3. **Kubernetes (Production)**
- Create K8s Secrets (do NOT use `.nocr.env`)
- Reference secrets in deployment manifests via `envFrom`
### Debug Mode
Enable configuration debug logging on startup:
```bash
export NOCR_DEBUG_MODE=true
```
This prints masked configuration values to help troubleshoot issues.
### Important Notes
- ❌ **Never commit secrets** - use `.example` files as templates
- ✅ Environment variables override all appsettings files
- ✅ All services use `.example` files for documentation
- ✅ Docker Compose uses `.nocr.env` file (gitignored)
## Service Ports
When running with docker-compose:
- telegram-client: 5050 (http://localhost:5050/health)
- telegram-listener: 5040 (http://localhost:5040/health)
- text-matcher: 5041 (http://localhost:5041/health)
- users: 5042 (http://localhost:5042/health)
- RabbitMQ: 5672 (AMQP), 15672 (Management UI - http://localhost:15672, admin/admin)
- MariaDB: 3316 (text-matcher), 3326 (users)
## Key Technologies
- **.NET 8** - All services built on .NET 8
- **ASP.NET Core Web APIs** - REST endpoints and hosting
- **Entity Framework Core with MariaDB** - ORM for text-matcher and users databases
- **WTelegramClient** - MTProto API client for telegram-listener
- **Telegram.Bot** - Bot API client for telegram-client
- **Rebus** - Message bus abstraction over RabbitMQ
- **RabbitMQ** - Message broker for async event-driven communication
- **Docker & Docker Compose** - Local development and containerization
- **Kaniko** - Container image building in CI/CD
- **Drone CI** - CI/CD pipeline on Kubernetes
## NuGet Package Management
The project uses **Central Package Management (CPM)** with **Package Source Mapping** to manage NuGet dependencies:
### Package Sources
- **nuget.org**: All public packages (Microsoft.*, Serilog.*, etc.)
- **musk** (private): Internal `Nocr.*` contract packages
### Configuration
The `nuget.config` file in the project root defines package source mapping:
```xml
<packageSourceMapping>
<packageSource key="musk">
<package pattern="Nocr.*" />
</packageSource>
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
</packageSourceMapping>
```
### How It Works
1. **Local Development**: Copy `nuget.config` to each submodule root when needed
2. **CI/CD**: Drone automatically copies `nuget.config` to `/root/.nuget/NuGet/NuGet.Config`
3. **Docker Builds**: Kaniko copies `nuget.config` to submodule root before building
This eliminates NuGet warning NU1507 and ensures consistent package resolution across all environments.
## CI/CD Pipeline
**📖 See [_deploy/README.md](_deploy/README.md) for full CI/CD documentation.**
The project uses Drone CI with 5 specialized pipelines:
### 1. Feature Validation (`feature/*`, `fix/*` branches)
**Purpose**: Test feature branches before merging to main
**How to trigger**:
```bash
# Create feature branch and push
git checkout -b feature/my-new-feature
# ... make changes ...
git add .
git commit -m "Add new feature"
git push origin feature/my-new-feature
```
**What happens**:
- Runs build + tests with Testcontainers support
- Provides fast feedback before merge
- Duration: ~3-5 minutes
### 2. Main Validation (`main` branch)
**Purpose**: Ensure main branch stays healthy after merge
**How to trigger**:
```bash
# Merge feature to main
git checkout main
git merge feature/my-new-feature
git push origin main
```
**What happens**:
- Same checks as feature validation
- Ensures main branch builds and tests pass
- Duration: ~3-5 minutes
### 3. Contracts-Only Publish
**Purpose**: Publish NuGet contracts without building Docker images (fast iteration)
**How to trigger**:
```bash
# Update contracts in a submodule
cd telegram-listener
# ... update Async.Api.Contracts ...
git add . && git commit -m "Add MessageEdited event"
git push
# From parent repo, tag with contracts_only marker
cd ..
git add telegram-listener
git commit -m "contracts_only:telegram_listener - Add MessageEdited event"
git tag 0.7.41
git push && git push --tags
```
**What happens**:
- Publishes only the specified service's NuGet contracts
- Skips Docker image builds
- Duration: ~2 minutes
**Services**: `telegram_listener`, `text_matcher`, `users`
### 4. Full Release
**Purpose**: Complete release - contracts + images + deployment
**How to trigger**:
```bash
# Make your changes, commit to main
git add .
git commit -m "Implement new feature"
git push
# Tag for full release (no special markers in commit message)
git tag v1.3.0
git push --tags
```
**What happens**:
1. Publishes all NuGet contracts (parallel)
2. Builds all Docker images with Kaniko (parallel, after contracts)
3. Deploys to Kubernetes (automatic for `v*` tags)
- Duration: ~8-10 minutes
### 5. Deploy-Only
**Purpose**: Deploy existing images without rebuilding (rollbacks, hotfixes)
**How to trigger**:
```bash
# Deploy already-built images (e.g., rollback to previous version)
git commit --allow-empty -m "deploy_only: Rollback to v1.2.9"
git tag deploy-v1.2.9
git push && git push --tags
# Or deploy latest images
git commit --allow-empty -m "deploy_only: Deploy latest"
git tag deploy-latest
git push && git push --tags
```
**What happens**:
- Skips building entirely
- Deploys specified tag's images (or `latest`)
- Duration: ~1 minute
## Development Workflows
### Making Changes to a Single Service
When working on a specific service (e.g., telegram-listener):
```bash
# 1. Navigate to the submodule
cd telegram-listener
# 2. Create a feature branch in the submodule
git checkout -b feature/new-feature
# 3. Make your changes to the code
# ... edit files ...
# 4. Test locally (if tests exist)
dotnet test
# 5. Commit and push in the submodule
git add .
git commit -m "Add new feature"
git push origin feature/new-feature
# 6. Return to parent repo and update submodule reference
cd ..
git add telegram-listener
git commit -m "Update telegram-listener: Add new feature"
git push origin main
```
### Updating Contract Packages
When you change event contracts (Async.Api.Contracts) or REST contracts (Api.Contracts):
```bash
# 1. Make changes to contracts in the submodule
cd text-matcher
# ... edit Nocr.TextMatcher.Async.Api.Contracts ...
git add . && git commit -m "Add new event contract"
git push origin main
# 2. Return to parent repo and publish contracts only
cd ..
git add text-matcher
git commit -m "contracts_only:text_matcher - Add new event contract"
git tag 0.8.6
git push && git push --tags
# 3. Other services can now reference the new contract version
# Update their .csproj or Directory.Packages.props to use version 0.8.6
```
### Working Across Multiple Services
When implementing a feature that spans multiple services:
```bash
# 1. Update each submodule in sequence
cd telegram-listener
git checkout -b feature/cross-service-feature
# ... make changes ...
git commit -m "Part 1: Update listener"
git push origin feature/cross-service-feature
cd ../text-matcher
git checkout -b feature/cross-service-feature
# ... make changes ...
git commit -m "Part 2: Update matcher"
git push origin feature/cross-service-feature
# 2. Update parent repo to reference all changes
cd ..
git add telegram-listener text-matcher
git commit -m "Implement cross-service feature"
git push origin main
# 3. Tag for full release
git tag v1.5.0
git push --tags
```
## Common CI/CD Workflows
### Test a Feature Branch
```bash
git checkout -b feature/add-logging
# ... make changes ...
git add . && git commit -m "Add structured logging"
git push origin feature/add-logging
# ✅ Drone runs feature-validation pipeline
```
### Publish Contracts After API Changes
```bash
cd text-matcher
# Update Nocr.TextMatcher.Async.Api.Contracts
git add . && git commit -m "Add UserDeleted event"
git push
cd ..
git add text-matcher
git commit -m "contracts_only:text_matcher - Add UserDeleted event"
git tag 0.8.5
git push && git push --tags
# ✅ Drone publishes text-matcher contracts to NuGet
```
### Full Release Workflow
```bash
# Work on main branch
git add .
git commit -m "Implement payment processing"
git push
# ✅ Drone runs main-validation
# Tag for release
git tag v2.0.0
git push --tags
# ✅ Drone runs full-release pipeline:
# 1. Publishes all contracts
# 2. Builds all Docker images
# 3. Deploys to Kubernetes
```
### Emergency Rollback
```bash
# Rollback to last known good version
git commit --allow-empty -m "deploy_only: Emergency rollback to v1.9.5"
git tag rollback-v1.9.5
git push && git push --tags
# ✅ Drone deploys v1.9.5 images immediately (~1 minute)
```
### Deployment Scripts
Located in `_deploy/scripts/`:
- **deploy.sh** - Deploy services to Kubernetes with health checks
- **rollback.sh** - Rollback deployments to previous version
- **health-check.sh** - Check health of all services
### Key Optimizations
- **Shared NuGet cache** across pipeline steps (~60% faster builds)
- **Parallel execution** for independent operations
- **Proper dependency order**: Contracts → Images → Deploy
- **Kaniko layer caching** for faster Docker builds
- **Docker-in-Docker** support for Testcontainers in tests
## Troubleshooting
### Submodule Issues
**Problem:** Submodule directories are empty after cloning
```bash
# Solution: Initialize submodules
git submodule update --init --recursive
```
**Problem:** Submodule is in detached HEAD state
```bash
# Solution: Check out the main branch in the submodule
cd <submodule-name>
git checkout main
git pull origin main
cd ..
```
**Problem:** Changes in submodule not reflected in parent repo
```bash
# Solution: Update submodule reference in parent
cd .. # Return to parent repo
git add <submodule-name>
git commit -m "Update <submodule-name> reference"
```
### Build Issues
**Problem:** NuGet package restore fails with NU1507 warning
```bash
# Solution: Copy nuget.config to submodule root
cp nuget.config telegram-listener/
cd telegram-listener && dotnet restore
```
**Problem:** Cannot find Nocr.* contract packages
```bash
# Solution: Ensure you have access to the private NuGet feed
# Check nuget.config has the correct source configuration
# Verify credentials for the "musk" package source
```
### Docker Compose Issues
**Problem:** Service cannot connect to RabbitMQ
```bash
# Solution: Use the Docker network hostname, not localhost
# In docker-compose environment: nocr-rabbitmq:5672
# In local development: localhost:5672
```
**Problem:** Database connection fails
```bash
# Solution: Wait for database health checks to pass
# Check docker-compose logs for database container
docker-compose logs nocr-text-matcher-db
# Database takes 10-30 seconds to be ready on first start
```
### CI/CD Issues
**Problem:** Pipeline doesn't trigger on tag
```bash
# Solution: Ensure tag matches expected patterns
# Feature validation: feature/*, fix/*, issues/*
# Main validation: main branch
# Contracts: Tag with commit message containing "contracts_only:<service>"
# Full release: Any tag without special markers
# Deploy-only: Tag with commit message containing "deploy_only:"
```
**Problem:** Docker image build fails in Kaniko
```bash
# Solution: Check if nuget.config is correctly copied
# Verify _deploy/deploy.sh contains nuget.config copy step
# Check Drone logs for nuget.config presence in build context
```

563
CONFIGURATION.md Normal file
View File

@ -0,0 +1,563 @@
# Руководство по конфигурации
Этот документ описывает, как настроить микросервисную систему NOCR для различных сценариев развертывания.
## Содержание
1. [Обзор системы конфигурации](#обзор-системы-конфигурации)
2. [Сценарии развертывания](#сценарии-развертывания)
- [Локальная разработка (VS Code)](#локальная-разработка-vs-code)
- [Docker Compose](#docker-compose)
- [Kubernetes](#kubernetes)
3. [Приоритет конфигурации](#приоритет-конфигурации)
4. [Настройки для конкретных сервисов](#настройки-для-конкретных-сервисов)
5. [Режим отладки](#режим-отладки)
6. [Устранение неполадок](#устранение-неполадок)
---
## Обзор системы конфигурации
Система использует многоуровневый подход конфигурации ASP.NET Core со следующими источниками (от низшего к высшему приоритету):
1. `appsettings.json` - Базовая конфигурация (коммитится в git)
2. `appsettings.{Environment}.json` - Настройки для конкретного окружения (некоторые коммитятся, некоторые нет)
3. User Secrets - Секреты только для разработки (хранятся локально, не в git)
4. **Переменные окружения** - **Наивысший приоритет** (переопределяют всё)
### Важные примечания
- **Никогда не коммитьте секреты в git**
- Используйте файлы `.example` как шаблоны
- Переменные окружения всегда имеют наивысший приоритет
- Каждый сервис имеет свои требования к конфигурации
---
## Сценарии развертывания
### Локальная разработка (VS Code)
Для отладки отдельных сервисов в VS Code:
#### Шаг 1: Создание конфигурации для окружения
Для каждого сервиса, который вы хотите запустить, создайте `appsettings.Development.json` из примера:
```bash
# Telegram Listener
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.Development.json.example appsettings.Development.json
# Text Matcher
cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.Development.json.example appsettings.Development.json
# Users
cd users/src/Nocr.Users.Host
cp appsettings.Development.json.example appsettings.Development.json
# Telegram Client
cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.Development.json.example appsettings.Development.json
```
#### Шаг 2: Заполните ваши значения
Отредактируйте каждый файл `appsettings.Development.json` и замените placeholder-значения на ваши реальные учетные данные.
**Пример для Telegram Listener:**
```json
{
"RebusRabbitMqOptions": {
"ConnectionString": "amqp://admin:admin@localhost:5672/"
},
"WTelegramClientOptions": {
"ApiId": "22101230",
"ApiHash": "c72f884d8eb84cb7134a14362bff060b",
"PhoneNumber": "79167310711"
}
}
```
#### Шаг 3: Альтернатива - использование переменных окружения
Вместо создания `appsettings.Development.json`, вы можете использовать переменные окружения в `.vscode/launch.json`:
```json
{
"name": "Запуск сервиса",
"type": "coreclr",
"request": "launch",
"env": {
"ASPNETCORE_ENVIRONMENT": "Development",
"RebusRabbitMqOptions__ConnectionString": "amqp://admin:admin@localhost:5672/",
"WTelegramClientOptions__ApiId": "22101230",
"WTelegramClientOptions__ApiHash": "c72f884d8eb84cb7134a14362bff060b",
"WTelegramClientOptions__PhoneNumber": "79167310711"
}
}
```
**Примечание:** Используйте двойное подчеркивание `__` для представления вложенных секций конфигурации в переменных окружения.
#### Шаг 4: Запуск инфраструктуры
Вам нужны работающие RabbitMQ и базы данных:
```bash
# Запустить только инфраструктурные сервисы
docker-compose up nocr-rabbitmq nocr-text-matcher-db nocr-users-db -d
```
#### Шаг 5: Отладка в VS Code
Нажмите F5 или используйте панель Run and Debug для запуска вашего сервиса.
---
### Docker Compose
Для запуска всей системы локально с Docker:
#### Шаг 1: Создание файла .nocr.env
```bash
cd /путь/к/корню/проекта
cp .nocr.env.example .nocr.env
```
#### Шаг 2: Редактирование .nocr.env
Откройте `.nocr.env` и заполните ваши реальные учетные данные:
```bash
# Telegram Listener - получить на https://my.telegram.org/apps
WTelegramClientOptions__ApiId=22101230
WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b
WTelegramClientOptions__PhoneNumber=79167310711
# Telegram Client Bot - получить у @BotFather
TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz
# Опционально: включить отладочное логирование
# NOCR_DEBUG_MODE=true
```
#### Шаг 3: Создание конфигурационных файлов для DockerCompose
Каждому сервису нужен `appsettings.DockerCompose.json`:
```bash
# Для каждого сервиса
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json
cd text-matcher/src/Nocr.TextMatcher.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json
cd users/src/Nocr.Users.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json
cd telegram-client/src/Nocr.TelegramClient.Host
cp appsettings.DockerCompose.json.example appsettings.DockerCompose.json
```
**Примечание:** Эти файлы содержат имена хостов Docker-сети (например, `nocr-rabbitmq:5672` вместо `localhost:5672`).
#### Шаг 4: Запуск всех сервисов
```bash
docker-compose up
```
Или со сборкой:
```bash
docker-compose up --build
```
#### Шаг 5: Проверка работоспособности сервисов
- Telegram Listener: http://localhost:5040/health
- Text Matcher: http://localhost:5041/health
- Users: http://localhost:5042/health
- Telegram Client: http://localhost:5050/health
- RabbitMQ Management: http://localhost:15672 (admin/admin)
---
### Kubernetes
Для production-развертывания на Kubernetes:
#### Шаг 1: Создание Kubernetes Secrets
**НЕ используйте файл `.nocr.env` в K8s.** Вместо этого создайте Secrets:
```bash
# Создать namespace
kubectl create namespace nocr
# Создать secrets для Telegram Listener
kubectl create secret generic telegram-listener-secrets \
--from-literal=WTelegramClientOptions__ApiId=22101230 \
--from-literal=WTelegramClientOptions__ApiHash=c72f884d8eb84cb7134a14362bff060b \
--from-literal=WTelegramClientOptions__PhoneNumber=79167310711 \
-n nocr
# Создать secrets для Telegram Client
kubectl create secret generic telegram-client-secrets \
--from-literal=TelegramBotOptions__Token=1234567890:ABCdefGHIjklMNOpqrsTUVwxyz \
-n nocr
```
#### Шаг 2: Ссылка на Secrets в Deployment
Пример манифеста Kubernetes deployment:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: telegram-listener
namespace: nocr
spec:
template:
spec:
containers:
- name: telegram-listener
image: nocr-telegram-listener:latest
env:
- name: ASPNETCORE_ENVIRONMENT
value: "Production"
envFrom:
- secretRef:
name: telegram-listener-secrets
```
#### Шаг 3: Создание appsettings.Production.json
Создайте production-конфигурационные файлы с именами сервисов K8s:
```json
{
"RebusRabbitMqOptions": {
"ConnectionString": "amqp://admin:admin@rabbitmq-service:5672/"
}
}
```
**Примечание:** Секреты из переменных окружения переопределят эти значения.
#### Шаг 4: Деплой
```bash
kubectl apply -f deployment/
```
---
## Приоритет конфигурации
Понимание приоритета критически важно при устранении проблем с конфигурацией.
### Порядок приоритета (от низшего к высшему)
1. ⬇️ `appsettings.json` (базовый)
2. ⬆️ `appsettings.{Environment}.json` (например, Development, DockerCompose, Production)
3. ⬆️ User Secrets (только Development)
4. ⬆️⬆️ **Переменные окружения (ВСЕГДА ВЫИГРЫВАЮТ)**
### Пример сценария
Если у вас есть:
- `appsettings.json`: `"ConnectionString": ""`
- `appsettings.DockerCompose.json`: `"ConnectionString": "amqp://admin:admin@nocr-rabbitmq:5672/"`
- Переменная окружения: `RebusRabbitMqOptions__ConnectionString=amqp://admin:admin@localhost:5672/`
**Результат:** Переменная окружения побеждает! Подключение будет использовать `localhost:5672`.
Именно поэтому мы удалили `appsettings.protected.json` - он неправильно переопределял настройки Docker Compose.
---
## Настройки для конкретных сервисов
### Telegram Listener
**Обязательная конфигурация:**
```json
{
"RebusRabbitMqOptions": {
"ConnectionString": "amqp://user:pass@host:port/"
},
"WTelegramClientOptions": {
"ApiId": "YOUR_API_ID",
"ApiHash": "YOUR_API_HASH",
"PhoneNumber": "YOUR_PHONE_NUMBER"
}
}
```
**Как получить учетные данные Telegram:**
1. Посетите https://my.telegram.org/apps
2. Войдите с вашим номером телефона
3. Создайте новое приложение
4. Скопируйте `api_id` и `api_hash`
---
### Text Matcher
**Обязательная конфигурация:**
```json
{
"ConnectionStrings": {
"TextMatcherContext": "server=host;port=3306;database=nocr_text_matcher;uid=root;pwd=password"
},
"RebusRabbitMqOptions": {
"ConnectionString": "amqp://user:pass@host:port/"
}
}
```
**Порты по окружениям:**
- Development: `localhost:3316`
- Docker Compose: `nocr-text-matcher-db:3306`
- Kubernetes: `text-matcher-db-service:3306`
---
### Users
**Обязательная конфигурация:**
```json
{
"ConnectionStrings": {
"UsersContext": "server=host;port=3306;database=nocr_users;uid=root;pwd=password"
}
}
```
**Порты по окружениям:**
- Development: `localhost:3326`
- Docker Compose: `nocr-users-db:3306`
- Kubernetes: `users-db-service:3306`
---
### Telegram Client
**Обязательная конфигурация:**
```json
{
"RebusRabbitMqOptions": {
"ConnectionString": "amqp://user:pass@host:port/"
},
"UsersRestEaseOptions": {
"BasePath": "http://users-service:8080"
},
"TextMatcherRestEaseOptions": {
"BasePath": "http://text-matcher-service:8080"
},
"TelegramBotOptions": {
"Token": "YOUR_BOT_TOKEN"
}
}
```
**Как получить Bot Token:**
1. Откройте Telegram и найдите @BotFather
2. Отправьте команду `/newbot`
3. Следуйте инструкциям для создания вашего бота
4. Скопируйте предоставленный токен
---
## Режим отладки
Включите режим отладки для вывода маскированных значений конфигурации при запуске.
### Как включить
Установите переменную окружения:
```bash
export NOCR_DEBUG_MODE=true
```
Или в `.nocr.env`:
```bash
NOCR_DEBUG_MODE=true
```
Или в `docker-compose.yml`:
```yaml
environment:
NOCR_DEBUG_MODE: "true"
```
### Пример вывода
Когда режим отладки включен, вы увидите маскированную конфигурацию при запуске:
```
=== [NOCR_DEBUG] Значения конфигурации ===
[NOCR_DEBUG] RebusRabbitMqOptions:
[NOCR_DEBUG] ConnectionString: amqp://admin:***@nocr-rabbitmq:5672/
[NOCR_DEBUG] InputQueueName: nocr.telegram.listener.queue
[NOCR_DEBUG] DirectExchangeName: nocr.direct
[NOCR_DEBUG] TopicsExchangeName: nocr.topics
[NOCR_DEBUG] WTelegramClientOptions:
[NOCR_DEBUG] ApiId: 22101230
[NOCR_DEBUG] ApiHash: c7...0b
[NOCR_DEBUG] PhoneNumber: 79...11
=== [NOCR_DEBUG] Конец конфигурации ===
```
**Безопасность:** Пароли и секреты автоматически маскируются для защиты конфиденциальных данных.
---
## Устранение неполадок
### Проблема: Сервис не может подключиться к RabbitMQ
**Симптомы:**
```
Connection refused to nocr-rabbitmq:5672
```
**Решение:**
1. Проверьте, работает ли RabbitMQ: `docker-compose ps nocr-rabbitmq`
2. Включите режим отладки: `NOCR_DEBUG_MODE=true`
3. Проверьте ConnectionString в выводе отладки
4. Для Docker Compose убедитесь, что hostname - `nocr-rabbitmq:5672`, НЕ `localhost:5672`
5. Проверьте, не переопределяет ли переменная окружения appsettings
---
### Проблема: Сервис не может подключиться к базе данных
**Симптомы:**
```
Unable to connect to any of the specified MySQL hosts
```
**Решение:**
1. Проверьте, что база данных запущена: `docker-compose ps nocr-text-matcher-db`
2. Включите режим отладки для просмотра маскированной строки подключения
3. Проверьте hostname в строке подключения:
- Docker Compose: `nocr-text-matcher-db:3306`
- Локальная разработка: `localhost:3316`
4. Подождите прохождения healthcheck (может занять 10-30 секунд при первом запуске)
---
### Проблема: Значения конфигурации не применяются
**Симптомы:**
- Настройки в `appsettings.Development.json` игнорируются
- Сервис использует неправильную конфигурацию
**Решение:**
1. Проверьте, что `ASPNETCORE_ENVIRONMENT` установлен правильно:
- VS Code: `Development`
- Docker Compose: `DockerCompose`
- K8s: `Production`
2. Включите режим отладки для просмотра загруженных значений
3. Проверьте переменные окружения, переопределяющие ваши настройки
4. Помните: **Переменные окружения всегда выигрывают!**
---
### Проблема: Секреты раскрыты в логах
**Решение:**
- Режим отладки автоматически маскирует конфиденциальные значения
- Никогда не логируйте конфигурацию в production без маскировки
- Проверьте формат вывода отладки в `Startup.cs`:
- Пароли: `amqp://user:***@host`
- Секреты: `ab...yz` (только первые 2 + последние 2 символа)
---
### Проблема: Отсутствует файл .nocr.env
**Симптомы:**
```
docker-compose up падает с ошибкой отсутствующих переменных окружения
```
**Решение:**
1. Скопируйте файл-пример: `cp .nocr.env.example .nocr.env`
2. Заполните ваши реальные значения
3. Убедитесь, что `.nocr.env` находится в корне проекта (на том же уровне, что и `docker-compose.yml`)
---
## Краткая справка
### Соглашение об именовании переменных окружения
ASP.NET Core использует двойное подчеркивание `__` для представления вложенной JSON-структуры:
```bash
# JSON: { "Section": { "Key": "Value" } }
# Переменная окружения:
Section__Key=Value
# Примеры:
WTelegramClientOptions__ApiId=12345
RebusRabbitMqOptions__ConnectionString=amqp://localhost
```
### Расположение файлов
```
flea/ # Корень проекта
├── .nocr.env # Ваши секреты (gitignored)
├── .nocr.env.example # Шаблон (коммитится)
├── docker-compose.yml # Ссылается на .nocr.env
├── telegram-listener/
│ └── src/Nocr.TelegramListener.Host/
│ ├── appsettings.json # Базовый (коммитится)
│ ├── appsettings.Development.json # Локальная разработка (gitignored)
│ ├── appsettings.Development.json.example # Шаблон (коммитится)
│ ├── appsettings.DockerCompose.json # Docker (gitignored)
│ └── appsettings.DockerCompose.json.example # Шаблон (коммитится)
└── [та же структура для text-matcher, users, telegram-client]
```
---
## Рекомендации по безопасности
1. ✅ **Никогда не коммитьте секреты** - используйте файлы `.example` как шаблоны
2. ✅ **Используйте переменные окружения** для конфиденциальных данных в production
3. ✅ **Используйте K8s Secrets** для развертывания в Kubernetes
4. ✅ **Включайте режим отладки** только при устранении неполадок
5. ✅ **Ротируйте учетные данные** регулярно
6. ✅ **Используйте User Secrets** для локальной разработки (опционально)
7. ❌ **Не коммитьте** `.nocr.env` или `appsettings.*.json` (кроме файлов `.example`)
8. ❌ **Не используйте** `appsettings.protected.json` (удален из кода)
---
## Дополнительные ресурсы
- [Документация конфигурации ASP.NET Core](https://learn.microsoft.com/ru-ru/aspnet/core/fundamentals/configuration/)
- [Переменные окружения Docker Compose](https://docs.docker.com/compose/environment-variables/)
- [Kubernetes Secrets](https://kubernetes.io/ru/docs/concepts/configuration/secret/)
- [Документация Telegram API](https://core.telegram.org/api)
- [Telegram Bot API](https://core.telegram.org/bots/api)

450
README.md
View File

@ -1,4 +1,446 @@
# Спец проект для работы с зависимостями, внешними и внутренними сервисами и приложениями.
# Попытка собрать.
[![Build Status](https://drone.musk.fun/api/badges/nocr/local-env/status.svg?ref=refs/heads/main)](https://drone.musk.fun/nocr/local-env)
# NOCR - Система мониторинга Telegram-каналов
[![Build Status](https://drone.musk.fun/api/badges/nocr/flea/status.svg?ref=refs/heads/main)](https://drone.musk.fun/nocr/flea)
Микросервисная система на базе Telegram Bot для мониторинга и уведомлений о текстовых совпадениях в открытых каналах и чатах.
## 📋 Обзор проекта
Это родительский проект, содержащий все сервисы как **git submodules**. Каждый сервис является отдельным репозиторием, что обеспечивает независимую разработку и версионирование.
### Архитектура сервисов
Система состоит из 4 основных микросервисов, взаимодействующих через RabbitMQ:
#### 1. [telegram-listener](https://gitea.musk.fun/nocr/telegram-listener)
**Назначение:** Сканирование открытых Telegram-каналов и чатов
- Подписка на каналы через Telegram MTProto API
- Публикация событий о новых и отредактированных сообщениях в message bus
- Хранение сессий пользователей
#### 2. [text-matcher](https://gitea.musk.fun/nocr/text-matcher)
**Назначение:** Сопоставление сообщений с пользовательскими подписками
- Обработка событий от telegram-listener
- Проверка совпадений по регулярным выражениям и ключевым словам
- Хранение истории совпадений в БД (MariaDB)
- Публикация событий о найденных совпадениях
#### 3. [telegram-client](https://gitea.musk.fun/nocr/telegram-client)
**Назначение:** Клиентский интерфейс бота для взаимодействия с пользователями
- Прием команд от пользователей через Telegram Bot API
- Управление подписками на каналы
- Отправка уведомлений о найденных совпадениях
- Интеграция с users и text-matcher через REST API
#### 4. [users](https://gitea.musk.fun/nocr/users)
**Назначение:** Управление пользователями и их настройками
- CRUD операции для пользователей
- Хранение предпочтений и подписок
- REST API для других сервисов
---
## 🚀 Быстрый старт
### Вариант 1: Docker Compose (рекомендуется)
```bash
# 1. Создайте файл с секретами
cp .nocr.env.example .nocr.env
# 2. Заполните ваши Telegram API ключи в .nocr.env
# - API ID и Hash: https://my.telegram.org/apps
# - Bot Token: @BotFather в Telegram
# 3. Запустите все сервисы
docker-compose up
```
**Проверка работоспособности:**
- Telegram Listener: http://localhost:5040/health
- Text Matcher: http://localhost:5041/health
- Users: http://localhost:5042/health
- Telegram Client: http://localhost:5050/health
- RabbitMQ UI: http://localhost:15672 (admin/admin)
### Вариант 2: Локальная разработка (VS Code)
```bash
# 1. Запустите инфраструктуру (RabbitMQ + базы данных)
docker-compose up nocr-rabbitmq nocr-text-matcher-db nocr-users-db -d
# 2. Создайте конфигурационные файлы для каждого сервиса
cd telegram-listener/src/Nocr.TelegramListener.Host
cp appsettings.Development.json.example appsettings.Development.json
# Повторите для всех сервисов
# 3. Заполните ваши секреты в appsettings.Development.json
# 4. Запустите сервисы через VS Code (F5)
```
**📖 Подробная документация:** См. [CONFIGURATION.md](CONFIGURATION.md)
---
## 🏗️ Архитектура взаимодействия
```
┌──────────────────┐ ┌───────────────┐ ┌──────────────────┐
│ Telegram Channel │────────▶│ Listener │────────▶│ Text Matcher │
│ (MTProto API) │ │ (Scan msgs) │ │ (Match rules) │
└──────────────────┘ └───────┬───────┘ └────────┬─────────┘
│ │
│ │
│ RabbitMQ │
│ Message Bus │
│ │
▼ ▼
┌───────────────┐ ┌──────────────────┐
│ Event Topics │ │ Telegram Bot │
& Queues │◀────────│ Client │
└───────────────┘ │ (Notifications) │
└──────────────────┘
┌──────────────────┐
│ Users Service │
│ (User Management)│
└──────────────────┘
```
**Паттерн взаимодействия:**
- **Event-driven**: Асинхронная обработка через RabbitMQ
- **REST API**: Синхронные запросы между telegram-client и другими сервисами
- **Clean Architecture**: Разделение на слои Host/AppServices/Core/Persistence
**📖 Подробная диаграмма:** См. [architecture.md](architecture.md)
---
## 🔧 Технологический стек
- **.NET 8** - Фреймворк для всех сервисов
- **ASP.NET Core** - Web API
- **Entity Framework Core** - ORM для MariaDB
- **WTelegramClient** - MTProto API для telegram-listener
- **Telegram.Bot** - Bot API для telegram-client
- **Rebus** - Message bus поверх RabbitMQ
- **MariaDB** - Базы данных для text-matcher и users
- **RabbitMQ** - Брокер сообщений
- **Docker & Docker Compose** - Контейнеризация
- **Drone CI** - CI/CD на Kubernetes
---
## 📦 CI/CD Pipeline
Проект использует **Drone CI** с 5 специализированными пайплайнами:
### 1. 🧪 Feature Validation
**Триггер:** Push в ветки `feature/*`, `fix/*`, `issues/*`
**Длительность:** ~3-5 минут
- Сборка всех сервисов
- Запуск тестов с Testcontainers
- Быстрая обратная связь перед PR
### 2. ✅ Main Validation
**Триггер:** Push в ветку `main`
**Длительность:** ~3-5 минут
- Валидация после мерджа в main
- Гарантия стабильности основной ветки
### 3. 📝 Contracts-Only Publish
**Триггер:** Tag с сообщением `contracts_only:<service>`
**Длительность:** ~2 минуты
- Публикация NuGet-пакетов контрактов
- Без сборки Docker-образов
```bash
git commit -m "contracts_only:telegram_listener - Add MessageEdited event"
git tag v1.2.4-contracts && git push origin v1.2.4-contracts
```
### 4. 🚀 Full Release
**Триггер:** Tag без специальных маркеров
**Длительность:** ~8-10 минут
1. Публикация всех контрактов (параллельно)
2. Сборка Docker-образов с Kaniko (параллельно)
3. Деплой в Kubernetes (для тегов `v*`)
```bash
git tag v1.3.0 && git push origin v1.3.0
```
### 5. ⚡ Deploy-Only
**Триггер:** Tag с сообщением `deploy_only:`
**Длительность:** ~1 минута
- Деплой существующих образов без пересборки
- Откат к предыдущей версии
```bash
git commit --allow-empty -m "deploy_only: Rollback to v1.2.9"
git tag v1.2.9-deploy && git push origin v1.2.9-deploy
```
**📖 Полная документация CI/CD:** См. [_deploy/README.md](_deploy/README.md)
---
## 🛠️ Команды разработки
### Сборка
```bash
# Сборка конкретного сервиса
cd telegram-listener && dotnet build
# Сборка всех сервисов
dotnet build telegram-listener/Nocr.TelegramListener.sln
dotnet build telegram-client/Nocr.TelegramClient.sln
dotnet build text-matcher/Nocr.TextMatcher.sln
dotnet build users/Nocr.Users.sln
```
### Тестирование
```bash
# Запуск тестов (text-matcher)
cd text-matcher && dotnet test
```
### Миграции БД
```bash
# Добавить новую миграцию
cd text-matcher
./src/Nocr.TextMatcher.Migrator/AddMigration.sh MyMigrationName
cd users
./src/Nocr.Users.Migrator/AddMigration.sh MyMigrationName
# Применение миграций автоматически при запуске через docker-compose
```
### Работа с субмодулями
```bash
# Инициализация субмодулей после клонирования
git submodule update --init --recursive
# Обновление всех субмодулей до последних коммитов
git submodule update --remote --merge
# Автокоммит изменений в субмодулях (скрипт)
./commit-submodules.sh "Update submodules"
```
---
## 📊 Порты сервисов
При запуске через Docker Compose:
| Сервис | Порт | Health Check |
|--------|------|--------------|
| telegram-listener | 5040 | http://localhost:5040/health |
| text-matcher | 5041 | http://localhost:5041/health |
| users | 5042 | http://localhost:5042/health |
| telegram-client | 5050 | http://localhost:5050/health |
| RabbitMQ AMQP | 5672 | - |
| RabbitMQ Management | 15672 | http://localhost:15672 |
| MariaDB (text-matcher) | 3316 | - |
| MariaDB (users) | 3326 | - |
---
## 🔐 Конфигурация
### Приоритет источников конфигурации
1. **appsettings.json** (базовые настройки)
2. **appsettings.{Environment}.json** (Development/DockerCompose/Production)
3. **User Secrets** (только для Development)
4. **Environment Variables** ⬆️ **НАИВЫСШИЙ ПРИОРИТЕТ**
### Важные правила
- ❌ **Никогда не коммитьте секреты** в git
- ✅ Используйте файлы `.example` как шаблоны
- ✅ Environment Variables переопределяют все остальное
- ✅ Docker Compose использует `.nocr.env` (gitignored)
- ✅ Kubernetes использует Secrets
### Режим отладки
Для вывода маскированных значений конфигурации:
```bash
export NOCR_DEBUG_MODE=true
```
**📖 Полное руководство:** См. [CONFIGURATION.md](CONFIGURATION.md)
---
## 📚 Документация
- [CONFIGURATION.md](CONFIGURATION.md) - Подробное руководство по конфигурации
- [_deploy/README.md](_deploy/README.md) - CI/CD пайплайны и деплой
- [architecture.md](architecture.md) - Архитектурная диаграмма
- [CLAUDE.md](CLAUDE.md) - Инструкции для Claude Code AI
### Документация сервисов
- [telegram-listener/README.md](telegram-listener/README.md)
- [telegram-client/README.md](telegram-client/README.md)
- [text-matcher/README.md](text-matcher/README.md)
- [users/README.md](users/README.md)
---
## 🤝 Рабочий процесс разработки
### Создание новой функции
```bash
# 1. Создайте feature-ветку
git checkout -b feature/add-filter
# 2. Работайте в субмодуле
cd telegram-listener
git checkout -b feature/add-filter
# Внесите изменения...
git commit -m "Add new filter"
git push origin feature/add-filter
# 3. Обновите родительский проект
cd ..
git add telegram-listener
git commit -m "Update telegram-listener: Add new filter"
git push origin feature/add-filter
# 4. Создайте PR и дождитесь прохождения CI
```
### Релиз новой версии
```bash
# 1. Убедитесь, что все изменения в main
git checkout main && git pull
# 2. Создайте тег
git tag v1.4.0 -m "Release v1.4.0: New features"
git push origin v1.4.0
# 3. Drone автоматически:
# - Опубликует NuGet-пакеты
# - Соберет Docker-образы
# - Задеплоит в Kubernetes (для v* тегов)
```
---
## 🐛 Troubleshooting
### Сервис не подключается к RabbitMQ
```bash
# 1. Проверьте, что RabbitMQ запущен
docker-compose ps nocr-rabbitmq
# 2. Включите режим отладки
export NOCR_DEBUG_MODE=true
# 3. Проверьте ConnectionString в логах
# Для Docker: должно быть nocr-rabbitmq:5672, НЕ localhost:5672
```
### Ошибка подключения к базе данных
```bash
# 1. Проверьте статус БД
docker-compose ps nocr-text-matcher-db
# 2. Подождите прохождения healthcheck (10-30 сек при первом запуске)
# 3. Проверьте порт в connection string:
# - Docker: nocr-text-matcher-db:3306
# - Local: localhost:3316
```
### Pipeline падает на тестах
```bash
# 1. Проверьте, что Testcontainers может подключиться к Docker
# В pipeline должна быть переменная DOCKER_HOST
# 2. Запустите тесты локально
cd text-matcher
dotnet test
# 3. Проверьте логи в Drone UI
```
---
## 📈 Мониторинг
### Проверка здоровья сервисов
```bash
# Локально
curl http://localhost:5040/health # telegram-listener
curl http://localhost:5041/health # text-matcher
curl http://localhost:5042/health # users
curl http://localhost:5050/health # telegram-client
# В Kubernetes (из deploy pod)
cd _deploy/scripts
./health-check.sh
```
### Просмотр логов
```bash
# Docker Compose
docker-compose logs -f telegram-listener
docker-compose logs -f text-matcher
# Kubernetes
kubectl logs -f deployment/telegram-listener -n nocr
kubectl logs -f deployment/text-matcher -n nocr
```
---
## 📝 История изменений (15-16 октября 2025)
### Основные улучшения
- ✨ **Новый CI/CD**: 5 специализированных пайплайнов Drone
- 🔧 **Исправления .drone.yml**: Корректная обработка YAML с многострочными командами
- 📦 **Contracts-only пайплайн**: Быстрая публикация контрактов без пересборки образов
- 🚀 **Deploy-only пайплайн**: Быстрый откат и деплой существующих образов
- 🔄 **MessageEdited события**: Отдельная обработка отредактированных сообщений
- 📝 **Автокоммит субмодулей**: Скрипт для упрощения работы с субмодулями
- ⚡ **Оптимизации**: Shared NuGet cache, параллельная сборка, Kaniko caching
### Исправленные проблемы
- 🐛 Исправлен парсинг YAML в `.drone.yml` (строка 126)
- 🐛 Корректная обработка `${DRONE_COMMIT_SHA:0:7}` в командах
- 🐛 Правильные отступы в многострочных bash-скриптах
- 🐛 Зависимости между этапами pipeline
---
## 📄 Лицензия
[Укажите лицензию вашего проекта]
## 👥 Авторы
[Укажите авторов проекта]
---
**Дополнительные ссылки:**
- [Drone CI](https://drone.musk.fun/nocr/flea)
- [Gitea Repository](https://gitea.musk.fun/nocr/flea)

379
_deploy/README.md Normal file
View File

@ -0,0 +1,379 @@
# 🚀 Документация CI/CD Pipeline NOCR
## 📋 Обзор
Проект NOCR использует современную многопайплайновую CI/CD систему на базе Drone CI, работающую на Kubernetes. Этот документ описывает 5 специализированных пайплайнов и способы их использования.
---
## 🎯 Архитектура пайплайнов
### Пайплайн 1: **Feature Validation (Валидация функциональных веток)**
**Триггер:** Push в ветки `feature/*`, `fix/*` или `issues/*`
**Назначение:** Быстрая обратная связь для разработчиков
**Длительность:** ~3-5 минут
**Что делает:**
- Клонирует репозиторий с субмодулями
- Восстанавливает все NuGet-пакеты (общий кэш)
- Собирает все 4 сервиса в Debug-режиме
- Запускает unit и integration тесты с Testcontainers
**Пример рабочего процесса:**
```bash
git checkout -b feature/add-new-filter
# Внесите изменения...
git add .
git commit -m "Add new filter functionality"
git push origin feature/add-new-filter
```
Drone автоматически запустит тесты. Проверьте результаты перед созданием PR.
---
### Пайплайн 2: **Main Validation (Валидация основной ветки)**
**Триггер:** Push в ветку `main`
**Назначение:** Валидация основной ветки после мерджа
**Длительность:** ~3-5 минут
**Что делает:**
- То же, что и Feature Validation
- Гарантирует, что ветка main всегда в рабочем состоянии
**Пример рабочего процесса:**
```bash
# После мерджа PR в main
# Пайплайн запускается автоматически
```
---
### Пайплайн 3: **Contracts-Only Publish (Публикация только контрактов)**
**Триггер:** Тег с сообщением коммита, содержащим `contracts_only:<service>`
**Назначение:** Быстрая публикация NuGet-пакетов контрактов без сборки Docker-образов
**Длительность:** ~2 минуты
**Что делает:**
- Упаковывает контракты указанного сервиса в NuGet-пакеты
- Публикует во внутреннем NuGet-фиде
- Пропускает сборку Docker-образов
**Пример рабочего процесса:**
```bash
# Обновите контракты telegram-listener
cd telegram-listener
# Внесите изменения в Async.Api.Contracts...
git add .
git commit -m "contracts_only:telegram_listener - Add MessageEdited event"
git push origin main
# Создайте тег
git tag v1.2.4-contracts
git push origin v1.2.4-contracts
```
**Поддерживаемые маркеры:**
- `contracts_only:telegram_listener`
- `contracts_only:text_matcher`
- `contracts_only:users`
---
### Пайплайн 4: **Full Release (Полный релиз)**
**Триггер:** Тег на main БЕЗ `contracts_only` или `deploy_only` в сообщении коммита
**Назначение:** Полный цикл релиза
**Длительность:** ~8-10 минут
**Что делает:**
1. **Этап 1:** Публикация всех контрактов в NuGet (параллельно)
2. **Этап 2:** Сборка всех Docker-образов с Kaniko (3 параллельных потока)
3. **Этап 3:** Деплой в Kubernetes (только для тегов, начинающихся с `v`)
**Пример рабочего процесса:**
```bash
# Готовы к релизу
git tag v1.3.0
git push origin v1.3.0
# Drone выполнит:
# 1. Публикацию контрактов
# 2. Сборку образов (с тегами v1.3.0, commit SHA, и latest)
# 3. Деплой в k8s (если тег начинается с 'v')
```
**Создаваемые теги образов:**
- `hub.musk.fun/k8s/nocr/telegram_listener:v1.3.0`
- `hub.musk.fun/k8s/nocr/telegram_listener:abc1234` (commit SHA)
- `hub.musk.fun/k8s/nocr/telegram_listener:latest`
---
### Пайплайн 5: **Deploy-Only (Только деплой)**
**Триггер:** Тег с сообщением коммита, содержащим `deploy_only: <version>`
**Назначение:** Быстрый деплой уже собранных образов
**Длительность:** ~1 минута
**Что делает:**
- Пропускает сборку
- Деплоит указанные образы в Kubernetes
- Полезно для отката или продвижения существующих образов
**ВАЖНО:** В сообщении коммита нужно указать версию образов для деплоя:
**Пример рабочего процесса:**
```bash
# Деплой существующих образов v1.2.9
git commit --allow-empty -m "deploy_only: v1.2.9"
git tag deploy-v1.2.9
git push origin deploy-v1.2.9
# Откат на предыдущую версию v1.2.8
git commit --allow-empty -m "deploy_only: v1.2.8"
git tag rollback-v1.2.8
git push origin rollback-v1.2.8
```
**Формат commit message:**
- `deploy_only: v1.2.9` - деплоит образы с тегом `v1.2.9`
- Скрипт извлекает версию из сообщения и использует соответствующие образы
- Образы должны существовать в registry (собраны ранее через full-release)
---
## 🛠️ Скрипты деплоя
Все скрипты деплоя находятся в `_deploy/scripts/`:
### `deploy.sh`
**Назначение:** Деплой сервисов в Kubernetes
**Использование:**
```bash
# Full release (использует TAG для образов)
./deploy.sh v1.3.0 abc1234
# Deploy only (переопределяет тег образа)
./deploy.sh deploy-v1.2.9 def5678 v1.2.9
```
**Параметры:**
1. `tag` - Git тег (обязательный)
2. `commit-sha` - SHA коммита (опциональный)
3. `image-tag-override` - Переопределение тега образа (опциональный, для deploy_only)
**Возможности:**
- Обновляет deployment-манифесты новыми тегами образов
- Применяет манифесты в кластер
- Ожидает завершения rollout с таймаутом
- Запускает проверки здоровья после деплоя
- Показывает статус подов
### `rollback.sh`
**Назначение:** Откат деплоев к предыдущей версии
**Использование:**
```bash
# Откат одного сервиса
./rollback.sh telegram-listener
# Откат всех сервисов
./rollback.sh all
```
**Возможности:**
- Показывает историю ревизий
- Выполняет kubectl rollout undo
- Ожидает завершения отката
- Запускает проверки здоровья после отката
### `health-check.sh`
**Назначение:** Проверка здоровья всех сервисов NOCR
**Использование:**
```bash
./health-check.sh
```
**Проверяет:**
- Статус подов (Running/Ready)
- Health-эндпоинты (/health)
- Последние события для упавших подов
---
## 📦 Оптимизации
### Общий NuGet-кэш
Все пайплайны используют общий временный volume для NuGet-пакетов:
- Первый `dotnet restore` скачивает пакеты
- Последующие сборки переиспользуют кэшированные пакеты
- **~60% быстрее**, чем индивидуальное восстановление для каждого сервиса
### Параллельное выполнение
- Публикация контрактов: 3 сервиса параллельно
- Сборка Docker: 3 параллельных потока
- Независимые операции никогда не блокируют друг друга
### Кэширование Kaniko
Все сборки Kaniko используют:
- `--cache=true` - Включено кэширование слоев
- `--cache-repo=hub.musk.fun/k8s/cache/*` - Общий репозиторий кэша
- `--compressed-caching=true` - Быстрая передача кэша
---
## 🧪 Поддержка Testcontainers
Пайплайны валидации Feature и Main включают Docker-in-Docker сервис для Testcontainers:
```yaml
services:
- name: docker
image: docker:27-dind
privileged: true
```
Тесты могут использовать Testcontainers для запуска реальных баз данных, очередей сообщений и т.д.
---
## 🔒 Необходимые секреты
Настройте эти секреты в Drone:
- `hub_username` - Имя пользователя Docker registry
- `hub_password` - Пароль Docker registry
- `nuget_musk_api_key` - API-ключ NuGet-фида
---
## 📊 Дерево решений пайплайна
```
Push в feature/* → Feature Validation (сборка + тесты)
Push в main → Main Validation (сборка + тесты)
Тег + "contracts_only:" → Contracts Publish
Тег + "deploy_only:" → Deploy Only
Тег (без маркеров) → Full Release (контракты → образы → деплой)
```
---
## 🎓 Лучшие практики
1. **Функциональные ветки**
- Всегда создавайте функциональные ветки для новой работы
- Дайте CI валидировать перед мерджем в main
2. **Изменения контрактов**
- Используйте `contracts_only:` для быстрого обновления контрактов
- Другие сервисы могут сразу обновить ссылки
3. **Процесс релиза**
- Создавайте теги только из ветки main
- Используйте семантическое версионирование (v1.2.3)
- Теги, начинающиеся с `v`, автоматически деплоятся в k8s
4. **Экстренный откат**
```bash
# Быстрый откат через deploy-only
git commit --allow-empty -m "deploy_only: v1.2.8"
git tag rollback-v1.2.8
git push origin rollback-v1.2.8
# Или используйте скрипт отката напрямую в кластере
kubectl exec -it deploy-pod -- bash
cd /flea/_deploy/scripts
./rollback.sh all
```
5. **Мониторинг деплоев**
- Следите за UI Drone для прогресса пайплайна
- Проверяйте логи подов: `kubectl logs -f deployment/telegram-listener -n nocr`
- Запускайте проверки здоровья: `./_deploy/scripts/health-check.sh`
---
## 🐛 Устранение неполадок
### Пайплайн завис на "Waiting for contracts"
**Причина:** Публикация контракта упала
**Решение:** Проверьте NuGet-фид, проверьте API-ключ
### Сборка Docker падает с ошибкой "unauthorized"
**Причина:** Неверные учетные данные registry
**Решение:** Обновите секреты `hub_username` и `hub_password`
### Тесты падают с ошибкой "Cannot connect to Docker daemon"
**Причина:** Testcontainers не может достичь Docker-in-Docker сервиса
**Решение:** Проверьте, что переменная окружения `DOCKER_HOST` установлена правильно
### Деплой падает с ошибкой "ImagePullBackOff"
**Причина:** Образ не найден в registry
**Решение:** Проверьте, что образ был собран и запушен успешно на предыдущем этапе
### YAML-ошибки в .drone.yml
**Причина:** Неправильное форматирование многострочных команд или двоеточие в переменных
**Решение:**
- Используйте блоки `|` для многострочных команд
- Экранируйте переменные с двоеточиями (например, `"${VAR:0:7}"`)
- Проверьте YAML: `python3 -c "import yaml; list(yaml.safe_load_all(open('.drone.yml')))"`
---
## 📚 Дополнительные ресурсы
- [Документация Drone CI](https://docs.drone.io/)
- [Документация Kaniko](https://github.com/GoogleContainerTools/kaniko)
- [Testcontainers для .NET](https://dotnet.testcontainers.org/)
- [Kubernetes Deployments](https://kubernetes.io/ru/docs/concepts/workloads/controllers/deployment/)
---
## 🔄 История изменений (15-16 октября 2025)
### Исправлена критическая проблема с YAML
**Проблема:**
- Ошибка `yaml: line 126: mapping values are not allowed in this context`
- Многострочные команды `/kaniko/executor` содержали переменные с двоеточием (`${DRONE_COMMIT_SHA:0:7}`)
- YAML парсер интерпретировал двоеточие как разделитель ключ:значение
**Решение:**
1. Обернули все команды `/kaniko/executor` в многострочные блоки с `|`
2. Добавили обратные слэши `\` для продолжения команды
3. Экранировали аргументы с двоеточиями в кавычках
**Пример:**
```yaml
# До (неправильно):
commands:
- /kaniko/executor
--destination=hub.musk.fun/image:${DRONE_COMMIT_SHA:0:7}
# После (правильно):
commands:
- |
/kaniko/executor \
--destination=hub.musk.fun/image:${DRONE_COMMIT_SHA:0:7}
```
### Добавлены новые функции
- ✨ Контракты-только пайплайн для быстрой итерации
- 🚀 Деплой-только пайплайн для откатов
- 🔄 Обработка MessageEdited событий
- 📝 Скрипт автокоммита субмодулей
- ⚡ Оптимизированный общий NuGet-кэш
- 🔧 Параллельная сборка и деплой
---
## 📞 Поддержка
Если у вас возникли проблемы с CI/CD:
1. Проверьте логи в Drone UI: https://drone.musk.fun/nocr/flea
2. Проверьте статус подов: `kubectl get pods -n nocr`
3. Запустите health-check: `./health-check.sh`
4. Проверьте логи контейнеров: `kubectl logs -f <pod-name> -n nocr`
Для вопросов по инфраструктуре обращайтесь к DevOps-команде.

View File

@ -59,10 +59,10 @@ kubectl create secret generic wtelegram-client \
# Next secret got telegram bot auth.
kubectl create secret generic telegram-bot \
--from-literal=token=<value> \
--namespace nocr
--from-literal=token=<your-telegram-token> \
--from-literal=feedback_receiver_id_0=<your_telegram_channel_id> \
--from-literal=update_receiver_id_0=<your_telegram_channel_id> \
--namespace nocr
# 05. installing persistant rdbs engine

View File

@ -7,7 +7,7 @@ spec:
replicas: 1
selector:
matchLabels:
app: telegram-listener
app: telegram-listener
template:
metadata:
labels:
@ -27,22 +27,22 @@ spec:
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: host
key: host
- name: ASPNETCORE_rmqc_username
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: username
key: username
- name: ASPNETCORE_rmqc_password
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: password
key: password
- name: ASPNETCORE_rmqc_port
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: port
key: port
- name: RebusRabbitMqOptions__ConnectionString
value: "amqp://$(ASPNETCORE_rmqc_username):$(ASPNETCORE_rmqc_password)@$(ASPNETCORE_rmqc_host):$(ASPNETCORE_rmqc_port)/"
- name: WTelegramClientOptions__ApiId
@ -65,15 +65,26 @@ spec:
mountPath: /app/.secrets
- name: rw-files
mountPath: /tmp/rw-files
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 30
timeoutSeconds: 5
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
initContainers:
- name: "update-mysql-init"
image: "busybox:latest"
command: [
"sh",
"-ce",
"cp /app/.secrets/WTelegram.session /tmp/rw-files/WTelegram.session &&
chmod a+w /tmp/rw-files/WTelegram.session &&
echo The TL Container is Running ; sleep 5 " ]
command:
- "sh"
- "-ce"
- "cp /app/.secrets/WTelegram.session /tmp/rw-files/WTelegram.session && chmod a+w /tmp/rw-files/WTelegram.session && echo The TL Container is Running ; sleep 5"
volumeMounts:
- name: secrets
mountPath: /app/.secrets
@ -85,6 +96,7 @@ spec:
secretName: secretfiles
- name: rw-files
emptyDir: {}
---
apiVersion: apps/v1
kind: Deployment
@ -132,22 +144,22 @@ spec:
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: host
key: host
- name: rmqc_username
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: username
key: username
- name: rmqc_password
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: password
key: password
- name: rmqc_port
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: port
key: port
- name: RebusRabbitMqOptions__ConnectionString
value: "amqp://$(rmqc_username):$(rmqc_password)@$(rmqc_host):$(rmqc_port)/"
- name: mariadb_host
@ -163,6 +175,7 @@ spec:
key: mariadb-root-password
- name: ConnectionStrings__TextMatcherContext
value: "server=$(mariadb_host);port=3306;database=$(mariadb_database);uid=$(mariadb_user);pwd=$(mariadb_password)"
---
apiVersion: v1
kind: Service
@ -176,12 +189,13 @@ spec:
- protocol: TCP
port: 80
targetPort: 8080
type: NodePort # Change the service type to NodePort
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: users
name: users
namespace: nocr
spec:
replicas: 1
@ -233,6 +247,7 @@ spec:
key: mariadb-root-password
- name: ConnectionStrings__UsersContext
value: "server=$(mariadb_host);port=3306;database=$(mariadb_database);uid=$(mariadb_user);pwd=$(mariadb_password)"
---
apiVersion: v1
kind: Service
@ -246,13 +261,13 @@ spec:
- protocol: TCP
port: 80
targetPort: 8080
type: NodePort # Change the service type to NodePort
type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: telegram-client
name: telegram-client
namespace: nocr
spec:
replicas: 1
@ -267,7 +282,7 @@ spec:
imagePullSecrets:
- name: hubcred
containers:
- name: users
- name: telegram-client
image: hub.musk.fun/k8s/nocr/telegram_client:latest
env:
- name: "ASPNETCORE_ENVIRONMENT"
@ -276,22 +291,22 @@ spec:
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: host
key: host
- name: rmqc_username
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: username
key: username
- name: rmqc_password
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: password
key: password
- name: rmqc_port
valueFrom:
secretKeyRef:
name: rmqc-default-user
key: port
key: port
- name: RebusRabbitMqOptions__ConnectionString
value: "amqp://$(rmqc_username):$(rmqc_password)@$(rmqc_host):$(rmqc_port)/"
- name: mariadb_host
@ -305,3 +320,15 @@ spec:
secretKeyRef:
name: telegram-bot
key: token
- name: AdministrationOptions__EnableUpdateTelegramLogging
value: "true"
- name: AdministrationOptions__FeedbackReceiverIds__0
valueFrom:
secretKeyRef:
name: telegram-bot
key: feedback_receiver_id_0
- name: AdministrationOptions__UpdateReceiverIds__0
valueFrom:
secretKeyRef:
name: telegram-bot
key: update_receiver_id_0

113
_deploy/scripts/deploy.sh Executable file
View File

@ -0,0 +1,113 @@
#!/bin/bash
set -e
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 🚀 Nocr Services Deployment Script
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Usage: ./deploy.sh <tag> [commit-sha] [image-tag-override]
# Examples:
# Full release: ./deploy.sh v1.2.3 abc1234
# Deploy only: ./deploy.sh deploy-v1.2.3 def5678 v1.2.3
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TAG=${1:-latest}
COMMIT_SHA=${2:-}
NAMESPACE="nocr"
DEPLOYMENT_FILE="../k8s/deployment.yaml"
IMAGE_TAG_OVERRIDE=${3:-}
# Determine image tag to use
# Priority: 1) IMAGE_TAG_OVERRIDE (from command line), 2) TAG
if [ -n "$IMAGE_TAG_OVERRIDE" ]; then
IMAGE_TAG=$IMAGE_TAG_OVERRIDE
else
IMAGE_TAG=${TAG}
fi
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🚀 Starting deployment of Nocr services"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📦 Tag: $TAG"
echo "🏷️ Image Tag: $IMAGE_TAG"
if [ -n "$COMMIT_SHA" ]; then
echo "📝 Commit SHA: $COMMIT_SHA"
fi
echo "🎯 Namespace: $NAMESPACE"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Check if kubectl is available
if ! command -v kubectl &> /dev/null; then
echo "❌ kubectl not found. Please install kubectl."
exit 1
fi
# Check cluster connection
echo "🔍 Checking connection to Kubernetes cluster..."
if ! kubectl cluster-info &> /dev/null; then
echo "❌ Cannot connect to Kubernetes cluster."
exit 1
fi
echo "✅ Connected to cluster"
# Create temporary deployment file with updated image tags
TEMP_DEPLOYMENT=$(mktemp)
cp "$DEPLOYMENT_FILE" "$TEMP_DEPLOYMENT"
echo "🔧 Updating image tags in deployment manifests..."
# Update image tags for all services
sed -i "s|hub.musk.fun/k8s/nocr/telegram_listener:.*|hub.musk.fun/k8s/nocr/telegram_listener:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
sed -i "s|hub.musk.fun/k8s/nocr/telegram_client:.*|hub.musk.fun/k8s/nocr/telegram_client:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
sed -i "s|hub.musk.fun/k8s/nocr/text_matcher:.*|hub.musk.fun/k8s/nocr/text_matcher:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
sed -i "s|hub.musk.fun/k8s/nocr/text_matcher_migrator:.*|hub.musk.fun/k8s/nocr/text_matcher_migrator:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
sed -i "s|hub.musk.fun/k8s/nocr/users:.*|hub.musk.fun/k8s/nocr/users:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
sed -i "s|hub.musk.fun/k8s/nocr/users_migrator:.*|hub.musk.fun/k8s/nocr/users_migrator:${IMAGE_TAG}|g" "$TEMP_DEPLOYMENT"
echo "✅ Image tags updated"
# Apply deployments
echo "📦 Applying deployment manifests to cluster..."
kubectl apply -f "$TEMP_DEPLOYMENT" -n "$NAMESPACE"
# Clean up temp file
rm "$TEMP_DEPLOYMENT"
echo "✅ Manifests applied"
echo ""
# Force restart deployments to pull latest images
echo "🔄 Restarting deployments to pull new images..."
DEPLOYMENTS=("telegram-listener" "text-matcher" "users" "telegram-client")
for deployment in "${DEPLOYMENTS[@]}"; do
kubectl rollout restart deployment/"$deployment" -n "$NAMESPACE"
echo " ↻ Restarted $deployment"
done
echo ""
echo "⏳ Waiting for rollouts to complete..."
echo ""
TIMEOUT="300s"
for deployment in "${DEPLOYMENTS[@]}"; do
echo "🔄 Rolling out $deployment..."
if kubectl rollout status deployment/"$deployment" -n "$NAMESPACE" --timeout="$TIMEOUT"; then
echo "$deployment rolled out successfully"
else
echo "$deployment rollout failed or timed out"
echo "🔍 Pod status:"
kubectl get pods -n "$NAMESPACE" -l app="$deployment"
echo "🔍 Recent events:"
kubectl get events -n "$NAMESPACE" --sort-by='.lastTimestamp' | grep "$deployment" | tail -10
exit 1
fi
echo ""
done
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "✅ Deployment completed successfully!"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📊 Pod status:"
kubectl get pods -n "$NAMESPACE" -o wide
echo ""
echo "🔍 Running health checks..."
bash "$(dirname "$0")/health-check.sh"

108
_deploy/scripts/health-check.sh Executable file
View File

@ -0,0 +1,108 @@
#!/bin/bash
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# 🏥 Nocr Services Health Check Script
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Checks health endpoints and pod status for all Nocr services
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
NAMESPACE="nocr"
FAILED_CHECKS=0
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🏥 Running health checks for Nocr services"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Check if kubectl is available
if ! command -v kubectl &> /dev/null; then
echo "❌ kubectl not found. Please install kubectl."
exit 1
fi
# Function to check pod health
check_pod_health() {
local deployment=$1
local service_name=$2
echo ""
echo "🔍 Checking $service_name..."
# Get pod status
PODS=$(kubectl get pods -n "$NAMESPACE" -l app="$deployment" -o json)
# Check if any pods exist
POD_COUNT=$(echo "$PODS" | jq -r '.items | length')
if [ "$POD_COUNT" -eq 0 ]; then
echo "❌ No pods found for $service_name"
FAILED_CHECKS=$((FAILED_CHECKS + 1))
return 1
fi
# Check pod status
RUNNING_PODS=$(echo "$PODS" | jq -r '.items[] | select(.status.phase=="Running") | .metadata.name' | wc -l)
READY_PODS=$(echo "$PODS" | jq -r '.items[] | select(.status.conditions[] | select(.type=="Ready" and .status=="True")) | .metadata.name' | wc -l)
echo " 📊 Pods: $RUNNING_PODS running, $READY_PODS ready (total: $POD_COUNT)"
if [ "$RUNNING_PODS" -eq 0 ]; then
echo " ❌ No running pods for $service_name"
kubectl get pods -n "$NAMESPACE" -l app="$deployment"
FAILED_CHECKS=$((FAILED_CHECKS + 1))
return 1
fi
if [ "$READY_PODS" -eq 0 ]; then
echo " ❌ No ready pods for $service_name"
kubectl get pods -n "$NAMESPACE" -l app="$deployment"
echo " 🔍 Pod details:"
kubectl describe pods -n "$NAMESPACE" -l app="$deployment" | grep -A 20 "Conditions:"
FAILED_CHECKS=$((FAILED_CHECKS + 1))
return 1
fi
# Try to check health endpoint if service has one
case "$deployment" in
"telegram-listener"|"text-matcher"|"users"|"telegram-client")
POD_NAME=$(echo "$PODS" | jq -r '.items[0].metadata.name')
if [ -n "$POD_NAME" ]; then
echo " 🌐 Checking /health endpoint..."
if kubectl exec -n "$NAMESPACE" "$POD_NAME" -- curl -f -s http://localhost:8080/health > /dev/null 2>&1; then
echo " ✅ Health endpoint responding"
else
echo " ⚠️ Health endpoint not responding (might be warming up)"
fi
fi
;;
esac
echo "$service_name is healthy"
return 0
}
# Check each service
SERVICES=(
"telegram-listener:Telegram Listener"
"text-matcher:Text Matcher"
"users:Users Service"
"telegram-client:Telegram Client"
)
for service_info in "${SERVICES[@]}"; do
IFS=':' read -r deployment service_name <<< "$service_info"
check_pod_health "$deployment" "$service_name"
done
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📊 Summary:"
kubectl get pods -n "$NAMESPACE" -o wide
echo ""
if [ $FAILED_CHECKS -eq 0 ]; then
echo "✅ All health checks passed!"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
exit 0
else
echo "$FAILED_CHECKS health check(s) failed"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
exit 1
fi

105
_deploy/scripts/rollback.sh Executable file
View File

@ -0,0 +1,105 @@
#!/bin/bash
set -e
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# ⏮️ Nocr Services Rollback Script
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
# Usage: ./rollback.sh [deployment-name]
# Example: ./rollback.sh telegram-listener
# ./rollback.sh all (rolls back all deployments)
# ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
DEPLOYMENT=${1:-all}
NAMESPACE="nocr"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "⏮️ Starting rollback of Nocr services"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🎯 Deployment: $DEPLOYMENT"
echo "🎯 Namespace: $NAMESPACE"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Check if kubectl is available
if ! command -v kubectl &> /dev/null; then
echo "❌ kubectl not found. Please install kubectl."
exit 1
fi
# Check cluster connection
echo "🔍 Checking connection to Kubernetes cluster..."
if ! kubectl cluster-info &> /dev/null; then
echo "❌ Cannot connect to Kubernetes cluster."
exit 1
fi
echo "✅ Connected to cluster"
# Function to rollback a single deployment
rollback_deployment() {
local dep=$1
echo ""
echo "⏮️ Rolling back deployment: $dep"
# Check if deployment exists
if ! kubectl get deployment "$dep" -n "$NAMESPACE" &> /dev/null; then
echo "⚠️ Deployment $dep not found in namespace $NAMESPACE"
return 1
fi
# Show current revision
echo "📊 Current revision:"
kubectl rollout history deployment/"$dep" -n "$NAMESPACE" | tail -5
# Perform rollback
echo "🔄 Rolling back..."
if kubectl rollout undo deployment/"$dep" -n "$NAMESPACE"; then
echo "✅ Rollback command issued for $dep"
# Wait for rollback to complete
echo "⏳ Waiting for rollback to complete..."
if kubectl rollout status deployment/"$dep" -n "$NAMESPACE" --timeout=300s; then
echo "$dep rolled back successfully"
else
echo "$dep rollback failed or timed out"
echo "🔍 Pod status:"
kubectl get pods -n "$NAMESPACE" -l app="$dep"
return 1
fi
else
echo "❌ Failed to rollback $dep"
return 1
fi
}
# Rollback deployments
DEPLOYMENTS=("telegram-listener" "text-matcher" "users" "telegram-client")
if [ "$DEPLOYMENT" = "all" ]; then
echo "🔄 Rolling back all deployments..."
FAILED=0
for dep in "${DEPLOYMENTS[@]}"; do
if ! rollback_deployment "$dep"; then
FAILED=$((FAILED + 1))
fi
done
if [ $FAILED -gt 0 ]; then
echo ""
echo "$FAILED deployment(s) failed to rollback"
exit 1
fi
else
# Rollback single deployment
if ! rollback_deployment "$DEPLOYMENT"; then
exit 1
fi
fi
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "✅ Rollback completed successfully!"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📊 Pod status:"
kubectl get pods -n "$NAMESPACE" -o wide
echo ""
echo "🔍 Running health checks..."
bash "$(dirname "$0")/health-check.sh"

View File

@ -1,47 +1,389 @@
# Архитектура системы NOCR
Диаграмма взаимодействия между сервисами системы мониторинга Telegram-каналов.
```mermaid
classDiagram
TelegramListener <|-- Bus
TextMatcher <|-- TelegramClient
TextMatcher <|-- Bus
Bus <|-- TextMatcher
TelegramClient <|-- Bus
Users <|-- TelegramClient
Bus <|-- TelegramListener
TelegramListener --|> Bus : publishes
Bus --|> TextMatcher : subscribes
TextMatcher --|> Bus : publishes
Bus --|> TelegramClient : subscribes
TelegramClient --|> Users : REST API
TelegramClient --|> TextMatcher : REST API
class Bus{
+2024-03-28 18:27:50.551 [info] Installing honnef.co/go/tools/cmd/staticcheck@latest (/Users/nazarovsa/go/bin/staticcheck) SUCCEEDED
<<RabbitMQ Message Broker>>
+MessageReceived events
+MessageEdited events
+TextSubscriptionMatched events
+TextSubscriptionUpdated events
}
class TelegramListener {
+MessageReceived
-TextMatcher.TextMatchCreated
<<Сканер каналов>>
+publishes: MessageReceived
+publishes: MessageEdited
---
WTelegramClient (MTProto API)
Хранение сессий пользователей
}
class TextMatcher {
+TextMatchCreated
+TextMatchMatched
-TelegramListener.MessageReceived
+long Create(CreateTextMatchRequest request)
<<Обработчик совпадений>>
+publishes: TextSubscriptionMatched
+publishes: TextSubscriptionUpdated
-subscribes: MessageReceived
-subscribes: MessageEdited
---
REST API:
+long Create(CreateTextMatchRequest)
+TextMatchData? GetById(long id)
+TextMatchData[] GetByUserId(long userId)
+void Delete(long id)
+void Activate(long id)
+void Disable(long id)
---
БД: История совпадений (MariaDB)
Проверка регулярных выражений
}
class Users {
+long Create(CreateUserRequest request)
<<Управление пользователями>>
---
REST API:
+long Create(CreateUserRequest)
+UserData? GetById(long id)
+UserData? GetByIdentity(UserIdentityType identityType, string identity)
+UserData? GetByIdentity(type, identity)
---
БД: Пользователи и настройки (MariaDB)
}
class TelegramClient {
-TextMatcher.TextMatchMatched
<<Telegram Bot интерфейс>>
-subscribes: TextSubscriptionMatched
-subscribes: TextSubscriptionUpdated
---
Telegram.Bot (Bot API)
Обработка команд пользователей
Отправка уведомлений
---
REST клиенты:
-Users.Create()
-Users.GetById()
-Users.GetByIdentity()
-TextMatcher.GetByUserId()
-TextMatcher.Create()
}
```
```
## Описание компонентов
### 1. TelegramListener (Сканер каналов)
**Технологии:** .NET 8, WTelegramClient, Rebus
**Ответственность:**
- Подключение к Telegram через MTProto API
- Подписка на открытые каналы и чаты
- Получение новых и отредактированных сообщений
- Публикация событий в RabbitMQ
**Публикуемые события:**
- `MessageReceived` - новое сообщение получено
- `MessageEdited` - сообщение отредактировано
**Примечание:** WTelegram отправляет оба события (`UpdateNewChannelMessage` и `UpdateEditChannelMessage`) для одного и того же сообщения. TelegramListener публикует отдельные события, чтобы избежать дублирования уведомлений.
---
### 2. TextMatcher (Обработчик совпадений)
**Технологии:** .NET 8, Entity Framework Core, MariaDB, Rebus
**Ответственность:**
- Подписка на события сообщений от TelegramListener
- Проверка совпадений с пользовательскими подписками (regex, ключевые слова)
- Хранение истории совпадений в БД
- Публикация событий о найденных совпадениях
- REST API для управления подписками
**Подписывается на события:**
- `MessageReceived` - проверка нового сообщения
- `MessageEdited` - обновление ранее найденного совпадения
**Публикуемые события:**
- `TextSubscriptionMatched` - новое совпадение найдено
- `TextSubscriptionUpdated` - обновление ранее найденного совпадения
**REST API эндпоинты:**
- `POST /api/textmatch` - Создать новую подписку
- `GET /api/textmatch/{id}` - Получить подписку по ID
- `GET /api/textmatch/user/{userId}` - Получить все подписки пользователя
- `DELETE /api/textmatch/{id}` - Удалить подписку
- `POST /api/textmatch/{id}/activate` - Активировать подписку
- `POST /api/textmatch/{id}/disable` - Отключить подписку
---
### 3. TelegramClient (Telegram Bot)
**Технологии:** .NET 8, Telegram.Bot, Rebus, RestEase
**Ответственность:**
- Интерфейс бота для взаимодействия с пользователями
- Обработка команд бота (/start, /subscribe, /list и т.д.)
- Подписка на события совпадений
- Отправка уведомлений пользователям
- Интеграция с Users и TextMatcher через REST API
**Подписывается на события:**
- `TextSubscriptionMatched` - отправка уведомления о новом совпадении
- `TextSubscriptionUpdated` - уведомление об обновлении
**REST клиенты:**
- **Users API:** управление пользователями
- **TextMatcher API:** управление подписками
---
### 4. Users (Управление пользователями)
**Технологии:** .NET 8, Entity Framework Core, MariaDB
**Ответственность:**
- CRUD операции для пользователей
- Хранение предпочтений и настроек
- REST API для других сервисов
**REST API эндпоинты:**
- `POST /api/users` - Создать нового пользователя
- `GET /api/users/{id}` - Получить пользователя по ID
- `GET /api/users/identity/{type}/{identity}` - Найти пользователя по Telegram ID
---
### 5. RabbitMQ (Message Bus)
**Технологии:** RabbitMQ 3.x, Rebus
**Ответственность:**
- Асинхронная доставка событий между сервисами
- Гарантия доставки сообщений
- Очереди и обмены (exchanges)
**Настройка:**
- **Direct Exchange:** `nocr.direct` - для точечной маршрутизации
- **Topic Exchange:** `nocr.topics` - для публикации/подписки по топикам
- **Очереди:** Каждый сервис имеет свою input queue
---
## Паттерны взаимодействия
### Event-Driven Architecture (Асинхронная)
```
TelegramListener → RabbitMQ → TextMatcher → RabbitMQ → TelegramClient
```
**Преимущества:**
- Слабая связанность сервисов
- Масштабируемость (можно добавить несколько обработчиков)
- Отказоустойчивость (сообщения не теряются при падении сервиса)
### Request-Response (Синхронная)
```
TelegramClient → REST API → Users
TelegramClient → REST API → TextMatcher
```
**Использование:**
- Создание/получение данных пользователей
- Управление подписками
- Запросы, требующие немедленного ответа
---
## Потоки данных
### Основной поток (Мониторинг и уведомления)
1. **Сканирование:**
- TelegramListener подключается к Telegram MTProto
- Получает новые сообщения из каналов
- Публикует `MessageReceived` в RabbitMQ
2. **Обработка:**
- TextMatcher получает `MessageReceived`
- Проверяет сообщение на совпадение с подписками
- Сохраняет историю в БД
- Публикует `TextSubscriptionMatched` в RabbitMQ
3. **Уведомление:**
- TelegramClient получает `TextSubscriptionMatched`
- Получает данные пользователя из Users API
- Отправляет уведомление через Telegram Bot API
### Поток редактирования сообщений
1. **Обновление:**
- TelegramListener получает `UpdateEditChannelMessage`
- Публикует `MessageEdited` в RabbitMQ
2. **Повторная проверка:**
- TextMatcher получает `MessageEdited`
- Проверяет, было ли это сообщение ранее найдено
- Если было, публикует `TextSubscriptionUpdated`
3. **Уведомление об обновлении:**
- TelegramClient получает `TextSubscriptionUpdated`
- Отправляет обновленное уведомление пользователю
---
## Clean Architecture
Каждый сервис следует принципам Clean Architecture:
```
┌─────────────────────────────────────────┐
│ Host (API / Entry Point) │ ← ASP.NET Core, Controllers
├─────────────────────────────────────────┤
│ AppServices (Business Logic) │ ← Use Cases, Handlers
├─────────────────────────────────────────┤
│ Core (Domain / Shared) │ ← Entities, Interfaces
├─────────────────────────────────────────┤
│ Persistence (Data Access) │ ← EF Core, Repositories
├─────────────────────────────────────────┤
│ Async.Api.Contracts (Event Contracts) │ ← RabbitMQ Events (NuGet)
├─────────────────────────────────────────┤
│ Api.Contracts (REST Contracts) │ ← DTOs, Requests (NuGet)
├─────────────────────────────────────────┤
│ Migrator (Database Migrations) │ ← EF Core Migrations
└─────────────────────────────────────────┘
```
**Преимущества:**
- Независимость от фреймворков
- Тестируемость
- Независимость от UI/Database
- Переиспользование контрактов через NuGet
---
## Базы данных
### TextMatcher Database (MariaDB)
**Порт:** 3316 (локально), 3306 (Docker/K8s)
**Таблицы:**
- `TextMatches` - Пользовательские подписки
- `MatchHistory` - История найденных совпадений
- `__EFMigrationsHistory` - История миграций EF Core
### Users Database (MariaDB)
**Порт:** 3326 (локально), 3306 (Docker/K8s)
**Таблицы:**
- `Users` - Пользователи системы
- `UserPreferences` - Настройки пользователей
- `__EFMigrationsHistory` - История миграций EF Core
---
## Развертывание
### Docker Compose (Локальная разработка)
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ telegram- │ │ telegram- │ │ text- │ │ users │
│ listener │ │ client │ │ matcher │ │ │
│ :5040 │ │ :5050 │ │ :5041 │ │ :5042 │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │ │
└────────────────┴────────────────┴────────────────┘
┌─────────┴─────────┐
│ RabbitMQ │
│ :5672, :15672 │
└───────────────────┘
┌───────────────┴───────────────┐
│ │
┌───────┴────────┐ ┌────────┴────────┐
│ text-matcher- │ │ users-db │
│ db :3316 │ │ :3326 │
└────────────────┘ └─────────────────┘
```
### Kubernetes (Production)
- **Namespace:** nocr
- **Services:** ClusterIP для внутренней коммуникации
- **Deployments:** С health checks и resource limits
- **Secrets:** Для Telegram API credentials и Bot tokens
- **ConfigMaps:** Для конфигурации окружения
---
## Мониторинг и наблюдаемость
### Health Checks
Каждый сервис предоставляет `/health` эндпоинт:
- **telegram-listener:** http://localhost:5040/health
- **text-matcher:** http://localhost:5041/health
- **users:** http://localhost:5042/health
- **telegram-client:** http://localhost:5050/health
### Логирование
- Структурированное логирование через Serilog
- Логи выводятся в stdout для Docker/K8s
- Централизованное логирование через kubectl logs
### Метрики
- RabbitMQ Management UI: http://localhost:15672
- Мониторинг очередей и exchanges
- Просмотр активных подключений
---
## Масштабирование
### Горизонтальное масштабирование
**Можно масштабировать:**
- ✅ **TextMatcher** - несколько инстансов обрабатывают сообщения параллельно
- ✅ **TelegramClient** - несколько инстансов отправляют уведомления параллельно
- ✅ **Users** - stateless API, можно масштабировать без ограничений
**Ограничения:**
- ⚠️ **TelegramListener** - один инстанс на одну Telegram сессию (ограничение WTelegram)
### Вертикальное масштабирование
- Увеличение CPU/RAM для сервисов
- Оптимизация запросов к БД
- Настройка connection pools
---
## Безопасность
### Секреты
- Telegram API credentials хранятся в K8s Secrets
- Bot tokens хранятся в K8s Secrets
- Database credentials в environment variables
### Сетевая безопасность
- Все сервисы работают в приватной сети
- Только TelegramClient имеет доступ к внешним API
- RabbitMQ защищен credentials
### Валидация
- Входящие данные валидируются в AppServices
- REST API использует Data Annotations
- Event contracts строго типизированы
---
## Дополнительная информация
- **[README.md](README.md)** - Общая документация проекта
- **[CONFIGURATION.md](CONFIGURATION.md)** - Руководство по конфигурации
- **[_deploy/README.md](_deploy/README.md)** - CI/CD и деплой
- **[CLAUDE.md](CLAUDE.md)** - Инструкции для AI-ассистента
---
оследнее обновление: 16 октября 2025_

91
commit-all.sh Executable file
View File

@ -0,0 +1,91 @@
#!/bin/bash
# Check if commit message is provided
if [ -z "$1" ]; then
echo "Error: Commit message is required"
echo "Usage: ./commit-all.sh \"Your commit message\""
exit 1
fi
COMMIT_MSG="$1"
# Get list of submodules
SUBMODULES=$(git config --file .gitmodules --get-regexp path | awk '{ print $2 }')
if [ -z "$SUBMODULES" ]; then
echo "No submodules found in this repository"
exit 0
fi
echo "🚀 Starting commit and push for all submodules..."
echo "📝 Commit message: $COMMIT_MSG"
echo ""
# Counter for tracking results
SUCCESS_COUNT=0
FAILED_COUNT=0
SKIPPED_COUNT=0
# Iterate through each submodule
for submodule in $SUBMODULES; do
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📦 Processing submodule: $submodule"
if [ ! -d "$submodule" ]; then
echo "⚠️ Warning: Submodule directory not found, skipping..."
SKIPPED_COUNT=$((SKIPPED_COUNT + 1))
continue
fi
cd "$submodule" || {
echo "❌ Failed to enter directory $submodule"
FAILED_COUNT=$((FAILED_COUNT + 1))
cd - > /dev/null
continue
}
# Check if there are any changes
if [ -z "$(git status --porcelain)" ]; then
echo " No changes to commit, skipping..."
SKIPPED_COUNT=$((SKIPPED_COUNT + 1))
cd - > /dev/null
continue
fi
# Add all changes
echo " → git add -A"
git add -A
# Commit changes
echo " → git commit"
if git commit -m "$COMMIT_MSG"; then
# Push to main branch
echo " → git push origin main"
if git push origin main; then
echo "✅ Successfully committed and pushed"
SUCCESS_COUNT=$((SUCCESS_COUNT + 1))
else
echo "❌ Failed to push"
FAILED_COUNT=$((FAILED_COUNT + 1))
fi
else
echo "❌ Failed to commit"
FAILED_COUNT=$((FAILED_COUNT + 1))
fi
cd - > /dev/null
echo ""
done
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📊 Summary:"
echo " ✅ Success: $SUCCESS_COUNT"
echo " ❌ Failed: $FAILED_COUNT"
echo " ⏭️ Skipped: $SKIPPED_COUNT"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
if [ $FAILED_COUNT -gt 0 ]; then
exit 1
fi
exit 0

View File

@ -1,5 +1,3 @@
version: "3"
services:
nocr-telegram-client:
container_name: nocr-telegram-client
@ -8,18 +6,48 @@ services:
context: telegram-client
dockerfile: src/Nocr.TelegramClient.Host/Dockerfile
ports:
- 4999:8080
- 5050:8080
# IMPORTANT: Create .nocr.env file in project root with your secrets
# See .nocr.env.example for reference
env_file:
- .nocr.env
environment:
ASPNETCORE_ENVIRONMENT: DockerCompose
depends_on:
nocr-rabbitmq:
condition: service_healthy
#nocr-users:
# condition: service_healthy
#nocr-text-matcher:
# condition: service_healthy
#nocr-telegram-listener:
# condition: service_healthy
nocr-telegram-listener:
condition: service_healthy
nocr-users:
condition: service_healthy
nocr-text-matcher:
condition: service_healthy
nocr-telegram-listener:
container_name: nocr-telegram-listener
image: nocr-telegram-listener:latest
build:
context: telegram-listener
dockerfile: src/Nocr.TelegramListener.Host/Dockerfile
ports:
- 5040:8080
# IMPORTANT: Create .nocr.env file in project root with your secrets
# See .nocr.env.example for reference
env_file:
- .nocr.env
environment:
ASPNETCORE_ENVIRONMENT: DockerCompose
session_pathname: '/app/WTelegram.session'
volumes:
- ./WTelegram.session:/app/WTelegram.session
depends_on:
nocr-rabbitmq:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
nocr-text-matcher:
container_name: nocr-text-matcher
@ -28,7 +56,11 @@ services:
context: text-matcher
dockerfile: src/Nocr.TextMatcher.Host/Dockerfile
ports:
- 5001:8080
- 5041:8080
# IMPORTANT: Create .nocr.env file in project root with your secrets
# See .nocr.env.example for reference
env_file:
- .nocr.env
environment:
ASPNETCORE_ENVIRONMENT: DockerCompose
depends_on:
@ -38,6 +70,11 @@ services:
condition: service_healthy
nocr-text-matcher-migrator:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
nocr-text-matcher-migrator:
build:
@ -51,23 +88,6 @@ services:
nocr-text-matcher-db:
condition: service_healthy
nocr-telegram-listener:
container_name: nocr-telegram-listener
image: nocr-telegram-listener:latest
build:
context: telegram-listener
dockerfile: src/Nocr.TelegramListener.Host/Dockerfile
ports:
- 5000:8080
environment:
ASPNETCORE_ENVIRONMENT: DockerCompose
session_pathname: '/app/WTelegram.session'
volumes:
- ./WTelegram.session:/app/WTelegram.session
depends_on:
nocr-rabbitmq:
condition: service_healthy
nocr-users:
container_name: nocr-users
image: nocr-users:latest
@ -75,7 +95,11 @@ services:
context: users
dockerfile: src/Nocr.Users.Host/Dockerfile
ports:
- 4998:8080
- 5042:8080
# IMPORTANT: Create .nocr.env file in project root with your secrets
# See .nocr.env.example for reference
env_file:
- .nocr.env
environment:
ASPNETCORE_ENVIRONMENT: DockerCompose
depends_on:
@ -83,6 +107,11 @@ services:
condition: service_healthy
nocr-users-migrator:
condition: service_completed_successfully
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
nocr-users-migrator:
build:
@ -101,7 +130,7 @@ services:
image: mariadb:latest
restart: always
ports:
- 3306:3306
- 3316:3306
environment:
MARIADB_ROOT_PASSWORD: toor
MARIADB_DATABASE: nocr_text_matcher
@ -116,7 +145,7 @@ services:
image: mariadb:latest
restart: always
ports:
- 3307:3306
- 3326:3306
environment:
MARIADB_ROOT_PASSWORD: toor
MARIADB_DATABASE: nocr_users

21
nuget.config Normal file
View File

@ -0,0 +1,21 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
<add key="musk" value="https://gitea.musk.fun/api/packages/nocr/nuget/index.json" />
</packageSources>
<!-- Package Source Mapping for Central Package Management -->
<packageSourceMapping>
<!-- All Nocr.* packages come from the private 'musk' feed -->
<packageSource key="musk">
<package pattern="Nocr.*" />
</packageSource>
<!-- All other packages come from nuget.org -->
<packageSource key="nuget.org">
<package pattern="*" />
</packageSource>
</packageSourceMapping>
</configuration>

16
prepare-build.sh Executable file
View File

@ -0,0 +1,16 @@
#!/bin/bash
# Prepares the build environment by copying nuget.config to all submodules
# This is required for both local docker-compose builds and CI/CD
set -e
echo "📦 Copying nuget.config to all submodules..."
for submodule in telegram-client telegram-listener text-matcher users; do
if [ -d "$submodule" ]; then
cp nuget.config "$submodule/"
echo "✓ Copied to $submodule/"
fi
done
echo "✅ Build preparation complete!"

@ -1 +1 @@
Subproject commit 4e330a2455bd8994bb39aa1182785d80cb45e51e
Subproject commit e8ee01cbe809867305ff02611155cddffccbb401

@ -1 +1 @@
Subproject commit 44d4fb6c55dc2f93425fd199f82272eebe9511f2
Subproject commit 9ad504753550414edd96ef4194a3e0ee7e604e28

@ -1 +1 @@
Subproject commit 2c6a2f6c6ea965e48e6692040de537f4f95603a8
Subproject commit 1a55061e4b1b125d1b911f479fc830fb94cd93cc

2
users

@ -1 +1 @@
Subproject commit b7736a189127adf7ff153e2fecc54fe4cbe6b90a
Subproject commit 30ae0c043ca2f8958223d6fb343b8b3e3561d727