kerrad/telegram-helper-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: добавлена система миграций БД и CI/CD пайплайны

Browse Source 
- Создана система отслеживания миграций (MigrationRepository, таблица migrations)
- Добавлен скрипт apply_migrations.py для автоматического применения миграций
- Созданы CI/CD пайплайны (.github/workflows/ci.yml, deploy.yml)
- Обновлена документация по миграциям в database-patterns.md
- Миграции применяются автоматически при деплое в продакшн
This commit is contained in:
Andrey
2026-01-25 23:17:09 +03:00
parent 07e72c4d14
commit e2b1353408
109 changed files with 1342 additions and 1441 deletions
Download Patch File Download Diff File Expand all files Collapse all files

106
.cursor/rules/database-patterns.md
View File

@@ -112,6 +112,106 @@ class UserRepository(DatabaseConnection):
## Миграции
- SQL миграции в `database/schema.sql`
- Python скрипты для миграций в `scripts/`
- Всегда проверяйте существование таблиц перед созданием: `CREATE TABLE IF NOT EXISTS`
### Обзор
Система миграций автоматически отслеживает и применяет изменения схемы БД. Миграции хранятся в `scripts/` и применяются автоматически при деплое.
### Создание миграции
1. **Создайте файл** в `scripts/` с понятным именем (например, `add_user_email_column.py`)
2. **Обязательные требования:**
- Функция `async def main(db_path: str)`
- Использует `aiosqlite` для работы с БД
- **Идемпотентна** - можно запускать несколько раз без ошибок
- Проверяет текущее состояние перед применением изменений
3. **Пример структуры:**
```python
#!/usr/bin/env python3
import argparse
import asyncio
import os
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
async def main(db_path: str) -> None:
"""Основная функция миграции."""
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error(f"База данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем текущее состояние
cursor = await conn.execute("PRAGMA table_info(users)")
columns = await cursor.fetchall()
# Проверяем, нужно ли применять изменения
column_exists = any(col[1] == "email" for col in columns)
if not column_exists:
await conn.execute("ALTER TABLE users ADD COLUMN email TEXT")
await conn.commit()
logger.info("Колонка email добавлена")
else:
logger.info("Колонка email уже существует")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Добавление колонки email")
parser.add_argument(
"--db",
default=os.environ.get("DATABASE_PATH", DEFAULT_DB_PATH),
help="Путь к БД",
)
args = parser.parse_args()
asyncio.run(main(args.db))
```
### Применение миграций
**Локально:**
```bash
python3 scripts/apply_migrations.py --dry-run # проверить
python3 scripts/apply_migrations.py # применить
```
**В продакшене:** Применяются автоматически при деплое через CI/CD (перед перезапуском контейнера).
### Важные правила
1. **Идемпотентность** - всегда проверяйте состояние перед изменением:
```python
# ✅ Правильно
cursor = await conn.execute("PRAGMA table_info(users)")
columns = await cursor.fetchall()
if not any(col[1] == "email" for col in columns):
await conn.execute("ALTER TABLE users ADD COLUMN email TEXT")
# ❌ Неправильно - упадет при повторном запуске
await conn.execute("ALTER TABLE users ADD COLUMN email TEXT")
```
2. **Порядок применения** - миграции применяются в алфавитном порядке по имени файла
3. **Исключения** - следующие скрипты не считаются миграциями:
- `apply_migrations.py`, `backfill_migrations.py`, `test_s3_connection.py`, `voice_cleanup.py`
### Регистрация существующих миграций
Если миграции уже применены, но не зарегистрированы:
```bash
python3 scripts/backfill_migrations.py # зарегистрировать все существующие
```

8
.cursor/rules/middleware-patterns.mdc Normal file
View File

@@ -0,0 +1,8 @@
---
name: middleware-patterns
description: This is a new rule
---
# Overview
Insert overview text here. The agent will only see this should they choose to apply the rule.

93
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,93 @@
name: CI pipeline
on:
push:
branches: [ 'dev-*', 'feature-*' ]
pull_request:
branches: [ 'dev-*', 'feature-*', 'main' ]
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
name: Test & Code Quality
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
- name: Code formatting check (Black)
run: |
echo "🔍 Checking code formatting with Black..."
black --check . || (echo "❌ Code formatting issues found. Run 'black .' to fix." && exit 1)
- name: Import sorting check (isort)
run: |
echo "🔍 Checking import sorting with isort..."
isort --check-only . || (echo "❌ Import sorting issues found. Run 'isort .' to fix." && exit 1)
- name: Linting (flake8) - Critical errors
run: |
echo "🔍 Running flake8 linter (critical errors only)..."
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics || true
- name: Linting (flake8) - Warnings
run: |
echo "🔍 Running flake8 linter (warnings)..."
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics || true
continue-on-error: true
- name: Run tests
run: |
echo "🧪 Running tests..."
python -m pytest tests/ -v --tb=short
- name: Send test success notification
if: success()
uses: appleboy/telegram-action@v1.0.0
with:
to: ${{ secrets.TELEGRAM_CHAT_ID }}
token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
message: |
✅ CI Tests Passed
📦 Repository: telegram-helper-bot
🌿 Branch: ${{ github.ref_name }}
📝 Commit: ${{ github.sha }}
👤 Author: ${{ github.actor }}
✅ All tests passed! Code quality checks completed successfully.
🔗 View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
continue-on-error: true
- name: Send test failure notification
if: failure()
uses: appleboy/telegram-action@v1.0.0
with:
to: ${{ secrets.TELEGRAM_CHAT_ID }}
token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
message: |
❌ CI Tests Failed
📦 Repository: telegram-helper-bot
🌿 Branch: ${{ github.ref_name }}
📝 Commit: ${{ github.sha }}
👤 Author: ${{ github.actor }}
❌ Tests failed! Deployment blocked. Please fix the issues and try again.
🔗 View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
continue-on-error: true

307
.github/workflows/deploy.yml vendored Normal file
View File

@@ -0,0 +1,307 @@
name: Deploy to Production
on:
push:
branches: [ main ]
workflow_dispatch:
inputs:
action:
description: 'Action to perform'
required: true
type: choice
options:
- deploy
- rollback
rollback_commit:
description: 'Commit hash to rollback to (optional, uses last successful if empty)'
required: false
type: string
jobs:
deploy:
runs-on: ubuntu-latest
name: Deploy to Production
if: |
github.event_name == 'push' ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.action == 'deploy')
concurrency:
group: production-deploy-telegram-helper-bot
cancel-in-progress: false
environment:
name: production
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: main
- name: Deploy to server
uses: appleboy/ssh-action@v1.0.0
with:
host: ${{ vars.SERVER_HOST || secrets.SERVER_HOST }}
username: ${{ vars.SERVER_USER || secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
port: ${{ vars.SSH_PORT || secrets.SSH_PORT || 22 }}
script: |
set -e
export TELEGRAM_BOT_TOKEN="${{ secrets.TELEGRAM_BOT_TOKEN }}"
export TELEGRAM_TEST_BOT_TOKEN="${{ secrets.TELEGRAM_TEST_BOT_TOKEN }}"
echo "🚀 Starting deployment to production..."
cd /home/prod
# Сохраняем информацию о коммите
CURRENT_COMMIT=$(git rev-parse HEAD)
COMMIT_MESSAGE=$(git log -1 --pretty=format:"%s" || echo "Unknown")
COMMIT_AUTHOR=$(git log -1 --pretty=format:"%an" || echo "Unknown")
TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")
echo "📝 Current commit: $CURRENT_COMMIT"
echo "📝 Commit message: $COMMIT_MESSAGE"
echo "📝 Author: $COMMIT_AUTHOR"
# Записываем в историю деплоев
HISTORY_FILE="/home/prod/.deploy_history_telegram_helper_bot.txt"
HISTORY_SIZE="${DEPLOY_HISTORY_SIZE:-10}"
echo "${TIMESTAMP}|${CURRENT_COMMIT}|${COMMIT_MESSAGE}|${COMMIT_AUTHOR}|deploying" >> "$HISTORY_FILE"
tail -n "$HISTORY_SIZE" "$HISTORY_FILE" > "${HISTORY_FILE}.tmp" && mv "${HISTORY_FILE}.tmp" "$HISTORY_FILE"
# Обновляем код
echo "📥 Pulling latest changes from main..."
sudo chown -R deploy:deploy /home/prod/bots/telegram-helper-bot || true
cd /home/prod/bots/telegram-helper-bot
git fetch origin main
git reset --hard origin/main
sudo chown -R deploy:deploy /home/prod/bots/telegram-helper-bot || true
NEW_COMMIT=$(git rev-parse HEAD)
echo "✅ Code updated: $CURRENT_COMMIT → $NEW_COMMIT"
# Применяем миграции БД перед перезапуском контейнера
echo "🔄 Applying database migrations..."
DB_PATH="/home/prod/bots/telegram-helper-bot/database/tg-bot-database.db"
if [ -f "$DB_PATH" ]; then
cd /home/prod/bots/telegram-helper-bot
python3 scripts/apply_migrations.py --db "$DB_PATH" || {
echo "❌ Ошибка при применении миграций!"
exit 1
}
echo "✅ Миграции применены успешно"
else
echo "⚠️ База данных не найдена, пропускаем миграции (будет создана при первом запуске)"
fi
# Валидация docker-compose
echo "🔍 Validating docker-compose configuration..."
cd /home/prod
docker-compose config > /dev/null || exit 1
echo "✅ docker-compose.yml is valid"
# Проверка дискового пространства
MIN_FREE_GB=5
AVAILABLE_SPACE=$(df -BG /home/prod 2>/dev/null | tail -1 | awk '{print $4}' | sed 's/G//' || echo "0")
echo "💾 Available disk space: ${AVAILABLE_SPACE}GB"
if [ "$AVAILABLE_SPACE" -lt "$MIN_FREE_GB" ]; then
echo "⚠️ Insufficient disk space! Cleaning up Docker resources..."
docker system prune -f --volumes || true
fi
# Пересобираем и перезапускаем контейнер бота
echo "🔨 Rebuilding and restarting telegram-bot container..."
cd /home/prod
export TELEGRAM_BOT_TOKEN TELEGRAM_TEST_BOT_TOKEN
docker-compose stop telegram-bot || true
docker-compose build --pull telegram-bot
docker-compose up -d telegram-bot
echo "✅ Telegram bot container rebuilt and started"
# Ждем немного и проверяем healthcheck
echo "⏳ Waiting for container to start..."
sleep 10
if docker ps | grep -q bots_telegram_bot; then
echo "✅ Container is running"
else
echo "❌ Container failed to start!"
docker logs bots_telegram_bot --tail 50 || true
exit 1
fi
- name: Update deploy history
if: always()
uses: appleboy/ssh-action@v1.0.0
with:
host: ${{ vars.SERVER_HOST || secrets.SERVER_HOST }}
username: ${{ vars.SERVER_USER || secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
port: ${{ vars.SSH_PORT || secrets.SSH_PORT || 22 }}
script: |
HISTORY_FILE="/home/prod/.deploy_history_telegram_helper_bot.txt"
if [ -f "$HISTORY_FILE" ]; then
DEPLOY_STATUS="failed"
if [ "${{ job.status }}" = "success" ]; then
DEPLOY_STATUS="success"
fi
sed -i '$s/|deploying$/|'"$DEPLOY_STATUS"'/' "$HISTORY_FILE"
echo "✅ Deploy history updated: $DEPLOY_STATUS"
fi
- name: Send deployment notification
if: always()
uses: appleboy/telegram-action@v1.0.0
with:
to: ${{ secrets.TELEGRAM_CHAT_ID }}
token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
message: |
${{ job.status == 'success' && '✅' || '❌' }} Deployment: ${{ job.status }}
📦 Repository: telegram-helper-bot
🌿 Branch: main
📝 Commit: ${{ github.event.pull_request.merge_commit_sha || github.sha }}
👤 Author: ${{ github.event.pull_request.user.login || github.actor }}
${{ github.event.pull_request.number && format('🔀 PR: #{0}', github.event.pull_request.number) || '' }}
${{ job.status == 'success' && '✅ Deployment successful! Container restarted with migrations applied.' || '❌ Deployment failed! Check logs for details.' }}
🔗 View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
continue-on-error: true
rollback:
runs-on: ubuntu-latest
name: Rollback to Previous Version
if: |
github.event_name == 'workflow_dispatch' &&
github.event.inputs.action == 'rollback'
environment:
name: production
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: main
- name: Rollback on server
uses: appleboy/ssh-action@v1.0.0
with:
host: ${{ vars.SERVER_HOST || secrets.SERVER_HOST }}
username: ${{ vars.SERVER_USER || secrets.SERVER_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
port: ${{ vars.SSH_PORT || secrets.SSH_PORT || 22 }}
script: |
set -e
export TELEGRAM_BOT_TOKEN="${{ secrets.TELEGRAM_BOT_TOKEN }}"
export TELEGRAM_TEST_BOT_TOKEN="${{ secrets.TELEGRAM_TEST_BOT_TOKEN }}"
echo "🔄 Starting rollback..."
cd /home/prod
# Определяем коммит для отката
ROLLBACK_COMMIT="${{ github.event.inputs.rollback_commit }}"
HISTORY_FILE="/home/prod/.deploy_history_telegram_helper_bot.txt"
if [ -z "$ROLLBACK_COMMIT" ]; then
echo "📝 No commit specified, finding last successful deploy..."
if [ -f "$HISTORY_FILE" ]; then
ROLLBACK_COMMIT=$(grep "|success$" "$HISTORY_FILE" | tail -1 | cut -d'|' -f2 || echo "")
fi
if [ -z "$ROLLBACK_COMMIT" ]; then
echo "❌ No successful deploy found in history!"
echo "💡 Please specify commit hash manually or check deploy history"
exit 1
fi
fi
echo "📝 Rolling back to commit: $ROLLBACK_COMMIT"
# Проверяем, что коммит существует
cd /home/prod/bots/telegram-helper-bot
if ! git cat-file -e "$ROLLBACK_COMMIT" 2>/dev/null; then
echo "❌ Commit $ROLLBACK_COMMIT not found!"
exit 1
fi
# Сохраняем текущий коммит
CURRENT_COMMIT=$(git rev-parse HEAD)
COMMIT_MESSAGE=$(git log -1 --pretty=format:"%s" "$ROLLBACK_COMMIT" || echo "Rollback")
TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")
echo "📝 Current commit: $CURRENT_COMMIT"
echo "📝 Target commit: $ROLLBACK_COMMIT"
echo "📝 Commit message: $COMMIT_MESSAGE"
# Исправляем права перед откатом
sudo chown -R deploy:deploy /home/prod/bots/telegram-helper-bot || true
# Откатываем код
echo "🔄 Rolling back code..."
git fetch origin main
git reset --hard "$ROLLBACK_COMMIT"
# Исправляем права после отката
sudo chown -R deploy:deploy /home/prod/bots/telegram-helper-bot || true
echo "✅ Code rolled back: $CURRENT_COMMIT → $ROLLBACK_COMMIT"
# Валидация docker-compose
echo "🔍 Validating docker-compose configuration..."
cd /home/prod
docker-compose config > /dev/null || exit 1
echo "✅ docker-compose.yml is valid"
# Проверка дискового пространства
MIN_FREE_GB=5
AVAILABLE_SPACE=$(df -BG /home/prod 2>/dev/null | tail -1 | awk '{print $4}' | sed 's/G//' || echo "0")
echo "💾 Available disk space: ${AVAILABLE_SPACE}GB"
if [ "$AVAILABLE_SPACE" -lt "$MIN_FREE_GB" ]; then
echo "⚠️ Insufficient disk space! Cleaning up Docker resources..."
docker system prune -f --volumes || true
fi
# Пересобираем и перезапускаем контейнер
echo "🔨 Rebuilding and restarting telegram-bot container..."
cd /home/prod
export TELEGRAM_BOT_TOKEN TELEGRAM_TEST_BOT_TOKEN
docker-compose stop telegram-bot || true
docker-compose build --pull telegram-bot
docker-compose up -d telegram-bot
echo "✅ Telegram bot container rebuilt and started"
# Записываем в историю
echo "${TIMESTAMP}|${ROLLBACK_COMMIT}|Rollback to: ${COMMIT_MESSAGE}|github-actions|rolled_back" >> "$HISTORY_FILE"
HISTORY_SIZE="${DEPLOY_HISTORY_SIZE:-10}"
tail -n "$HISTORY_SIZE" "$HISTORY_FILE" > "${HISTORY_FILE}.tmp" && mv "${HISTORY_FILE}.tmp" "$HISTORY_FILE"
echo "✅ Rollback completed successfully"
- name: Send rollback notification
if: always()
uses: appleboy/telegram-action@v1.0.0
with:
to: ${{ secrets.TELEGRAM_CHAT_ID }}
token: ${{ secrets.TELEGRAM_BOT_TOKEN }}
message: |
${{ job.status == 'success' && '🔄' || '❌' }} Rollback: ${{ job.status }}
📦 Repository: telegram-helper-bot
🌿 Branch: main
📝 Rolled back to: ${{ github.event.inputs.rollback_commit || 'Last successful commit' }}
👤 Triggered by: ${{ github.actor }}
${{ job.status == 'success' && '✅ Rollback completed successfully! Services restored to previous version.' || '❌ Rollback failed! Check logs for details.' }}
🔗 View details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
continue-on-error: true

BIN
:memory: Normal file
View File

Binary file not shown.

11
database/__init__.py
View File

@@ -9,13 +9,12 @@
- async_db: основной класс AsyncBotDB
"""
from .models import (
User, BlacklistUser, UserMessage, TelegramPost, PostContent,
MessageContentLink, Admin, Migration, AudioMessage, AudioListenRecord, AudioModerate
)
from .repository_factory import RepositoryFactory
from .base import DatabaseConnection
from .async_db import AsyncBotDB
from .base import DatabaseConnection
from .models import (Admin, AudioListenRecord, AudioMessage, AudioModerate,
BlacklistUser, MessageContentLink, Migration, PostContent,
TelegramPost, User, UserMessage)
from .repository_factory import RepositoryFactory
# Для обратной совместимости экспортируем старый интерфейс
__all__ = [

30
database/async_db.py
View File

@@ -1,11 +1,11 @@
import aiosqlite
from datetime import datetime
from typing import Optional, List, Dict, Any, Tuple
from typing import Any, Dict, List, Optional, Tuple
import aiosqlite
from database.models import (Admin, AudioMessage, BlacklistHistoryRecord,
BlacklistUser, PostContent, TelegramPost, User,
UserMessage)
from database.repository_factory import RepositoryFactory
from database.models import (
User, BlacklistUser, BlacklistHistoryRecord, UserMessage, TelegramPost, PostContent,
Admin, AudioMessage
)
class AsyncBotDB:
@@ -403,25 +403,9 @@ class AsyncBotDB:
await self.factory.audio.delete_audio_record_by_file_name(file_name)
# Методы для миграций
async def get_migration_version(self) -> int:
"""Получение текущей версии миграции."""
return await self.factory.migrations.get_migration_version()
async def get_current_version(self) -> Optional[int]:
"""Возвращает текущую последнюю версию миграции."""
return await self.factory.migrations.get_current_version()
async def update_version(self, new_version: int, script_name: str):
"""Обновляет версию миграций в таблице migrations."""
await self.factory.migrations.update_version(new_version, script_name)
async def create_table(self, sql_script: str):
"""Создает таблицу в базе. Используется в миграциях."""
await self.factory.migrations.create_table(sql_script)
async def update_migration_version(self, version: int, script_name: str):
"""Обновление версии миграции."""
await self.factory.migrations.update_version(version, script_name)
await self.factory.migrations.create_table_from_sql(sql_script)
# Методы для voice bot welcome tracking
async def check_voice_bot_welcome_received(self, user_id: int) -> bool:

3
database/base.py
View File

@@ -1,6 +1,7 @@
import os
import aiosqlite
from typing import Optional
import aiosqlite
from logs.custom_logger import logger

5
database/models.py
View File

@@ -1,6 +1,6 @@
from dataclasses import dataclass
from datetime import datetime
from typing import Optional, List
from typing import List, Optional
@dataclass
@@ -89,9 +89,8 @@ class Admin:
@dataclass
class Migration:
"""Модель миграции."""
version: int
script_name: str
created_at: Optional[str] = None
applied_at: Optional[str] = None
@dataclass

15
database/repositories/__init__.py
View File

@@ -9,17 +9,20 @@
- post_repository: работа с постами
- admin_repository: работа с администраторами
- audio_repository: работа с аудио
- migration_repository: работа с миграциями БД
"""
from .user_repository import UserRepository
from .blacklist_repository import BlacklistRepository
from .blacklist_history_repository import BlacklistHistoryRepository
from .message_repository import MessageRepository
from .post_repository import PostRepository
from .admin_repository import AdminRepository
from .audio_repository import AudioRepository
from .blacklist_history_repository import BlacklistHistoryRepository
from .blacklist_repository import BlacklistRepository
from .message_repository import MessageRepository
from .migration_repository import MigrationRepository
from .post_repository import PostRepository
from .user_repository import UserRepository
__all__ = [
'UserRepository', 'BlacklistRepository', 'BlacklistHistoryRepository',
'MessageRepository', 'PostRepository', 'AdminRepository', 'AudioRepository'
'MessageRepository', 'PostRepository', 'AdminRepository', 'AudioRepository',
'MigrationRepository'
]

1
database/repositories/admin_repository.py
View File

@@ -1,4 +1,5 @@
from typing import Optional
from database.base import DatabaseConnection
from database.models import Admin

7
database/repositories/audio_repository.py
View File

@@ -1,7 +1,8 @@
from typing import Optional, List, Dict, Any
from database.base import DatabaseConnection
from database.models import AudioMessage, AudioListenRecord, AudioModerate
from datetime import datetime
from typing import Any, Dict, List, Optional
from database.base import DatabaseConnection
from database.models import AudioListenRecord, AudioMessage, AudioModerate
class AudioRepository(DatabaseConnection):

1
database/repositories/blacklist_history_repository.py
View File

@@ -1,4 +1,5 @@
from typing import Optional
from database.base import DatabaseConnection
from database.models import BlacklistHistoryRecord

3
database/repositories/blacklist_repository.py
View File

@@ -1,4 +1,5 @@
from typing import Optional, List, Dict
from typing import Dict, List, Optional
from database.base import DatabaseConnection
from database.models import BlacklistUser

1
database/repositories/message_repository.py
View File

@@ -1,5 +1,6 @@
from datetime import datetime
from typing import Optional
from database.base import DatabaseConnection
from database.models import UserMessage

79
database/repositories/migration_repository.py Normal file
View File

@@ -0,0 +1,79 @@
"""Репозиторий для работы с миграциями базы данных."""
import aiosqlite
from database.base import DatabaseConnection
class MigrationRepository(DatabaseConnection):
"""Репозиторий для управления миграциями базы данных."""
async def create_table(self):
"""Создает таблицу migrations, если она не существует."""
query = """
CREATE TABLE IF NOT EXISTS migrations (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
script_name TEXT NOT NULL UNIQUE,
applied_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))
)
"""
await self._execute_query(query)
self.logger.info("Таблица migrations создана или уже существует")
async def get_applied_migrations(self) -> list[str]:
"""Возвращает список имен примененных скриптов миграций."""
conn = None
try:
conn = await self._get_connection()
cursor = await conn.execute("SELECT script_name FROM migrations ORDER BY applied_at")
rows = await cursor.fetchall()
await cursor.close()
return [row[0] for row in rows]
except Exception as e:
self.logger.error(f"Ошибка при получении списка миграций: {e}")
raise
finally:
if conn:
await conn.close()
async def is_migration_applied(self, script_name: str) -> bool:
"""Проверяет, применена ли миграция."""
conn = None
try:
conn = await self._get_connection()
cursor = await conn.execute(
"SELECT COUNT(*) FROM migrations WHERE script_name = ?",
(script_name,)
)
row = await cursor.fetchone()
await cursor.close()
return row[0] > 0 if row else False
except Exception as e:
self.logger.error(f"Ошибка при проверке миграции {script_name}: {e}")
raise
finally:
if conn:
await conn.close()
async def mark_migration_applied(self, script_name: str) -> None:
"""Отмечает миграцию как примененную."""
conn = None
try:
conn = await self._get_connection()
await conn.execute(
"INSERT INTO migrations (script_name) VALUES (?)",
(script_name,)
)
await conn.commit()
self.logger.info(f"Миграция {script_name} отмечена как примененная")
except aiosqlite.IntegrityError:
self.logger.warning(f"Миграция {script_name} уже была применена ранее")
except Exception as e:
self.logger.error(f"Ошибка при отметке миграции {script_name}: {e}")
raise
finally:
if conn:
await conn.close()
async def create_table_from_sql(self, sql_script: str) -> None:
"""Создает таблицу из SQL скрипта. Используется в миграциях."""
await self._execute_query(sql_script)

5
database/repositories/post_repository.py
View File

@@ -1,7 +1,8 @@
from datetime import datetime
from typing import Optional, List, Tuple
from typing import List, Optional, Tuple
from database.base import DatabaseConnection
from database.models import TelegramPost, PostContent, MessageContentLink
from database.models import MessageContentLink, PostContent, TelegramPost
class PostRepository(DatabaseConnection):

3
database/repositories/user_repository.py
View File

@@ -1,5 +1,6 @@
from datetime import datetime
from typing import Optional, List, Dict, Any
from typing import Any, Dict, List, Optional
from database.base import DatabaseConnection
from database.models import User

22
database/repository_factory.py
View File

@@ -1,11 +1,14 @@
from typing import Optional
from database.repositories.user_repository import UserRepository
from database.repositories.blacklist_repository import BlacklistRepository
from database.repositories.blacklist_history_repository import BlacklistHistoryRepository
from database.repositories.message_repository import MessageRepository
from database.repositories.post_repository import PostRepository
from database.repositories.admin_repository import AdminRepository
from database.repositories.audio_repository import AudioRepository
from database.repositories.blacklist_history_repository import \
BlacklistHistoryRepository
from database.repositories.blacklist_repository import BlacklistRepository
from database.repositories.message_repository import MessageRepository
from database.repositories.migration_repository import MigrationRepository
from database.repositories.post_repository import PostRepository
from database.repositories.user_repository import UserRepository
class RepositoryFactory:
@@ -20,6 +23,7 @@ class RepositoryFactory:
self._post_repo: Optional[PostRepository] = None
self._admin_repo: Optional[AdminRepository] = None
self._audio_repo: Optional[AudioRepository] = None
self._migration_repo: Optional[MigrationRepository] = None
@property
def users(self) -> UserRepository:
@@ -70,8 +74,16 @@ class RepositoryFactory:
self._audio_repo = AudioRepository(self.db_path)
return self._audio_repo
@property
def migrations(self) -> MigrationRepository:
"""Возвращает репозиторий миграций."""
if self._migration_repo is None:
self._migration_repo = MigrationRepository(self.db_path)
return self._migration_repo
async def create_all_tables(self):
"""Создает все таблицы в базе данных."""
await self.migrations.create_table() # Сначала создаем таблицу миграций
await self.users.create_tables()
await self.blacklist.create_tables()
await self.blacklist_history.create_tables()

7
database/schema.sql
View File

@@ -126,6 +126,13 @@ CREATE TABLE IF NOT EXISTS audio_moderate (
FOREIGN KEY (user_id) REFERENCES our_users(user_id) ON DELETE CASCADE
);
-- Database migrations tracking
CREATE TABLE IF NOT EXISTS migrations (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
script_name TEXT NOT NULL UNIQUE,
applied_at INTEGER NOT NULL DEFAULT (strftime('%s', 'now'))
);
-- Create indexes for better performance
-- Optimized index for user_audio_listens - only user_id for "show all audio listened by user X"
CREATE INDEX IF NOT EXISTS idx_user_audio_listens_user_id ON user_audio_listens(user_id);

20
helper_bot/handlers/admin/__init__.py
View File

@@ -1,20 +1,10 @@
from .admin_handlers import admin_router
from .dependencies import AdminAccessMiddleware, BotDB, Settings
from .services import AdminService, User, BannedUser
from .exceptions import (
AdminError,
AdminAccessDeniedError,
UserNotFoundError,
InvalidInputError,
UserAlreadyBannedError
)
from .utils import (
return_to_admin_menu,
handle_admin_error,
format_user_info,
format_ban_confirmation,
escape_html
)
from .exceptions import (AdminAccessDeniedError, AdminError, InvalidInputError,
UserAlreadyBannedError, UserNotFoundError)
from .services import AdminService, BannedUser, User
from .utils import (escape_html, format_ban_confirmation, format_user_info,
handle_admin_error, return_to_admin_menu)
__all__ = [
'admin_router',

44
helper_bot/handlers/admin/admin_handlers.py
View File

@@ -1,36 +1,24 @@
from aiogram import Router, types, F
from aiogram.filters import Command, StateFilter, MagicData
from aiogram import F, Router, types
from aiogram.filters import Command, MagicData, StateFilter
from aiogram.fsm.context import FSMContext
from helper_bot.filters.main import ChatTypeFilter
from helper_bot.keyboards.keyboards import (
get_reply_keyboard_admin,
create_keyboard_with_pagination,
create_keyboard_for_ban_days,
create_keyboard_for_approve_ban,
create_keyboard_for_ban_reason
)
from helper_bot.handlers.admin.dependencies import AdminAccessMiddleware
from helper_bot.handlers.admin.exceptions import (InvalidInputError,
UserAlreadyBannedError)
from helper_bot.handlers.admin.services import AdminService
from helper_bot.handlers.admin.exceptions import (
UserAlreadyBannedError,
InvalidInputError
)
from helper_bot.handlers.admin.utils import (
return_to_admin_menu,
handle_admin_error,
format_user_info,
format_ban_confirmation,
escape_html
)
from logs.custom_logger import logger
from helper_bot.handlers.admin.utils import (escape_html,
format_ban_confirmation,
format_user_info,
handle_admin_error,
return_to_admin_menu)
from helper_bot.keyboards.keyboards import (create_keyboard_for_approve_ban,
create_keyboard_for_ban_days,
create_keyboard_for_ban_reason,
create_keyboard_with_pagination,
get_reply_keyboard_admin)
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time
)
from helper_bot.utils.metrics import db_query_time, track_errors, track_time
from logs.custom_logger import logger
# Создаем роутер с middleware для проверки доступа
admin_router = Router()

2
helper_bot/handlers/admin/constants.py
View File

@@ -1,6 +1,6 @@
"""Constants for admin handlers"""
from typing import Final, Dict
from typing import Dict, Final
# Admin button texts
ADMIN_BUTTON_TEXTS: Final[Dict[str, str]] = {

5
helper_bot/handlers/admin/dependencies.py
View File

@@ -1,11 +1,12 @@
from typing import Dict, Any
from typing import Any, Dict
try:
from typing import Annotated
except ImportError:
from typing_extensions import Annotated
from aiogram import BaseMiddleware
from aiogram.types import TelegramObject
from helper_bot.utils.base_dependency_factory import get_global_instance
from helper_bot.utils.helper_func import check_access
from logs.custom_logger import logger

20
helper_bot/handlers/admin/rate_limit_handlers.py
View File

@@ -1,22 +1,20 @@
"""
Обработчики команд для мониторинга rate limiting
"""
from aiogram import Router, types, F
from aiogram import F, Router, types
from aiogram.filters import Command, MagicData
from aiogram.fsm.context import FSMContext
from aiogram.types import FSInputFile
from helper_bot.filters.main import ChatTypeFilter
from helper_bot.middlewares.dependencies_middleware import DependenciesMiddleware
from helper_bot.utils.rate_limit_monitor import rate_limit_monitor, get_rate_limit_summary
from helper_bot.utils.rate_limit_metrics import update_rate_limit_gauges, get_rate_limit_metrics_summary
from logs.custom_logger import logger
from helper_bot.middlewares.dependencies_middleware import \
DependenciesMiddleware
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors
)
from helper_bot.utils.metrics import track_errors, track_time
from helper_bot.utils.rate_limit_metrics import (
get_rate_limit_metrics_summary, update_rate_limit_gauges)
from helper_bot.utils.rate_limit_monitor import (get_rate_limit_summary,
rate_limit_monitor)
from logs.custom_logger import logger
class RateLimitHandlers:

17
helper_bot/handlers/admin/services.py
View File

@@ -1,15 +1,14 @@
from typing import List, Optional
from datetime import datetime
from typing import List, Optional
from helper_bot.utils.helper_func import add_days_to_date, get_banned_users_buttons, get_banned_users_list
from helper_bot.handlers.admin.exceptions import UserAlreadyBannedError, InvalidInputError
from logs.custom_logger import logger
from helper_bot.handlers.admin.exceptions import (InvalidInputError,
UserAlreadyBannedError)
from helper_bot.utils.helper_func import (add_days_to_date,
get_banned_users_buttons,
get_banned_users_list)
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors
)
from helper_bot.utils.metrics import track_errors, track_time
from logs.custom_logger import logger
class User:

4
helper_bot/handlers/admin/utils.py
View File

@@ -1,10 +1,10 @@
import html
from typing import Optional
from aiogram import types
from aiogram.fsm.context import FSMContext
from helper_bot.keyboards.keyboards import get_reply_keyboard_admin
from helper_bot.handlers.admin.exceptions import AdminError
from helper_bot.keyboards.keyboards import get_reply_keyboard_admin
from logs.custom_logger import logger

11
helper_bot/handlers/callback/__init__.py
View File

@@ -1,10 +1,9 @@
from .callback_handlers import callback_router
from .services import PostPublishService, BanService
from .exceptions import UserBlockedBotError, PostNotFoundError, UserNotFoundError, PublishError, BanError
from .constants import (
CALLBACK_PUBLISH, CALLBACK_DECLINE, CALLBACK_BAN, CALLBACK_UNLOCK,
CALLBACK_RETURN, CALLBACK_PAGE
)
from .constants import (CALLBACK_BAN, CALLBACK_DECLINE, CALLBACK_PAGE,
CALLBACK_PUBLISH, CALLBACK_RETURN, CALLBACK_UNLOCK)
from .exceptions import (BanError, PostNotFoundError, PublishError,
UserBlockedBotError, UserNotFoundError)
from .services import BanService, PostPublishService
__all__ = [
'callback_router',

47
helper_bot/handlers/callback/callback_handlers.py
View File

@@ -1,37 +1,34 @@
import html
import traceback
import time
import traceback
from datetime import datetime
from aiogram import Router, F
from aiogram.types import CallbackQuery
from aiogram.fsm.context import FSMContext
from aiogram import F, Router
from aiogram.filters import MagicData
from helper_bot.handlers.voice.constants import CALLBACK_SAVE, CALLBACK_DELETE
from helper_bot.handlers.voice.services import AudioFileService
from helper_bot.keyboards.keyboards import create_keyboard_with_pagination, get_reply_keyboard_admin, \
create_keyboard_for_ban_reason
from helper_bot.utils.helper_func import get_banned_users_list, get_banned_users_buttons
from aiogram.fsm.context import FSMContext
from aiogram.types import CallbackQuery
from helper_bot.handlers.admin.utils import format_user_info
from helper_bot.handlers.voice.constants import CALLBACK_DELETE, CALLBACK_SAVE
from helper_bot.handlers.voice.services import AudioFileService
from helper_bot.keyboards.keyboards import (create_keyboard_for_ban_reason,
create_keyboard_with_pagination,
get_reply_keyboard_admin)
from helper_bot.utils.base_dependency_factory import get_global_instance
from .dependency_factory import get_post_publish_service, get_ban_service
from .exceptions import UserBlockedBotError, PostNotFoundError, UserNotFoundError, PublishError, BanError
from .constants import (
CALLBACK_PUBLISH, CALLBACK_DECLINE, CALLBACK_BAN, CALLBACK_UNLOCK,
CALLBACK_RETURN, CALLBACK_PAGE, MESSAGE_PUBLISHED, MESSAGE_DECLINED,
MESSAGE_USER_BANNED, MESSAGE_USER_UNLOCKED, MESSAGE_ERROR,
ERROR_BOT_BLOCKED
)
from helper_bot.utils.helper_func import (get_banned_users_buttons,
get_banned_users_list)
# Local imports - metrics
from helper_bot.utils.metrics import (db_query_time, track_errors,
track_file_operations, track_time)
from logs.custom_logger import logger
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time,
track_file_operations
)
from .constants import (CALLBACK_BAN, CALLBACK_DECLINE, CALLBACK_PAGE,
CALLBACK_PUBLISH, CALLBACK_RETURN, CALLBACK_UNLOCK,
ERROR_BOT_BLOCKED, MESSAGE_DECLINED, MESSAGE_ERROR,
MESSAGE_PUBLISHED, MESSAGE_USER_BANNED,
MESSAGE_USER_UNLOCKED)
from .dependency_factory import get_ban_service, get_post_publish_service
from .exceptions import (BanError, PostNotFoundError, PublishError,
UserBlockedBotError, UserNotFoundError)
callback_router = Router()

2
helper_bot/handlers/callback/constants.py
View File

@@ -1,4 +1,4 @@
from typing import Final, Dict
from typing import Dict, Final
# Callback data constants
CALLBACK_PUBLISH = "publish"

5
helper_bot/handlers/callback/dependency_factory.py
View File

@@ -1,10 +1,11 @@
from typing import Callable
from aiogram import Bot
from aiogram.client.default import DefaultBotProperties
from aiogram.fsm.context import FSMContext
from helper_bot.utils.base_dependency_factory import get_global_instance
from .services import PostPublishService, BanService
from .services import BanService, PostPublishService
def get_post_publish_service() -> PostPublishService:

49
helper_bot/handlers/callback/services.py
View File

@@ -1,36 +1,31 @@
from datetime import datetime, timedelta
import html
from typing import Dict, Any
from datetime import datetime, timedelta
from typing import Any, Dict
from aiogram import Bot
from aiogram import types
from aiogram import Bot, types
from aiogram.types import CallbackQuery
from helper_bot.utils.helper_func import (
send_text_message, send_photo_message, send_video_message,
send_video_note_message, send_audio_message, send_voice_message,
send_media_group_to_channel, delete_user_blacklist, get_text_message
)
from helper_bot.keyboards.keyboards import create_keyboard_for_ban_reason
from .exceptions import (
UserBlockedBotError, PostNotFoundError, UserNotFoundError,
PublishError, BanError
)
from .constants import (
CONTENT_TYPE_TEXT, CONTENT_TYPE_PHOTO, CONTENT_TYPE_VIDEO,
CONTENT_TYPE_VIDEO_NOTE, CONTENT_TYPE_AUDIO, CONTENT_TYPE_VOICE,
CONTENT_TYPE_MEDIA_GROUP, MESSAGE_POST_PUBLISHED, MESSAGE_POST_DECLINED,
MESSAGE_USER_BANNED_SPAM, ERROR_BOT_BLOCKED
)
from helper_bot.utils.helper_func import (delete_user_blacklist,
get_text_message, send_audio_message,
send_media_group_to_channel,
send_photo_message,
send_text_message,
send_video_message,
send_video_note_message,
send_voice_message)
# Local imports - metrics
from helper_bot.utils.metrics import (db_query_time, track_errors,
track_media_processing, track_time)
from logs.custom_logger import logger
# Local imports - metrics
from helper_bot.utils.metrics import (
track_media_processing,
track_time,
track_errors,
db_query_time
)
from .constants import (CONTENT_TYPE_AUDIO, CONTENT_TYPE_MEDIA_GROUP,
CONTENT_TYPE_PHOTO, CONTENT_TYPE_TEXT,
CONTENT_TYPE_VIDEO, CONTENT_TYPE_VIDEO_NOTE,
CONTENT_TYPE_VOICE, ERROR_BOT_BLOCKED,
MESSAGE_POST_DECLINED, MESSAGE_POST_PUBLISHED,
MESSAGE_USER_BANNED_SPAM)
from .exceptions import (BanError, PostNotFoundError, PublishError,
UserBlockedBotError, UserNotFoundError)
class PostPublishService:

25
helper_bot/handlers/group/__init__.py
View File

@@ -1,28 +1,13 @@
"""Group handlers package for Telegram bot"""
# Local imports - main components
from .group_handlers import (
group_router,
create_group_handlers,
GroupHandlers
)
# Local imports - services
from .services import (
AdminReplyService,
DatabaseProtocol
)
# Local imports - constants and utilities
from .constants import (
FSM_STATES,
ERROR_MESSAGES
)
from .exceptions import (
NoReplyToMessageError,
UserNotFoundError
)
from .constants import ERROR_MESSAGES, FSM_STATES
from .decorators import error_handler
from .exceptions import NoReplyToMessageError, UserNotFoundError
from .group_handlers import GroupHandlers, create_group_handlers, group_router
# Local imports - services
from .services import AdminReplyService, DatabaseProtocol
__all__ = [
# Main components

2
helper_bot/handlers/group/constants.py
View File

@@ -1,6 +1,6 @@
"""Constants for group handlers"""
from typing import Final, Dict
from typing import Dict, Final
# FSM States
FSM_STATES: Final[Dict[str, str]] = {

4
helper_bot/handlers/group/decorators.py
View File

@@ -6,7 +6,6 @@ from typing import Any, Callable
# Third-party imports
from aiogram import types
# Local imports
from logs.custom_logger import logger
@@ -22,7 +21,8 @@ def error_handler(func: Callable[..., Any]) -> Callable[..., Any]:
try:
message = next((arg for arg in args if isinstance(arg, types.Message)), None)
if message and hasattr(message, 'bot'):
from helper_bot.utils.base_dependency_factory import get_global_instance
from helper_bot.utils.base_dependency_factory import \
get_global_instance
bdf = get_global_instance()
important_logs = bdf.settings['Telegram']['important_logs']
await message.bot.send_message(

24
helper_bot/handlers/group/group_handlers.py
View File

@@ -3,26 +3,20 @@
# Third-party imports
from aiogram import Router, types
from aiogram.fsm.context import FSMContext
# Local imports - filters
from database.async_db import AsyncBotDB
from helper_bot.filters.main import ChatTypeFilter
# Local imports - modular components
from .constants import FSM_STATES, ERROR_MESSAGES
from .services import AdminReplyService
from .decorators import error_handler
from .exceptions import UserNotFoundError
# Local imports - metrics
from helper_bot.utils.metrics import metrics, track_errors, track_time
# Local imports - utilities
from logs.custom_logger import logger
# Local imports - metrics
from helper_bot.utils.metrics import (
metrics,
track_time,
track_errors
)
# Local imports - modular components
from .constants import ERROR_MESSAGES, FSM_STATES
from .decorators import error_handler
from .exceptions import UserNotFoundError
from .services import AdminReplyService
class GroupHandlers:
"""Main handler class for group messages"""
@@ -102,8 +96,8 @@ def init_legacy_router():
"""Initialize legacy router with global dependencies"""
global group_router
from helper_bot.utils.base_dependency_factory import get_global_instance
from helper_bot.keyboards.keyboards import get_reply_keyboard_leave_chat
from helper_bot.utils.base_dependency_factory import get_global_instance
bdf = get_global_instance()
#TODO: поменять архитектуру и подключить правильный BotDB

13
helper_bot/handlers/group/services.py
View File

@@ -1,22 +1,17 @@
"""Service classes for group handlers"""
# Standard library imports
from typing import Protocol, Optional
from typing import Optional, Protocol
# Third-party imports
from aiogram import types
# Local imports
from helper_bot.utils.helper_func import send_text_message
from .exceptions import NoReplyToMessageError, UserNotFoundError
# Local imports - metrics
from helper_bot.utils.metrics import db_query_time, track_errors, track_time
from logs.custom_logger import logger
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time
)
from .exceptions import NoReplyToMessageError, UserNotFoundError
class DatabaseProtocol(Protocol):

24
helper_bot/handlers/private/__init__.py
View File

@@ -1,27 +1,13 @@
"""Private handlers package for Telegram bot"""
# Local imports - main components
from .private_handlers import (
private_router,
create_private_handlers,
PrivateHandlers
)
# Local imports - services
from .services import (
BotSettings,
UserService,
PostService,
StickerService
)
# Local imports - constants and utilities
from .constants import (
FSM_STATES,
BUTTON_TEXTS,
ERROR_MESSAGES
)
from .constants import BUTTON_TEXTS, ERROR_MESSAGES, FSM_STATES
from .decorators import error_handler
from .private_handlers import (PrivateHandlers, create_private_handlers,
private_router)
# Local imports - services
from .services import BotSettings, PostService, StickerService, UserService
__all__ = [
# Main components

2
helper_bot/handlers/private/constants.py
View File

@@ -1,6 +1,6 @@
"""Constants for private handlers"""
from typing import Final, Dict
from typing import Dict, Final
# FSM States
FSM_STATES: Final[Dict[str, str]] = {

4
helper_bot/handlers/private/decorators.py
View File

@@ -6,7 +6,6 @@ from typing import Any, Callable
# Third-party imports
from aiogram import types
# Local imports
from logs.custom_logger import logger
@@ -22,7 +21,8 @@ def error_handler(func: Callable[..., Any]) -> Callable[..., Any]:
try:
message = next((arg for arg in args if isinstance(arg, types.Message)), None)
if message and hasattr(message, 'bot'):
from helper_bot.utils.base_dependency_factory import get_global_instance
from helper_bot.utils.base_dependency_factory import \
get_global_instance
bdf = get_global_instance()
important_logs = bdf.settings['Telegram']['important_logs']
await message.bot.send_message(

29
helper_bot/handlers/private/private_handlers.py
View File

@@ -5,37 +5,28 @@ import asyncio
from datetime import datetime
# Third-party imports
from aiogram import types, Router, F
from aiogram import F, Router, types
from aiogram.filters import Command, StateFilter
from aiogram.fsm.context import FSMContext
# Local imports - filters and middlewares
from database.async_db import AsyncBotDB
from helper_bot.filters.main import ChatTypeFilter
# Local imports - utilities
from helper_bot.keyboards import (get_reply_keyboard,
get_reply_keyboard_for_post)
from helper_bot.keyboards.keyboards import get_reply_keyboard_leave_chat
from helper_bot.middlewares.album_middleware import AlbumMiddleware
from helper_bot.middlewares.blacklist_middleware import BlacklistMiddleware
# Local imports - utilities
from helper_bot.keyboards import get_reply_keyboard, get_reply_keyboard_for_post
from helper_bot.keyboards.keyboards import get_reply_keyboard_leave_chat
from helper_bot.utils import messages
from helper_bot.utils.helper_func import (
get_first_name,
update_user_info,
check_user_emoji
)
from helper_bot.utils.helper_func import (check_user_emoji, get_first_name,
update_user_info)
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time
)
from helper_bot.utils.metrics import db_query_time, track_errors, track_time
# Local imports - modular components
from .constants import FSM_STATES, BUTTON_TEXTS, ERROR_MESSAGES
from .services import BotSettings, UserService, PostService, StickerService
from .constants import BUTTON_TEXTS, ERROR_MESSAGES, FSM_STATES
from .decorators import error_handler
from .services import BotSettings, PostService, StickerService, UserService
# Expose sleep for tests (tests patch helper_bot.handlers.private.private_handlers.sleep)
sleep = asyncio.sleep

41
helper_bot/handlers/private/services.py
View File

@@ -1,46 +1,31 @@
"""Service classes for private handlers"""
# Standard library imports
import random
import asyncio
import html
import random
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Dict, Callable, Any, Protocol, Union
from dataclasses import dataclass
from typing import Any, Callable, Dict, Protocol, Union
# Third-party imports
from aiogram import types
from aiogram.types import FSInputFile
from database.models import TelegramPost, User
from logs.custom_logger import logger
from helper_bot.keyboards import get_reply_keyboard_for_post
# Local imports - utilities
from helper_bot.utils.helper_func import (
get_first_name,
get_text_message,
determine_anonymity,
send_text_message,
send_photo_message,
send_media_group_message_to_private_chat,
prepare_media_group_from_middlewares,
send_video_message,
send_video_note_message,
send_audio_message,
send_voice_message,
add_in_db_media,
check_username_and_full_name
)
from helper_bot.keyboards import get_reply_keyboard_for_post
add_in_db_media, check_username_and_full_name, determine_anonymity,
get_first_name, get_text_message, prepare_media_group_from_middlewares,
send_audio_message, send_media_group_message_to_private_chat,
send_photo_message, send_text_message, send_video_message,
send_video_note_message, send_voice_message)
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time,
track_media_processing,
track_file_operations
)
from helper_bot.utils.metrics import (db_query_time, track_errors,
track_file_operations,
track_media_processing, track_time)
from logs.custom_logger import logger
class DatabaseProtocol(Protocol):

5
helper_bot/handlers/voice/cleanup_utils.py
View File

@@ -1,12 +1,13 @@
"""
Утилиты для очистки и диагностики проблем с голосовыми файлами
"""
import os
import asyncio
import os
from pathlib import Path
from typing import List, Tuple
from logs.custom_logger import logger
from helper_bot.handlers.voice.constants import VOICE_USERS_DIR
from logs.custom_logger import logger
class VoiceFileCleanupUtils:

2
helper_bot/handlers/voice/constants.py
View File

@@ -1,4 +1,4 @@
from typing import Final, Dict
from typing import Dict, Final
# Voice bot constants
VOICE_BOT_NAME = "voice"

28
helper_bot/handlers/voice/services.py
View File

@@ -1,26 +1,26 @@
import random
import asyncio
import traceback
import os
import random
import traceback
from datetime import datetime
from pathlib import Path
from typing import List, Optional, Tuple
from aiogram.types import FSInputFile
from helper_bot.handlers.voice.exceptions import VoiceMessageError, AudioProcessingError, DatabaseError, FileOperationError
from helper_bot.handlers.voice.constants import (
VOICE_USERS_DIR, STICK_DIR, STICK_PATTERN, STICKER_DELAY,
MESSAGE_DELAY_1, MESSAGE_DELAY_2, MESSAGE_DELAY_3, MESSAGE_DELAY_4
)
from helper_bot.handlers.voice.constants import (MESSAGE_DELAY_1,
MESSAGE_DELAY_2,
MESSAGE_DELAY_3,
MESSAGE_DELAY_4, STICK_DIR,
STICK_PATTERN, STICKER_DELAY,
VOICE_USERS_DIR)
from helper_bot.handlers.voice.exceptions import (AudioProcessingError,
DatabaseError,
FileOperationError,
VoiceMessageError)
# Local imports - metrics
from helper_bot.utils.metrics import db_query_time, track_errors, track_time
from logs.custom_logger import logger
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time
)
class VoiceMessage:
"""Модель голосового сообщения"""

8
helper_bot/handlers/voice/utils.py
View File

@@ -1,16 +1,12 @@
import time
import html
import time
from datetime import datetime
from typing import Optional
from helper_bot.handlers.voice.exceptions import DatabaseError
from helper_bot.utils.metrics import db_query_time, track_errors, track_time
from logs.custom_logger import logger
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time
)
def format_time_ago(date_from_db: str) -> Optional[str]:
"""Форматировать время с момента последней записи"""

38
helper_bot/handlers/voice/voice_handler.py
View File

@@ -2,33 +2,31 @@ import asyncio
from datetime import datetime
from pathlib import Path
from aiogram import Router, types, F
from aiogram.filters import Command, StateFilter, MagicData
from aiogram import F, Router, types
from aiogram.filters import Command, MagicData, StateFilter
from aiogram.fsm.context import FSMContext
from aiogram.types import FSInputFile
from helper_bot.filters.main import ChatTypeFilter
from helper_bot.middlewares.blacklist_middleware import BlacklistMiddleware
from helper_bot.middlewares.dependencies_middleware import DependenciesMiddleware
from helper_bot.utils import messages
from helper_bot.utils.helper_func import get_first_name, update_user_info, check_user_emoji, send_voice_message
from logs.custom_logger import logger
from helper_bot.handlers.private.constants import BUTTON_TEXTS, FSM_STATES
from helper_bot.handlers.voice.constants import *
from helper_bot.handlers.voice.services import VoiceBotService
from helper_bot.handlers.voice.utils import get_last_message_text, validate_voice_message, get_user_emoji_safe
from helper_bot.keyboards.keyboards import get_main_keyboard, get_reply_keyboard_for_voice
from helper_bot.handlers.voice.utils import (get_last_message_text,
get_user_emoji_safe,
validate_voice_message)
from helper_bot.keyboards import get_reply_keyboard
from helper_bot.handlers.private.constants import FSM_STATES
from helper_bot.handlers.private.constants import BUTTON_TEXTS
from helper_bot.keyboards.keyboards import (get_main_keyboard,
get_reply_keyboard_for_voice)
from helper_bot.middlewares.blacklist_middleware import BlacklistMiddleware
from helper_bot.middlewares.dependencies_middleware import \
DependenciesMiddleware
from helper_bot.utils import messages
from helper_bot.utils.helper_func import (check_user_emoji, get_first_name,
send_voice_message, update_user_info)
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors,
db_query_time,
track_file_operations
)
from helper_bot.utils.metrics import (db_query_time, track_errors,
track_file_operations, track_time)
from logs.custom_logger import logger
class VoiceHandlers:
def __init__(self, db, settings):

2
helper_bot/keyboards/__init__.py
View File

@@ -1 +1 @@
from .keyboards import get_reply_keyboard_for_post, get_reply_keyboard
from .keyboards import get_reply_keyboard, get_reply_keyboard_for_post

8
helper_bot/keyboards/keyboards.py
View File

@@ -1,11 +1,7 @@
from aiogram import types
from aiogram.utils.keyboard import ReplyKeyboardBuilder, InlineKeyboardBuilder
from aiogram.utils.keyboard import InlineKeyboardBuilder, ReplyKeyboardBuilder
# Local imports - metrics
from helper_bot.utils.metrics import (
track_time,
track_errors
)
from helper_bot.utils.metrics import track_errors, track_time
def get_reply_keyboard_for_post():

17
helper_bot/main.py
View File

@@ -1,21 +1,24 @@
import asyncio
import logging
from typing import Optional
from aiogram import Bot, Dispatcher
from aiogram.client.default import DefaultBotProperties
from aiogram.fsm.storage.memory import MemoryStorage
from aiogram.fsm.strategy import FSMStrategy
import logging
import asyncio
from typing import Optional
from helper_bot.handlers.admin import admin_router
from helper_bot.handlers.callback import callback_router
from helper_bot.handlers.group import group_router
from helper_bot.handlers.private import private_router
from helper_bot.handlers.voice import VoiceHandlers
from helper_bot.middlewares.dependencies_middleware import DependenciesMiddleware
from helper_bot.middlewares.blacklist_middleware import BlacklistMiddleware
from helper_bot.middlewares.metrics_middleware import MetricsMiddleware, ErrorMetricsMiddleware
from helper_bot.middlewares.dependencies_middleware import \
DependenciesMiddleware
from helper_bot.middlewares.metrics_middleware import (ErrorMetricsMiddleware,
MetricsMiddleware)
from helper_bot.middlewares.rate_limit_middleware import RateLimitMiddleware
from helper_bot.server_prometheus import start_metrics_server, stop_metrics_server
from helper_bot.server_prometheus import (start_metrics_server,
stop_metrics_server)
async def start_bot_with_retry(bot: Bot, dp: Dispatcher, max_retries: int = 5, base_delay: float = 1.0):

2
helper_bot/middlewares/album_middleware.py
View File

@@ -1,5 +1,5 @@
import asyncio
from typing import Any, Dict, Union, List, Optional
from typing import Any, Dict, List, Optional, Union
from aiogram import BaseMiddleware
from aiogram.types import Message

4
helper_bot/middlewares/blacklist_middleware.py
View File

@@ -1,9 +1,9 @@
from typing import Dict, Any
import html
from datetime import datetime
from typing import Any, Dict
from aiogram import BaseMiddleware, types
from aiogram.types import TelegramObject, Message, CallbackQuery
from aiogram.types import CallbackQuery, Message, TelegramObject
from helper_bot.utils.base_dependency_factory import get_global_instance
from logs.custom_logger import logger

2
helper_bot/middlewares/dependencies_middleware.py
View File

@@ -1,7 +1,7 @@
from typing import Any, Dict
from aiogram import BaseMiddleware
from aiogram.types import TelegramObject
from helper_bot.utils.base_dependency_factory import get_global_instance
from logs.custom_logger import logger

28
helper_bot/middlewares/metrics_middleware.py
View File

@@ -3,25 +3,29 @@ Enhanced Metrics middleware for aiogram 3.x.
Automatically collects ALL available metrics for comprehensive monitoring.
"""
from typing import Any, Awaitable, Callable, Dict, Union, Optional
from aiogram import BaseMiddleware
from aiogram.types import TelegramObject, Message, CallbackQuery
from aiogram.enums import ChatType
import time
import logging
import asyncio
import logging
import time
from typing import Any, Awaitable, Callable, Dict, Optional, Union
from aiogram import BaseMiddleware
from aiogram.enums import ChatType
from aiogram.types import CallbackQuery, Message, TelegramObject
from ..utils.metrics import metrics
# Import button command mapping
try:
from ..handlers.private.constants import BUTTON_COMMAND_MAPPING
from ..handlers.admin.constants import (ADMIN_BUTTON_COMMAND_MAPPING,
ADMIN_COMMANDS)
from ..handlers.callback.constants import CALLBACK_COMMAND_MAPPING
from ..handlers.admin.constants import ADMIN_BUTTON_COMMAND_MAPPING, ADMIN_COMMANDS
from ..handlers.voice.constants import (
BUTTON_COMMAND_MAPPING as VOICE_BUTTON_COMMAND_MAPPING,
COMMAND_MAPPING as VOICE_COMMAND_MAPPING,
from ..handlers.private.constants import BUTTON_COMMAND_MAPPING
from ..handlers.voice.constants import \
BUTTON_COMMAND_MAPPING as VOICE_BUTTON_COMMAND_MAPPING
from ..handlers.voice.constants import \
CALLBACK_COMMAND_MAPPING as VOICE_CALLBACK_COMMAND_MAPPING
)
from ..handlers.voice.constants import \
COMMAND_MAPPING as VOICE_COMMAND_MAPPING
except ImportError:
# Fallback if constants not available
BUTTON_COMMAND_MAPPING = {}

11
helper_bot/middlewares/rate_limit_middleware.py
View File

@@ -1,13 +1,14 @@
"""
Middleware для автоматического применения rate limiting ко всем входящим сообщениям
"""
from typing import Callable, Dict, Any, Awaitable, Union
from aiogram import BaseMiddleware
from aiogram.types import Message, CallbackQuery, InlineQuery, ChatMemberUpdated, Update
from aiogram.exceptions import TelegramRetryAfter, TelegramAPIError
from logs.custom_logger import logger
from typing import Any, Awaitable, Callable, Dict, Union
from aiogram import BaseMiddleware
from aiogram.exceptions import TelegramAPIError, TelegramRetryAfter
from aiogram.types import (CallbackQuery, ChatMemberUpdated, InlineQuery,
Message, Update)
from helper_bot.utils.rate_limiter import telegram_rate_limiter
from logs.custom_logger import logger
class RateLimitMiddleware(BaseMiddleware):

4
helper_bot/server_prometheus.py
View File

@@ -5,8 +5,10 @@ Provides /metrics endpoint and health check for the bot.
"""
import asyncio
from aiohttp import web
from typing import Optional
from aiohttp import web
from .utils.metrics import metrics
# Импортируем логгер из проекта

10
helper_bot/utils/auto_unban_scheduler.py
View File

@@ -1,18 +1,14 @@
import asyncio
from datetime import datetime, timezone, timedelta
from datetime import datetime, timedelta, timezone
from typing import Optional
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.triggers.cron import CronTrigger
from helper_bot.utils.base_dependency_factory import get_global_instance
from logs.custom_logger import logger
from .metrics import (
track_time,
track_errors,
db_query_time
)
from .metrics import db_query_time, track_errors, track_time
class AutoUnbanScheduler:
"""

2
helper_bot/utils/base_dependency_factory.py
View File

@@ -1,9 +1,9 @@
import os
import sys
from typing import Optional
from dotenv import load_dotenv
from database.async_db import AsyncBotDB
from dotenv import load_dotenv
from helper_bot.utils.s3_storage import S3StorageService

26
helper_bot/utils/helper_func.py
View File

@@ -1,12 +1,12 @@
import asyncio
import html
import os
import random
import time
import tempfile
import asyncio
import time
from datetime import datetime, timedelta
from time import sleep
from typing import List, Dict, Any, Optional, TYPE_CHECKING, Union
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Union
try:
import emoji as _emoji_lib
@@ -16,20 +16,16 @@ except ImportError:
_emoji_lib_available = False
from aiogram import types
from aiogram.types import InputMediaPhoto, FSInputFile, InputMediaVideo, InputMediaAudio, InputMediaDocument
from helper_bot.utils.base_dependency_factory import BaseDependencyFactory, get_global_instance
from logs.custom_logger import logger
from aiogram.types import (FSInputFile, InputMediaAudio, InputMediaDocument,
InputMediaPhoto, InputMediaVideo)
from database.models import TelegramPost
from helper_bot.utils.base_dependency_factory import (BaseDependencyFactory,
get_global_instance)
from logs.custom_logger import logger
# Local imports - metrics
from .metrics import (
track_time,
track_errors,
db_query_time,
track_media_processing,
track_file_operations,
)
from .metrics import (db_query_time, track_errors, track_file_operations,
track_media_processing, track_time)
bdf = get_global_instance()
#TODO: поменять архитектуру и подключить правильный BotDB
@@ -653,7 +649,7 @@ async def send_media_group_to_channel(bot, chat_id: int, post_content: List, pos
@track_errors("helper_func", "send_text_message")
async def send_text_message(chat_id, message: types.Message, post_text: str, markup: types.ReplyKeyboardMarkup = None):
from .rate_limiter import send_with_rate_limit
# Экранируем post_text для безопасного использования в HTML
safe_post_text = html.escape(str(post_text)) if post_text else ""

7
helper_bot/utils/messages.py
View File

@@ -1,12 +1,7 @@
import html
# Local imports - metrics
from .metrics import (
metrics,
track_time,
track_errors
)
from .metrics import metrics, track_errors, track_time
constants = {
'HELLO_MESSAGE': "Привет, username!👋🏼&Меня зовут Виби, я бот канала 'Влюбленный Бийск'❤🤖"

16
helper_bot/utils/metrics.py
View File

@@ -3,14 +3,16 @@ Metrics module for Telegram bot monitoring with Prometheus.
Provides predefined metrics for bot commands, errors, performance, and user activity.
"""
from typing import Dict, Any, Optional
from prometheus_client import Counter, Histogram, Gauge, generate_latest, CONTENT_TYPE_LATEST
from prometheus_client.core import CollectorRegistry
import time
import os
from functools import wraps
import asyncio
import os
import time
from contextlib import asynccontextmanager
from functools import wraps
from typing import Any, Dict, Optional
from prometheus_client import (CONTENT_TYPE_LATEST, Counter, Gauge, Histogram,
generate_latest)
from prometheus_client.core import CollectorRegistry
# Метрики rate limiter теперь создаются в основном классе
@@ -387,7 +389,7 @@ class BotMetrics:
"""Update rate limit gauge metrics."""
try:
from .rate_limit_monitor import rate_limit_monitor
# Обновляем количество активных чатов
self.rate_limit_active_chats.set(len(rate_limit_monitor.stats))

5
helper_bot/utils/rate_limit_monitor.py
View File

@@ -2,9 +2,10 @@
Мониторинг и статистика rate limiting
"""
import time
from typing import Dict, List, Optional
from dataclasses import dataclass, field
from collections import defaultdict, deque
from dataclasses import dataclass, field
from typing import Dict, List, Optional
from logs.custom_logger import logger

10
helper_bot/utils/rate_limiter.py
View File

@@ -3,10 +3,12 @@ Rate limiter для предотвращения Flood control ошибок в T
"""
import asyncio
import time
from typing import Dict, Optional, Any, Callable
from dataclasses import dataclass
from aiogram.exceptions import TelegramRetryAfter, TelegramAPIError
from typing import Any, Callable, Dict, Optional
from aiogram.exceptions import TelegramAPIError, TelegramRetryAfter
from logs.custom_logger import logger
from .metrics import metrics
@@ -182,7 +184,9 @@ class TelegramRateLimiter:
# Глобальный экземпляр rate limiter
from helper_bot.config.rate_limit_config import get_rate_limit_config, RateLimitSettings
from helper_bot.config.rate_limit_config import (RateLimitSettings,
get_rate_limit_config)
def _create_rate_limit_config(settings: RateLimitSettings) -> RateLimitConfig:
"""Создает RateLimitConfig из RateLimitSettings"""

5
helper_bot/utils/s3_storage.py
View File

@@ -1,11 +1,12 @@
"""
Сервис для работы с S3 хранилищем.
"""
import aioboto3
import os
import tempfile
from typing import Optional
from pathlib import Path
from typing import Optional
import aioboto3
from logs.custom_logger import logger

2
helper_bot/utils/state.py
View File

@@ -1,4 +1,4 @@
from aiogram.fsm.state import StatesGroup, State
from aiogram.fsm.state import State, StatesGroup
class StateUser(StatesGroup):

1
logs/custom_logger.py
View File

@@ -1,6 +1,7 @@
import datetime
import os
import sys
from loguru import logger
# Remove default handler

8
run_helper.py
View File

@@ -1,8 +1,8 @@
import asyncio
import os
import sys
import signal
import sqlite3
import sys
# Ensure project root is on sys.path for module resolution
CURRENT_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -10,12 +10,11 @@ if CURRENT_DIR not in sys.path:
sys.path.insert(0, CURRENT_DIR)
from helper_bot.main import start_bot
from helper_bot.utils.base_dependency_factory import get_global_instance
from helper_bot.utils.auto_unban_scheduler import get_auto_unban_scheduler
from helper_bot.utils.base_dependency_factory import get_global_instance
from logs.custom_logger import logger
async def main():
"""Основная функция запуска"""
@@ -69,7 +68,8 @@ async def main():
# Останавливаем планировщик метрик
try:
from helper_bot.utils.metrics_scheduler import stop_metrics_scheduler
from helper_bot.utils.metrics_scheduler import \
stop_metrics_scheduler
stop_metrics_scheduler()
logger.info("Планировщик метрик остановлен")
except Exception as e:

68
scripts/add_ban_author_column_to_blacklist.py
View File

@@ -1,68 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт миграции для добавления колонки ban_author в таблицу blacklist.
Колонка хранит user_id администратора, инициировавшего бан.
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
import aiosqlite
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
from logs.custom_logger import logger # noqa: E402
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def _column_exists(rows: list, name: str) -> bool:
"""PRAGMA table_info returns (cid, name, type, notnull, dflt_value, pk)."""
for row in rows:
if row[1] == name:
return True
return False
async def main(db_path: str) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем наличие колонки ban_author
cursor = await conn.execute("PRAGMA table_info(blacklist)")
rows = await cursor.fetchall()
await cursor.close()
if not _column_exists(rows, "ban_author"):
logger.info("Добавление колонки ban_author в blacklist")
await conn.execute(
"ALTER TABLE blacklist "
"ADD COLUMN ban_author INTEGER REFERENCES our_users (user_id) ON DELETE SET NULL"
)
await conn.commit()
print("Колонка ban_author добавлена в таблицу blacklist.")
else:
print("Колонка ban_author уже существует в таблице blacklist.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Добавление колонки ban_author в blacklist"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
args = parser.parse_args()
asyncio.run(main(args.db))

138
scripts/add_is_anonymous_column.py
View File

@@ -1,138 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт миграции для добавления колонки is_anonymous в таблицу post_from_telegram_suggest.
Для существующих записей определяет is_anonymous на основе текста или устанавливает NULL.
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
from helper_bot.utils.helper_func import determine_anonymity
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def _column_exists(rows: list, name: str) -> bool:
"""PRAGMA table_info returns (cid, name, type, notnull, dflt_value, pk)."""
for row in rows:
if row[1] == name:
return True
return False
async def main(db_path: str) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем наличие колонки is_anonymous
cursor = await conn.execute(
"PRAGMA table_info(post_from_telegram_suggest)"
)
rows = await cursor.fetchall()
await cursor.close()
if not _column_exists(rows, "is_anonymous"):
logger.info("Добавление колонки is_anonymous в post_from_telegram_suggest")
await conn.execute(
"ALTER TABLE post_from_telegram_suggest "
"ADD COLUMN is_anonymous INTEGER"
)
await conn.commit()
print("Колонка is_anonymous добавлена.")
else:
print("Колонка is_anonymous уже существует.")
# Получаем все записи с текстом для обновления
cursor = await conn.execute(
"SELECT message_id, text FROM post_from_telegram_suggest WHERE text IS NOT NULL"
)
posts = await cursor.fetchall()
await cursor.close()
updated_count = 0
null_count = 0
# Обновляем каждую запись
for message_id, text in posts:
try:
# Определяем is_anonymous на основе текста
# Если текст пустой или None, устанавливаем NULL (legacy)
if not text or not text.strip():
is_anonymous = None
else:
is_anonymous = determine_anonymity(text)
# Преобразуем bool в int для SQLite (True -> 1, False -> 0, None -> None)
is_anonymous_int = None if is_anonymous is None else (1 if is_anonymous else 0)
await conn.execute(
"UPDATE post_from_telegram_suggest SET is_anonymous = ? WHERE message_id = ?",
(is_anonymous_int, message_id)
)
if is_anonymous is not None:
updated_count += 1
else:
null_count += 1
except Exception as e:
logger.error(f"Ошибка при обработке поста message_id={message_id}: {e}")
# В случае ошибки устанавливаем NULL
await conn.execute(
"UPDATE post_from_telegram_suggest SET is_anonymous = NULL WHERE message_id = ?",
(message_id,)
)
null_count += 1
# Обновляем записи без текста (устанавливаем NULL)
cursor = await conn.execute(
"SELECT COUNT(*) FROM post_from_telegram_suggest WHERE text IS NULL"
)
row = await cursor.fetchone()
posts_without_text = row[0] if row else 0
await cursor.close()
if posts_without_text > 0:
await conn.execute(
"UPDATE post_from_telegram_suggest SET is_anonymous = NULL WHERE text IS NULL"
)
null_count += posts_without_text
await conn.commit()
total_updated = updated_count + null_count
logger.info(
f"Миграция завершена. Обновлено записей: {total_updated} "
f"(определено: {updated_count}, установлено NULL: {null_count})"
)
print(f"Миграция завершена.")
print(f"Обновлено записей: {total_updated}")
print(f" - Определено is_anonymous: {updated_count}")
print(f" - Установлено NULL: {null_count}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Добавление колонки is_anonymous в post_from_telegram_suggest"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
args = parser.parse_args()
asyncio.run(main(args.db))

166
scripts/add_published_posts_support.py
View File

@@ -1,166 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт миграции для добавления поддержки опубликованных постов:
1. Добавляет колонку published_message_id в таблицу post_from_telegram_suggest
2. Создает таблицу published_post_content для хранения медиафайлов опубликованных постов
3. Создает индексы для производительности
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def _column_exists(rows: list, name: str) -> bool:
"""Проверяет существование колонки в таблице.
PRAGMA table_info returns (cid, name, type, notnull, dflt_value, pk)."""
for row in rows:
if row[1] == name:
return True
return False
async def main(db_path: str, dry_run: bool = False) -> None:
"""Выполняет миграцию БД для поддержки опубликованных постов."""
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
changes_made = []
# 1. Проверяем и добавляем колонку published_message_id
cursor = await conn.execute(
"PRAGMA table_info(post_from_telegram_suggest)"
)
rows = await cursor.fetchall()
await cursor.close()
if not _column_exists(rows, "published_message_id"):
if dry_run:
print("DRY RUN: Будет добавлена колонка published_message_id в post_from_telegram_suggest")
changes_made.append("Добавление колонки published_message_id")
else:
logger.info("Добавление колонки published_message_id в post_from_telegram_suggest")
await conn.execute(
"ALTER TABLE post_from_telegram_suggest "
"ADD COLUMN published_message_id INTEGER"
)
await conn.commit()
print("✓ Колонка published_message_id добавлена в post_from_telegram_suggest")
changes_made.append("Добавлена колонка published_message_id")
else:
print("✓ Колонка published_message_id уже существует в post_from_telegram_suggest")
# 2. Проверяем и создаем таблицу published_post_content
cursor = await conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='published_post_content'"
)
table_exists = await cursor.fetchone()
await cursor.close()
if not table_exists:
if dry_run:
print("DRY RUN: Будет создана таблица published_post_content")
changes_made.append("Создание таблицы published_post_content")
else:
logger.info("Создание таблицы published_post_content")
await conn.execute("""
CREATE TABLE IF NOT EXISTS published_post_content (
published_message_id INTEGER NOT NULL,
content_name TEXT NOT NULL,
content_type TEXT,
published_at INTEGER NOT NULL,
PRIMARY KEY (published_message_id, content_name)
)
""")
await conn.commit()
print("✓ Таблица published_post_content создана")
changes_made.append("Создана таблица published_post_content")
else:
print("✓ Таблица published_post_content уже существует")
# 3. Проверяем и создаем индексы
indexes = [
("idx_published_post_content_message_id",
"CREATE INDEX IF NOT EXISTS idx_published_post_content_message_id "
"ON published_post_content(published_message_id)"),
("idx_post_from_telegram_suggest_published",
"CREATE INDEX IF NOT EXISTS idx_post_from_telegram_suggest_published "
"ON post_from_telegram_suggest(published_message_id)")
]
for index_name, index_sql in indexes:
cursor = await conn.execute(
"SELECT name FROM sqlite_master WHERE type='index' AND name=?",
(index_name,)
)
index_exists = await cursor.fetchone()
await cursor.close()
if not index_exists:
if dry_run:
print(f"DRY RUN: Будет создан индекс {index_name}")
changes_made.append(f"Создание индекса {index_name}")
else:
logger.info(f"Создание индекса {index_name}")
await conn.execute(index_sql)
await conn.commit()
print(f"✓ Индекс {index_name} создан")
changes_made.append(f"Создан индекс {index_name}")
else:
print(f"✓ Индекс {index_name} уже существует")
# Финальная статистика
if dry_run:
if changes_made:
print("\n" + "="*60)
print("DRY RUN: Следующие изменения будут выполнены:")
for change in changes_made:
print(f" - {change}")
print("="*60)
else:
print("\n✓ Все необходимые изменения уже применены. Ничего делать не нужно.")
else:
if changes_made:
logger.info(f"Миграция завершена. Выполнено изменений: {len(changes_made)}")
print(f"\n✓ Миграция завершена успешно!")
print(f"Выполнено изменений: {len(changes_made)}")
for change in changes_made:
print(f" - {change}")
else:
print("\n✓ Все необходимые изменения уже применены. Ничего делать не нужно.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Добавление поддержки опубликованных постов в БД"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или переменная окружения DB_PATH)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Показать что будет сделано без выполнения изменений",
)
args = parser.parse_args()
asyncio.run(main(args.db, dry_run=args.dry_run))

241
scripts/apply_migrations.py Normal file
View File

@@ -0,0 +1,241 @@
#!/usr/bin/env python3
"""
Скрипт для автоматического применения миграций базы данных.
Сканирует папку scripts/ и применяет все новые миграции, которые еще не были применены.
"""
import argparse
import asyncio
import importlib.util
import os
import subprocess
import sys
from pathlib import Path
from typing import List, Tuple
# Исключаем служебные скрипты из миграций
EXCLUDED_SCRIPTS = {
'apply_migrations.py',
'test_s3_connection.py',
'voice_cleanup.py',
}
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def get_migration_scripts(scripts_dir: Path) -> List[Tuple[str, Path]]:
"""
Получает список скриптов миграций из папки scripts.
Возвращает список кортежей (имя_файла, путь_к_файлу), отсортированный по имени файла.
"""
scripts = []
for script_file in sorted(scripts_dir.glob("*.py")):
if script_file.name not in EXCLUDED_SCRIPTS:
scripts.append((script_file.name, script_file))
return scripts
async def is_migration_script(script_path: Path) -> bool:
"""
Проверяет, является ли скрипт миграцией.
Миграция должна иметь функцию main() с параметром db_path.
"""
try:
spec = importlib.util.spec_from_file_location("migration_script", script_path)
if spec is None or spec.loader is None:
return False
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
# Проверяем наличие функции main
if hasattr(module, 'main'):
import inspect
sig = inspect.signature(module.main)
# Проверяем, что функция принимает db_path
params = list(sig.parameters.keys())
return 'db_path' in params
return False
except Exception:
# Если не удалось проверить, считаем что это не миграция
return False
async def apply_migration(script_path: Path, db_path: str) -> bool:
"""
Применяет миграцию, запуская скрипт.
Returns:
True если миграция применена успешно, False в противном случае.
"""
script_name = script_path.name
try:
# Запускаем скрипт как отдельный процесс
result = subprocess.run(
[sys.executable, str(script_path), "--db", db_path],
cwd=script_path.parent.parent,
capture_output=True,
text=True,
timeout=300 # 5 минут максимум на миграцию
)
if result.returncode == 0:
if result.stdout:
print(f" {result.stdout.strip()}")
return True
else:
print(f" ❌ Ошибка:")
if result.stdout:
print(f" STDOUT: {result.stdout}")
if result.stderr:
print(f" STDERR: {result.stderr}")
return False
except subprocess.TimeoutExpired:
print(f" ❌ Превышен лимит времени (5 минут)")
return False
except Exception as e:
print(f" ❌ Ошибка: {e}")
return False
async def main(db_path: str, dry_run: bool = False) -> None:
"""
Основная функция для применения миграций.
Args:
db_path: Путь к базе данных
dry_run: Если True, только показывает какие миграции будут применены
"""
# Импортируем зависимости только когда они действительно нужны
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
# Проверяем наличие необходимых зависимостей
try:
import aiosqlite
except ImportError:
print("❌ Ошибка: модуль aiosqlite не установлен.")
print("💡 Установите зависимости: pip install -r requirements.txt")
sys.exit(1)
# Импортируем logger
try:
from logs.custom_logger import logger
except ImportError:
import logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)
# Импортируем MigrationRepository напрямую из файла
migration_repo_path = project_root / "database" / "repositories" / "migration_repository.py"
if not migration_repo_path.exists():
print(f"❌ Файл migration_repository.py не найден: {migration_repo_path}")
sys.exit(1)
spec = importlib.util.spec_from_file_location("migration_repository", migration_repo_path)
if spec is None or spec.loader is None:
print("❌ Не удалось загрузить модуль migration_repository")
sys.exit(1)
migration_module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(migration_module)
MigrationRepository = migration_module.MigrationRepository
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error(f"База данных не найдена: {db_path}")
print(f"❌ Ошибка: база данных не найдена: {db_path}")
return
scripts_dir = project_root / "scripts"
if not scripts_dir.exists():
logger.error(f"Папка scripts не найдена: {scripts_dir}")
print(f"❌ Ошибка: папка scripts не найдена: {scripts_dir}")
return
# Инициализируем репозиторий миграций напрямую
migration_repo = MigrationRepository(db_path)
await migration_repo.create_table()
# Получаем список примененных миграций
applied_migrations = await migration_repo.get_applied_migrations()
logger.info(f"Примененных миграций: {len(applied_migrations)}")
# Получаем все скрипты миграций
all_scripts = get_migration_scripts(scripts_dir)
# Фильтруем только миграции
migration_scripts = []
for script_name, script_path in all_scripts:
if await is_migration_script(script_path):
migration_scripts.append((script_name, script_path))
else:
logger.debug(f"Скрипт {script_name} не является миграцией, пропускаем")
# Находим новые миграции
new_migrations = [
(name, path) for name, path in migration_scripts
if name not in applied_migrations
]
if not new_migrations:
print("✅ Все миграции уже применены")
logger.info("Новых миграций не найдено")
return
print(f"📋 Найдено новых миграций: {len(new_migrations)}")
for name, _ in new_migrations:
print(f" - {name}")
if dry_run:
print("\n🔍 DRY RUN: миграции не будут применены")
return
# Применяем миграции по порядку
print("\n🚀 Применение миграций...")
failed_migrations = []
for script_name, script_path in new_migrations:
print(f"📝 {script_name}...", end=" ", flush=True)
success = await apply_migration(script_path, db_path)
if success:
# Отмечаем миграцию как примененную
await migration_repo.mark_migration_applied(script_name)
print("✅")
else:
failed_migrations.append(script_name)
print("❌")
logger.error(f"Не удалось применить миграцию: {script_name}")
# Прерываем выполнение при ошибке
print(f"\n⚠️ Прерывание: миграция {script_name} завершилась с ошибкой")
break
if failed_migrations:
print(f"\n❌ Не удалось применить {len(failed_migrations)} миграций:")
for name in failed_migrations:
print(f" - {name}")
sys.exit(1)
else:
print(f"\n✅ Все миграции применены успешно ({len(new_migrations)} шт.)")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Применение миграций базы данных"
)
parser.add_argument(
"--db",
default=os.environ.get("DATABASE_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DATABASE_PATH из env)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Показать какие миграции будут применены без фактического применения",
)
args = parser.parse_args()
asyncio.run(main(args.db, args.dry_run))

82
scripts/backfill_post_status_legacy.py
View File

@@ -1,82 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт для проставления status='legacy' всем существующим записям в post_from_telegram_suggest.
Добавляет колонку status, если её нет, затем обновляет все строки.
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def _column_exists(rows: list, name: str) -> bool:
"""PRAGMA table_info returns (cid, name, type, notnull, dflt_value, pk)."""
for row in rows:
if row[1] == name:
return True
return False
async def main(db_path: str) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем наличие колонки status
cursor = await conn.execute(
"PRAGMA table_info(post_from_telegram_suggest)"
)
rows = await cursor.fetchall()
await cursor.close()
if not _column_exists(rows, "status"):
logger.info("Добавление колонки status в post_from_telegram_suggest")
await conn.execute(
"ALTER TABLE post_from_telegram_suggest "
"ADD COLUMN status TEXT NOT NULL DEFAULT 'suggest'"
)
await conn.commit()
print("Колонка status добавлена.")
else:
print("Колонка status уже существует.")
# Обновляем все существующие записи на legacy
await conn.execute(
"UPDATE post_from_telegram_suggest SET status = 'legacy'"
)
await conn.commit()
cursor = await conn.execute("SELECT changes()")
row = await cursor.fetchone()
updated = row[0] if row else 0
await cursor.close()
logger.info("Обновлено записей в post_from_telegram_suggest: %d", updated)
print(f"Обновлено записей: {updated}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Backfill status='legacy' для post_from_telegram_suggest"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
args = parser.parse_args()
asyncio.run(main(args.db))

148
scripts/clean_post_text.py
View File

@@ -1,148 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт для приведения текста постов к "сырому" виду.
Удаляет форматирование, добавленное функцией get_text_message(), оставляя только исходный текст.
"""
import argparse
import asyncio
import html
import os
import re
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
# Паттерны для определения форматированного текста
PREFIX = "Пост из ТГ:\n"
ANONYMOUS_SUFFIX = "\n\nПост опубликован анонимно"
AUTHOR_SUFFIX_PATTERN = re.compile(r"\n\nАвтор поста: .+$")
def extract_raw_text(formatted_text: str) -> str:
"""
Извлекает сырой текст из форматированного текста поста.
Args:
formatted_text: Форматированный текст поста
Returns:
str: Сырой текст или исходный текст, если форматирование не обнаружено
"""
if not formatted_text:
return ""
# Проверяем, начинается ли текст с префикса
if not formatted_text.startswith(PREFIX):
# Текст уже в сыром виде или имеет другой формат
return formatted_text
# Извлекаем текст после префикса
text_after_prefix = formatted_text[len(PREFIX):]
# Проверяем, заканчивается ли текст на "Пост опубликован анонимно"
if text_after_prefix.endswith(ANONYMOUS_SUFFIX):
raw_text = text_after_prefix[:-len(ANONYMOUS_SUFFIX)]
# Проверяем, заканчивается ли текст на "Автор поста: ..."
elif AUTHOR_SUFFIX_PATTERN.search(text_after_prefix):
raw_text = AUTHOR_SUFFIX_PATTERN.sub("", text_after_prefix)
else:
# Не удалось определить формат, возвращаем текст без префикса
raw_text = text_after_prefix
# Декодируем HTML-экранирование
raw_text = html.unescape(raw_text)
return raw_text
async def main(db_path: str, dry_run: bool = False) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Получаем все записи с текстом
cursor = await conn.execute(
"SELECT message_id, text FROM post_from_telegram_suggest WHERE text IS NOT NULL AND text != ''"
)
posts = await cursor.fetchall()
await cursor.close()
updated_count = 0
skipped_count = 0
error_count = 0
print(f"Найдено записей для обработки: {len(posts)}")
if dry_run:
print("РЕЖИМ ПРОВЕРКИ (dry-run): изменения не будут сохранены")
# Обрабатываем каждую запись
for message_id, formatted_text in posts:
try:
# Извлекаем сырой текст
raw_text = extract_raw_text(formatted_text)
# Проверяем, изменился ли текст
if raw_text == formatted_text:
skipped_count += 1
continue
if dry_run:
print(f"\n[DRY-RUN] message_id={message_id}:")
print(f" Было: {formatted_text[:100]}...")
print(f" Станет: {raw_text[:100]}...")
else:
# Обновляем запись
await conn.execute(
"UPDATE post_from_telegram_suggest SET text = ? WHERE message_id = ?",
(raw_text, message_id)
)
updated_count += 1
except Exception as e:
logger.error(f"Ошибка при обработке поста message_id={message_id}: {e}")
error_count += 1
if not dry_run:
await conn.commit()
total_processed = updated_count + skipped_count + error_count
logger.info(
f"Обработка завершена. Всего записей: {total_processed}, "
f"обновлено: {updated_count}, пропущено: {skipped_count}, ошибок: {error_count}"
)
print(f"\nОбработка завершена:")
print(f" - Всего записей: {total_processed}")
print(f" - Обновлено: {updated_count}")
print(f" - Пропущено (уже в сыром виде): {skipped_count}")
print(f" - Ошибок: {error_count}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Приведение текста постов к 'сырому' виду"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Режим проверки без сохранения изменений",
)
args = parser.parse_args()
asyncio.run(main(args.db, args.dry_run))

94
scripts/create_blacklist_history_table.py
View File

@@ -1,94 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт миграции для создания таблицы blacklist_history.
Таблица хранит историю всех операций бана/разбана пользователей.
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
def _table_exists(rows: list, table_name: str) -> bool:
"""Проверяет существование таблицы по результатам PRAGMA table_list."""
for row in rows:
if row[1] == table_name: # name column
return True
return False
async def main(db_path: str) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем наличие таблицы blacklist_history
cursor = await conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='blacklist_history'"
)
rows = await cursor.fetchall()
await cursor.close()
if not rows:
logger.info("Создание таблицы blacklist_history")
# Создаем таблицу
await conn.execute("""
CREATE TABLE IF NOT EXISTS blacklist_history (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
message_for_user TEXT,
date_ban INTEGER NOT NULL,
date_unban INTEGER,
ban_author INTEGER,
created_at INTEGER DEFAULT (strftime('%s', 'now')),
updated_at INTEGER DEFAULT (strftime('%s', 'now')),
FOREIGN KEY (user_id) REFERENCES our_users(user_id) ON DELETE CASCADE,
FOREIGN KEY (ban_author) REFERENCES our_users(user_id) ON DELETE SET NULL
)
""")
# Создаем индексы
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_blacklist_history_user_id ON blacklist_history(user_id)"
)
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_blacklist_history_date_ban ON blacklist_history(date_ban)"
)
await conn.execute(
"CREATE INDEX IF NOT EXISTS idx_blacklist_history_date_unban ON blacklist_history(date_unban)"
)
await conn.commit()
logger.info("Таблица blacklist_history и индексы успешно созданы")
print("Таблица blacklist_history и индексы успешно созданы.")
else:
print("Таблица blacklist_history уже существует.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Создание таблицы blacklist_history для истории банов/разбанов"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
args = parser.parse_args()
asyncio.run(main(args.db))

125
scripts/migrate_blacklist_to_history.py
View File

@@ -1,125 +0,0 @@
#!/usr/bin/env python3
"""
Скрипт миграции для переноса записей из blacklist в blacklist_history.
Переносит все существующие записи из таблицы blacklist в таблицу blacklist_history.
"""
import argparse
import asyncio
import os
import sys
from pathlib import Path
from datetime import datetime
project_root = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(project_root))
import aiosqlite
from logs.custom_logger import logger
DEFAULT_DB_PATH = "database/tg-bot-database.db"
async def main(db_path: str) -> None:
db_path = os.path.abspath(db_path)
if not os.path.exists(db_path):
logger.error("База данных не найдена: %s", db_path)
print(f"Ошибка: база данных не найдена: {db_path}")
return
async with aiosqlite.connect(db_path) as conn:
await conn.execute("PRAGMA foreign_keys = ON")
# Проверяем наличие таблицы blacklist_history
cursor = await conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='blacklist_history'"
)
rows = await cursor.fetchall()
await cursor.close()
if not rows:
logger.error("Таблица blacklist_history не найдена. Сначала запустите create_blacklist_history_table.py")
print("Ошибка: таблица blacklist_history не найдена. Сначала запустите create_blacklist_history_table.py")
return
# Получаем все записи из blacklist
cursor = await conn.execute(
"SELECT user_id, message_for_user, date_to_unban, created_at, ban_author FROM blacklist"
)
blacklist_records = await cursor.fetchall()
await cursor.close()
if not blacklist_records:
print("В таблице blacklist нет записей для переноса.")
logger.info("В таблице blacklist нет записей для переноса")
return
logger.info("Найдено записей в blacklist для переноса: %d", len(blacklist_records))
print(f"Найдено записей в blacklist для переноса: {len(blacklist_records)}")
# Получаем текущее время в Unix timestamp
current_time = int(datetime.now().timestamp())
# Переносим записи в blacklist_history
migrated_count = 0
skipped_count = 0
for record in blacklist_records:
user_id, message_for_user, date_to_unban, created_at, ban_author = record
# Проверяем, нет ли уже записи для этого user_id с таким же date_ban
# (чтобы избежать дубликатов при повторном запуске)
date_ban = created_at if created_at is not None else current_time
check_cursor = await conn.execute(
"SELECT id FROM blacklist_history WHERE user_id = ? AND date_ban = ?",
(user_id, date_ban)
)
existing = await check_cursor.fetchone()
await check_cursor.close()
if existing:
logger.debug("Запись для user_id=%d с date_ban=%d уже существует, пропускаем", user_id, date_ban)
skipped_count += 1
continue
# Вставляем запись в blacklist_history
await conn.execute(
"""
INSERT INTO blacklist_history
(user_id, message_for_user, date_ban, date_unban, ban_author, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
user_id,
message_for_user,
date_ban,
date_to_unban,
ban_author,
created_at if created_at is not None else current_time,
current_time
)
)
migrated_count += 1
await conn.commit()
logger.info(
"Миграция завершена. Перенесено записей: %d, пропущено (дубликаты): %d",
migrated_count,
skipped_count
)
print(f"Миграция завершена. Перенесено записей: {migrated_count}, пропущено (дубликаты): {skipped_count}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Перенос записей из blacklist в blacklist_history"
)
parser.add_argument(
"--db",
default=os.environ.get("DB_PATH", DEFAULT_DB_PATH),
help="Путь к БД (или DB_PATH)",
)
args = parser.parse_args()
asyncio.run(main(args.db))

1
scripts/test_s3_connection.py
View File

@@ -13,6 +13,7 @@ sys.path.insert(0, str(project_root))
# Загружаем .env файл
from dotenv import load_dotenv
env_path = os.path.join(project_root, '.env')
if os.path.exists(env_path):
load_dotenv(env_path)

2
scripts/voice_cleanup.py
View File

@@ -3,8 +3,8 @@
Скрипт для диагностики и очистки проблем с голосовыми файлами
"""
import asyncio
import sys
import os
import sys
from pathlib import Path
# Добавляем корневую директорию проекта в путь

8
tests/conftest.py
View File

@@ -1,11 +1,11 @@
import pytest
import asyncio
import os
import sys
from unittest.mock import Mock, AsyncMock, patch
from aiogram.types import Message, User, Chat
from aiogram.fsm.context import FSMContext
from unittest.mock import AsyncMock, Mock, patch
import pytest
from aiogram.fsm.context import FSMContext
from aiogram.types import Chat, Message, User
from database.async_db import AsyncBotDB
# Импортируем моки в самом начале

7
tests/conftest_message_repository.py
View File

@@ -1,9 +1,10 @@
import pytest
import tempfile
import os
import tempfile
from datetime import datetime
from database.repositories.message_repository import MessageRepository
import pytest
from database.models import UserMessage
from database.repositories.message_repository import MessageRepository
@pytest.fixture(scope="session")

7
tests/conftest_post_repository.py
View File

@@ -1,11 +1,12 @@
import pytest
import asyncio
import os
import tempfile
from datetime import datetime
from unittest.mock import Mock, AsyncMock
from unittest.mock import AsyncMock, Mock
import pytest
from database.models import MessageContentLink, PostContent, TelegramPost
from database.repositories.post_repository import PostRepository
from database.models import TelegramPost, PostContent, MessageContentLink
@pytest.fixture(scope="session")

3
tests/mocks.py
View File

@@ -1,10 +1,11 @@
"""
Моки для тестового окружения
"""
import sys
import os
import sys
from unittest.mock import Mock, patch
# Патчим загрузку настроек до импорта модулей
def setup_test_mocks():
"""Настройка моков для тестов"""

8
tests/test_admin_repository.py
View File

@@ -1,10 +1,10 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, patch
from database.repositories.admin_repository import AdminRepository
import pytest
from database.models import Admin
from database.repositories.admin_repository import AdminRepository
class TestAdminRepository:

3
tests/test_async_db.py
View File

@@ -1,5 +1,6 @@
from unittest.mock import AsyncMock, Mock, patch
import pytest
from unittest.mock import Mock, AsyncMock, patch
from database.async_db import AsyncBotDB

9
tests/test_audio_file_service.py
View File

@@ -1,10 +1,11 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock, mock_open
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, mock_open, patch
import pytest
from helper_bot.handlers.voice.exceptions import (DatabaseError,
FileOperationError)
from helper_bot.handlers.voice.services import AudioFileService
from helper_bot.handlers.voice.exceptions import FileOperationError, DatabaseError
@pytest.fixture

8
tests/test_audio_repository.py
View File

@@ -1,10 +1,10 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import pytest
from database.models import AudioListenRecord, AudioMessage, AudioModerate
from database.repositories.audio_repository import AudioRepository
from database.models import AudioMessage, AudioListenRecord, AudioModerate
class TestAudioRepository:

6
tests/test_audio_repository_schema.py
View File

@@ -1,8 +1,8 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import pytest
from database.repositories.audio_repository import AudioRepository

11
tests/test_auto_unban_integration.py
View File

@@ -1,9 +1,9 @@
import pytest
import sqlite3
import os
from datetime import datetime, timezone, timedelta
from unittest.mock import Mock, patch, AsyncMock
import sqlite3
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock, Mock, patch
import pytest
from helper_bot.utils.auto_unban_scheduler import AutoUnbanScheduler
@@ -155,8 +155,9 @@ class TestAutoUnbanIntegration:
}
# Создаем реальный экземпляр базы данных с тестовым файлом
from database.async_db import AsyncBotDB
import os
from database.async_db import AsyncBotDB
mock_factory.database = AsyncBotDB(test_db_path)
return mock_factory

9
tests/test_auto_unban_scheduler.py
View File

@@ -1,9 +1,10 @@
import pytest
import asyncio
from datetime import datetime, timezone, timedelta
from unittest.mock import Mock, patch, AsyncMock
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock, Mock, patch
from helper_bot.utils.auto_unban_scheduler import AutoUnbanScheduler, get_auto_unban_scheduler
import pytest
from helper_bot.utils.auto_unban_scheduler import (AutoUnbanScheduler,
get_auto_unban_scheduler)
class TestAutoUnbanScheduler:

9
tests/test_blacklist_history_repository.py
View File

@@ -1,10 +1,11 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, Mock, patch
from database.repositories.blacklist_history_repository import BlacklistHistoryRepository
import pytest
from database.models import BlacklistHistoryRecord
from database.repositories.blacklist_history_repository import \
BlacklistHistoryRepository
class TestBlacklistHistoryRepository:

8
tests/test_blacklist_repository.py
View File

@@ -1,10 +1,10 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, patch
from database.repositories.blacklist_repository import BlacklistRepository
import pytest
from database.models import BlacklistUser
from database.repositories.blacklist_repository import BlacklistRepository
class TestBlacklistRepository:

12
tests/test_callback_handlers.py
View File

@@ -1,13 +1,11 @@
import pytest
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from datetime import datetime
import time
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import pytest
from helper_bot.handlers.callback.callback_handlers import (
save_voice_message,
delete_voice_message
)
from helper_bot.handlers.voice.constants import CALLBACK_SAVE, CALLBACK_DELETE
delete_voice_message, save_voice_message)
from helper_bot.handlers.voice.constants import CALLBACK_DELETE, CALLBACK_SAVE
@pytest.fixture

13
tests/test_improved_media_processing.py
View File

@@ -2,18 +2,15 @@
Тесты для улучшенных методов обработки медиа
"""
import pytest
import os
import tempfile
from unittest.mock import Mock, AsyncMock, patch, MagicMock
from aiogram import types
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import pytest
from aiogram import types
from helper_bot.utils.helper_func import (
download_file,
add_in_db_media,
add_in_db_media_mediagroup,
send_media_group_message_to_private_chat
)
add_in_db_media, add_in_db_media_mediagroup, download_file,
send_media_group_message_to_private_chat)
class TestDownloadFile:

21
tests/test_keyboards_and_filters.py
View File

@@ -1,16 +1,15 @@
import pytest
from unittest.mock import Mock, patch, AsyncMock
from aiogram.types import ReplyKeyboardMarkup, KeyboardButton, InlineKeyboardMarkup, InlineKeyboardButton
from unittest.mock import AsyncMock, Mock, patch
from helper_bot.keyboards.keyboards import (
get_reply_keyboard,
get_reply_keyboard_admin,
get_reply_keyboard_for_post,
get_reply_keyboard_leave_chat,
create_keyboard_with_pagination
)
from helper_bot.filters.main import ChatTypeFilter
import pytest
from aiogram.types import (InlineKeyboardButton, InlineKeyboardMarkup,
KeyboardButton, ReplyKeyboardMarkup)
from database.async_db import AsyncBotDB
from helper_bot.filters.main import ChatTypeFilter
from helper_bot.keyboards.keyboards import (create_keyboard_with_pagination,
get_reply_keyboard,
get_reply_keyboard_admin,
get_reply_keyboard_for_post,
get_reply_keyboard_leave_chat)
class TestKeyboards:

5
tests/test_message_repository.py
View File

@@ -1,9 +1,10 @@
import pytest
import asyncio
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock
from database.repositories.message_repository import MessageRepository
import pytest
from database.models import UserMessage
from database.repositories.message_repository import MessageRepository
class TestMessageRepository:

7
tests/test_message_repository_integration.py
View File

@@ -1,10 +1,11 @@
import pytest
import asyncio
import tempfile
import os
import tempfile
from datetime import datetime
from database.repositories.message_repository import MessageRepository
import pytest
from database.models import UserMessage
from database.repositories.message_repository import MessageRepository
class TestMessageRepositoryIntegration:

5
tests/test_post_repository.py
View File

@@ -1,9 +1,10 @@
import pytest
import asyncio
from datetime import datetime
from unittest.mock import AsyncMock, MagicMock
import pytest
from database.models import MessageContentLink, PostContent, TelegramPost
from database.repositories.post_repository import PostRepository
from database.models import TelegramPost, PostContent, MessageContentLink
class TestPostRepository:

5
tests/test_post_repository_integration.py
View File

@@ -1,10 +1,11 @@
import pytest
import asyncio
import os
import tempfile
from datetime import datetime
import pytest
from database.models import MessageContentLink, PostContent, TelegramPost
from database.repositories.post_repository import PostRepository
from database.models import TelegramPost, PostContent, MessageContentLink
class TestPostRepositoryIntegration:

8
tests/test_post_service.py
View File

@@ -1,12 +1,12 @@
"""Tests for PostService"""
import pytest
from unittest.mock import Mock, AsyncMock, MagicMock, patch
from datetime import datetime
from aiogram import types
from unittest.mock import AsyncMock, MagicMock, Mock, patch
from helper_bot.handlers.private.services import PostService, BotSettings
import pytest
from aiogram import types
from database.models import TelegramPost, User
from helper_bot.handlers.private.services import BotSettings, PostService
class TestPostService:

21
tests/test_rate_limiter.py
View File

@@ -3,19 +3,18 @@
"""
import asyncio
import time
import pytest
from unittest.mock import AsyncMock, MagicMock, patch
from helper_bot.utils.rate_limiter import (
RateLimitConfig,
ChatRateLimiter,
GlobalRateLimiter,
RetryHandler,
TelegramRateLimiter,
send_with_rate_limit
)
from helper_bot.utils.rate_limit_monitor import RateLimitMonitor, RateLimitStats, record_rate_limit_request
from helper_bot.config.rate_limit_config import RateLimitSettings, get_rate_limit_config
import pytest
from helper_bot.config.rate_limit_config import (RateLimitSettings,
get_rate_limit_config)
from helper_bot.utils.rate_limit_monitor import (RateLimitMonitor,
RateLimitStats,
record_rate_limit_request)
from helper_bot.utils.rate_limiter import (ChatRateLimiter, GlobalRateLimiter,
RateLimitConfig, RetryHandler,
TelegramRateLimiter,
send_with_rate_limit)
class TestRateLimitConfig:

14
tests/test_refactored_admin_handlers.py
View File

@@ -1,14 +1,12 @@
from unittest.mock import AsyncMock, Mock, patch
import pytest
from unittest.mock import Mock, AsyncMock, patch
from aiogram import types
from aiogram.fsm.context import FSMContext
from helper_bot.handlers.admin.services import AdminService, User, BannedUser
from helper_bot.handlers.admin.exceptions import (
UserNotFoundError,
UserAlreadyBannedError,
InvalidInputError
)
from helper_bot.handlers.admin.exceptions import (InvalidInputError,
UserAlreadyBannedError,
UserNotFoundError)
from helper_bot.handlers.admin.services import AdminService, BannedUser, User
class TestAdminService:

Some files were not shown because too many files have changed in this diff Show More