refactor: remove RabbitMQ from the project

- Deleted RabbitMQ service from docker-compose configurations and related files.
- Updated environment variables and documentation to reflect the removal of RabbitMQ.
- Adjusted setup instructions and code to focus solely on Redis for backend operations.
This commit is contained in:
mykonos-ibiza 2025-08-05 21:29:36 +05:30
parent 09e5dfc012
commit 10d32c0487
11 changed files with 1381 additions and 1471 deletions

1
.gitignore vendored
View File

@ -192,7 +192,6 @@ supabase/.temp/storage-version
.env.scripts .env.scripts
redis_data redis_data
rabbitmq_data
.setup_progress .setup_progress

View File

@ -14,9 +14,6 @@ REDIS_PORT=6379
REDIS_PASSWORD= REDIS_PASSWORD=
REDIS_SSL=false REDIS_SSL=false
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
# LLM Providers: # LLM Providers:
ANTHROPIC_API_KEY= ANTHROPIC_API_KEY=
OPENAI_API_KEY= OPENAI_API_KEY=

View File

@ -23,10 +23,10 @@ docker compose down && docker compose up --build
You can run individual services from the docker-compose file. This is particularly useful during development: You can run individual services from the docker-compose file. This is particularly useful during development:
### Running only Redis and RabbitMQ ### Running only Redis
```bash ```bash
docker compose up redis rabbitmq docker compose up redis
``` ```
### Running only the API and Worker ### Running only the API and Worker
@ -37,16 +37,16 @@ docker compose up api worker
## Development Setup ## Development Setup
For local development, you might only need to run Redis and RabbitMQ, while working on the API locally. This is useful when: For local development, you might only need to run Redis, while working on the API locally. This is useful when:
- You're making changes to the API code and want to test them directly - You're making changes to the API code and want to test them directly
- You want to avoid rebuilding the API container on every change - You want to avoid rebuilding the API container on every change
- You're running the API service directly on your machine - You're running the API service directly on your machine
To run just Redis and RabbitMQ for development: To run just Redis for development:
```bash ```bash
docker compose up redis rabbitmq docker compose up redis
``` ```
Then you can run your API service locally with the following commands: Then you can run your API service locally with the following commands:
@ -79,9 +79,6 @@ SUPABASE_SERVICE_ROLE_KEY=your-service-role-key
# Infrastructure # Infrastructure
REDIS_HOST=redis # Use 'localhost' when running API locally REDIS_HOST=redis # Use 'localhost' when running API locally
REDIS_PORT=6379 REDIS_PORT=6379
RABBITMQ_HOST=rabbitmq # Use 'localhost' when running API locally
RABBITMQ_PORT=5672
# LLM Providers (at least one required) # LLM Providers (at least one required)
ANTHROPIC_API_KEY=your-anthropic-key ANTHROPIC_API_KEY=your-anthropic-key
OPENAI_API_KEY=your-openai-key OPENAI_API_KEY=your-openai-key
@ -119,8 +116,7 @@ When running services individually, make sure to:
1. Check your `.env` file and adjust any necessary environment variables 1. Check your `.env` file and adjust any necessary environment variables
2. Ensure Redis connection settings match your local setup (default: `localhost:6379`) 2. Ensure Redis connection settings match your local setup (default: `localhost:6379`)
3. Ensure RabbitMQ connection settings match your local setup (default: `localhost:5672`) 3. Update any service-specific environment variables if needed
4. Update any service-specific environment variables if needed
### Important: Redis Host Configuration ### Important: Redis Host Configuration
@ -129,22 +125,12 @@ When running the API locally with Redis in Docker, you need to set the correct R
- For Docker-to-Docker communication (when running both services in Docker): use `REDIS_HOST=redis` - For Docker-to-Docker communication (when running both services in Docker): use `REDIS_HOST=redis`
- For local-to-Docker communication (when running API locally): use `REDIS_HOST=localhost` - For local-to-Docker communication (when running API locally): use `REDIS_HOST=localhost`
### Important: RabbitMQ Host Configuration
When running the API locally with RabbitMQ in Docker, you need to set the correct RabbitMQ host in your `.env` file:
- For Docker-to-Docker communication (when running both services in Docker): use `RABBITMQ_HOST=rabbitmq`
- For local-to-Docker communication (when running API locally): use `RABBITMQ_HOST=localhost`
Example `.env` configuration for local development: Example `.env` configuration for local development:
```sh ```sh
REDIS_HOST=localhost # (instead of 'redis') REDIS_HOST=localhost # (instead of 'redis')
REDIS_PORT=6379 REDIS_PORT=6379
REDIS_PASSWORD= REDIS_PASSWORD=
RABBITMQ_HOST=localhost # (instead of 'rabbitmq')
RABBITMQ_PORT=5672
``` ```
--- ---

View File

@ -17,8 +17,6 @@ services:
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
rabbitmq:
condition: service_healthy
networks: networks:
- app-network - app-network
environment: environment:
@ -26,8 +24,6 @@ services:
- REDIS_PORT=6379 - REDIS_PORT=6379
- REDIS_PASSWORD= - REDIS_PASSWORD=
- LOG_LEVEL=INFO - LOG_LEVEL=INFO
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
logging: logging:
driver: "json-file" driver: "json-file"
options: options:
@ -57,8 +53,6 @@ services:
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
rabbitmq:
condition: service_healthy
networks: networks:
- app-network - app-network
environment: environment:
@ -66,8 +60,6 @@ services:
- REDIS_PORT=6379 - REDIS_PORT=6379
- REDIS_PASSWORD= - REDIS_PASSWORD=
- LOG_LEVEL=INFO - LOG_LEVEL=INFO
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
logging: logging:
driver: "json-file" driver: "json-file"
options: options:
@ -102,31 +94,9 @@ services:
max-size: "10m" max-size: "10m"
max-file: "3" max-file: "3"
rabbitmq:
image: rabbitmq
ports:
- "127.0.0.1:5672:5672"
volumes:
- rabbitmq_data:/var/lib/rabbitmq
restart: unless-stopped
networks:
- app-network
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks: networks:
app-network: app-network:
driver: bridge driver: bridge
volumes: volumes:
redis_data: redis_data:
rabbitmq_data:

View File

@ -45,7 +45,6 @@ dependencies = [
"pytesseract==0.3.13", "pytesseract==0.3.13",
"stripe==11.6.0", "stripe==11.6.0",
"dramatiq==1.18.0", "dramatiq==1.18.0",
"pika==1.3.2",
"prometheus-client==0.21.1", "prometheus-client==0.21.1",
"langfuse==2.60.5", "langfuse==2.60.5",
"Pillow>=10.4.0", "Pillow>=10.4.0",

View File

@ -15,7 +15,7 @@ import uuid
from agentpress.thread_manager import ThreadManager from agentpress.thread_manager import ThreadManager
from services.supabase import DBConnection from services.supabase import DBConnection
from services import redis from services import redis
from dramatiq.brokers.rabbitmq import RabbitmqBroker from dramatiq.brokers.redis import RedisBroker
import os import os
from services.langfuse import langfuse from services.langfuse import langfuse
from utils.retry import retry from utils.retry import retry
@ -23,10 +23,11 @@ from utils.retry import retry
import sentry_sdk import sentry_sdk
from typing import Dict, Any from typing import Dict, Any
rabbitmq_host = os.getenv('RABBITMQ_HOST', 'rabbitmq') redis_host = os.getenv('REDIS_HOST', 'redis')
rabbitmq_port = int(os.getenv('RABBITMQ_PORT', 5672)) redis_port = int(os.getenv('REDIS_PORT', 6379))
rabbitmq_broker = RabbitmqBroker(host=rabbitmq_host, port=rabbitmq_port, middleware=[dramatiq.middleware.AsyncIO()]) redis_broker = RedisBroker(host=redis_host, port=redis_port, middleware=[dramatiq.middleware.AsyncIO()])
dramatiq.set_broker(rabbitmq_broker)
dramatiq.set_broker(redis_broker)
_initialized = False _initialized = False

File diff suppressed because it is too large Load Diff

View File

@ -13,21 +13,6 @@ services:
timeout: 5s timeout: 5s
retries: 3 retries: 3
rabbitmq:
image: rabbitmq
ports:
- "5672:5672"
- "15672:15672"
volumes:
- rabbitmq_data:/var/lib/rabbitmq
restart: unless-stopped
healthcheck:
test: ["CMD", "rabbitmq-diagnostics", "-q", "ping"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
backend: backend:
image: ghcr.io/suna-ai/suna-backend:latest image: ghcr.io/suna-ai/suna-backend:latest
platform: linux/amd64 platform: linux/amd64
@ -45,13 +30,9 @@ services:
- REDIS_PORT=6379 - REDIS_PORT=6379
- REDIS_PASSWORD= - REDIS_PASSWORD=
- REDIS_SSL=False - REDIS_SSL=False
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
rabbitmq:
condition: service_healthy
worker: worker:
condition: service_started condition: service_started
@ -71,13 +52,9 @@ services:
- REDIS_PORT=6379 - REDIS_PORT=6379
- REDIS_PASSWORD= - REDIS_PASSWORD=
- REDIS_SSL=False - REDIS_SSL=False
- RABBITMQ_HOST=rabbitmq
- RABBITMQ_PORT=5672
depends_on: depends_on:
redis: redis:
condition: service_healthy condition: service_healthy
rabbitmq:
condition: service_healthy
frontend: frontend:
init: true init: true
@ -93,4 +70,3 @@ services:
volumes: volumes:
redis_data: redis_data:
rabbitmq_data:

View File

@ -164,10 +164,6 @@ REDIS_PORT=6379
REDIS_PASSWORD= REDIS_PASSWORD=
REDIS_SSL=false REDIS_SSL=false
# RABBITMQ
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
# LLM Providers # LLM Providers
ANTHROPIC_API_KEY=your-anthropic-key ANTHROPIC_API_KEY=your-anthropic-key
OPENAI_API_KEY=your-openai-key OPENAI_API_KEY=your-openai-key
@ -247,10 +243,10 @@ python start.py # Use the same to stop it later
This method requires you to start each component separately: This method requires you to start each component separately:
1. Start Redis and RabbitMQ (required for backend): 1. Start Redis (required for backend):
```bash ```bash
docker compose up redis rabbitmq -d docker compose up redis -d
# or # or
python start.py # Use the same to stop it later python start.py # Use the same to stop it later
``` ```

View File

@ -464,7 +464,7 @@ class SetupWizard:
"uv": "https://github.com/astral-sh/uv#installation", "uv": "https://github.com/astral-sh/uv#installation",
"node": "https://nodejs.org/en/download/", "node": "https://nodejs.org/en/download/",
"npm": "https://docs.npmjs.com/downloading-and-installing-node-js-and-npm", "npm": "https://docs.npmjs.com/downloading-and-installing-node-js-and-npm",
"docker": "https://docs.docker.com/get-docker/", # For Redis/RabbitMQ "docker": "https://docs.docker.com/get-docker/", # For Redis
} }
missing = [] missing = []
@ -1114,15 +1114,12 @@ class SetupWizard:
# --- Backend .env --- # --- Backend .env ---
is_docker = self.env_vars["setup_method"] == "docker" is_docker = self.env_vars["setup_method"] == "docker"
redis_host = "redis" if is_docker else "localhost" redis_host = "redis" if is_docker else "localhost"
rabbitmq_host = "rabbitmq" if is_docker else "localhost"
backend_env = { backend_env = {
"ENV_MODE": "local", "ENV_MODE": "local",
**self.env_vars["supabase"], **self.env_vars["supabase"],
"REDIS_HOST": redis_host, "REDIS_HOST": redis_host,
"REDIS_PORT": "6379", "REDIS_PORT": "6379",
"RABBITMQ_HOST": rabbitmq_host,
"RABBITMQ_PORT": "5672",
**self.env_vars["llm"], **self.env_vars["llm"],
**self.env_vars["search"], **self.env_vars["search"],
**self.env_vars["rapidapi"], **self.env_vars["rapidapi"],
@ -1372,7 +1369,7 @@ class SetupWizard:
print( print(
f"\n{Colors.BOLD}1. Start Infrastructure (in project root):{Colors.ENDC}" f"\n{Colors.BOLD}1. Start Infrastructure (in project root):{Colors.ENDC}"
) )
print(f"{Colors.CYAN} docker compose up redis rabbitmq -d{Colors.ENDC}") print(f"{Colors.CYAN} docker compose up redis -d{Colors.ENDC}")
print(f"\n{Colors.BOLD}2. Start Frontend (in a new terminal):{Colors.ENDC}") print(f"\n{Colors.BOLD}2. Start Frontend (in a new terminal):{Colors.ENDC}")
print(f"{Colors.CYAN} cd frontend && npm run dev{Colors.ENDC}") print(f"{Colors.CYAN} cd frontend && npm run dev{Colors.ENDC}")

View File

@ -57,7 +57,7 @@ def print_manual_instructions():
print("To start Suna, you need to run these commands in separate terminals:\n") print("To start Suna, you need to run these commands in separate terminals:\n")
print(f"{Colors.BOLD}1. Start Infrastructure (in project root):{Colors.ENDC}") print(f"{Colors.BOLD}1. Start Infrastructure (in project root):{Colors.ENDC}")
print(f"{Colors.CYAN} docker compose up redis rabbitmq -d{Colors.ENDC}\n") print(f"{Colors.CYAN} docker compose up redis -d{Colors.ENDC}\n")
print(f"{Colors.BOLD}2. Start Frontend (in a new terminal):{Colors.ENDC}") print(f"{Colors.BOLD}2. Start Frontend (in a new terminal):{Colors.ENDC}")
print(f"{Colors.CYAN} cd frontend && npm run dev{Colors.ENDC}\n") print(f"{Colors.CYAN} cd frontend && npm run dev{Colors.ENDC}\n")
@ -96,17 +96,17 @@ def main():
setup_method = "docker" setup_method = "docker"
if setup_method == "manual": if setup_method == "manual":
# For manual setup, we only manage infrastructure services (redis, rabbitmq) # For manual setup, we only manage infrastructure services (redis)
# and show instructions for the rest # and show instructions for the rest
print(f"{Colors.BLUE}{Colors.BOLD}Manual Setup Detected{Colors.ENDC}") print(f"{Colors.BLUE}{Colors.BOLD}Manual Setup Detected{Colors.ENDC}")
print("Managing infrastructure services (Redis, RabbitMQ)...\n") print("Managing infrastructure services (Redis)...\n")
force = "-f" in sys.argv force = "-f" in sys.argv
if force: if force:
print("Force awakened. Skipping confirmation.") print("Force awakened. Skipping confirmation.")
is_infra_up = subprocess.run( is_infra_up = subprocess.run(
["docker", "compose", "ps", "-q", "redis", "rabbitmq"], ["docker", "compose", "ps", "-q", "redis"],
capture_output=True, capture_output=True,
text=True, text=True,
shell=IS_WINDOWS, shell=IS_WINDOWS,
@ -136,7 +136,7 @@ def main():
print(f"\n{Colors.GREEN}✅ Infrastructure services stopped.{Colors.ENDC}") print(f"\n{Colors.GREEN}✅ Infrastructure services stopped.{Colors.ENDC}")
else: else:
subprocess.run( subprocess.run(
["docker", "compose", "up", "redis", "rabbitmq", "-d"], shell=IS_WINDOWS ["docker", "compose", "up", "redis", "-d"], shell=IS_WINDOWS
) )
print(f"\n{Colors.GREEN}✅ Infrastructure services started.{Colors.ENDC}") print(f"\n{Colors.GREEN}✅ Infrastructure services started.{Colors.ENDC}")
print_manual_instructions() print_manual_instructions()