Scheduled & Background Execution

Running Cleo as a persistent background process or in a scheduled manner is ideal for production systems where agents need to operate continuously or on a schedule without manual intervention. This guide covers methods for setting up Cleo in such an environment, from CRON jobs to process managers like systemd.

---

Running Cleo as a Background Service

In production environments, it's crucial that Cleo runs as a persistent background service, ensuring that agents stay active, tasks continue to execute, and any interruptions (such as reboots) are handled gracefully.

1. Using systemd

systemd is a system and service manager for Linux systems, allowing for background services that can be easily controlled, monitored, and restarted if needed.

a. Install systemd (if not installed)

Most Linux distributions, such as Ubuntu or CentOS, already include systemd. To confirm:

bashCopyEditsystemctl --version

b. Create a systemd Service File

Create a systemd service unit file to control Cleo. For example, create a file at /etc/systemd/system/cleo-agent.service:

iniCopyEdit[Unit]
Description=Cleo Agent Service
After=network.target

[Service]
ExecStart=/usr/bin/python /path/to/Cleo/main.py
WorkingDirectory=/path/to/Cleo/
Environment="PATH=/path/to/venv/bin:$PATH"
EnvironmentFile=/path/to/Cleo/.env
Restart=always
User=your-user
Group=your-group
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=multi-user.target

• ExecStart: Points to the Python command to run Cleo.

• EnvironmentFile: Points to the .env file for Cleo configuration.

• Restart=always: Ensures that Cleo restarts if it crashes or if the server reboots.

c. Enable and Start the Service

Once the service file is created, reload systemd and start the service:

bashCopyEditsudo systemctl daemon-reload
sudo systemctl enable cleo-agent.service
sudo systemctl start cleo-agent.service

To view the logs of the service:

bashCopyEditsudo journalctl -u cleo-agent.service

d. Managing the Service

To stop, restart, or check the status of Cleo:

bashCopyEditsudo systemctl stop cleo-agent.service
sudo systemctl restart cleo-agent.service
sudo systemctl status cleo-agent.service

---

Running Cleo on a Schedule

Sometimes you may want to run specific tasks on a recurring schedule. CRON jobs are an effective way to set up automated, time-based task executions. This can be useful for batch processing or periodic interactions with your agents.

1. Setting Up a CRON Job

a. Edit the CRON Table

Open the cron configuration file for your user:

bashCopyEditcrontab -e

b. Schedule Cleo to Run Periodically

For example, to run Cleo once every day at midnight:

bashCopyEdit0 0 * * * /usr/bin/python /path/to/Cleo/main.py --task "Run daily report"

This CRON entry tells the system to run Cleo at midnight every day. You can adjust this based on your needs.

---

Long-Running Tasks with Supervisor

Supervisor is another tool for managing background processes. Unlike systemd, Supervisor is more lightweight and can be used on systems that don’t use systemd.

a. Install Supervisor

On Ubuntu:

bashCopyEditsudo apt-get install supervisor

b. Configure Supervisor for Cleo

Create a configuration file for Cleo under /etc/supervisor/conf.d/cleo-agent.conf:

iniCopyEdit[program:cleo-agent]
command=/usr/bin/python /path/to/Cleo/main.py
autostart=true
autorestart=true
stderr_logfile=/var/log/cleo-agent.err.log
stdout_logfile=/var/log/cleo-agent.out.log

c. Control the Service

Start, stop, or restart Cleo with:

bashCopyEditsudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start cleo-agent

---

Task Scheduling via Code

You may also want to schedule tasks from within the Python code itself. Cleo can integrate with APScheduler, a flexible scheduling library, to define complex job schedules.

Example: Scheduling a Task with APScheduler

In main.py, you can define a recurring task with APScheduler:

pythonCopyEditfrom apscheduler.schedulers.blocking import BlockingScheduler
from cleo.agents import load_agent

def scheduled_task():
    agent = load_agent("lex")
    agent.run_task("Analyze market trends for today.")

scheduler = BlockingScheduler()
scheduler.add_job(scheduled_task, 'interval', hours=1)  # Runs every 1 hour
scheduler.start()

This example uses APScheduler to run a task every hour. You can modify the interval, day of week, or specific times based on your needs.

---

High Availability and Failover

For critical systems, consider setting up failover mechanisms to ensure Cleo continues running even in the event of server failures:

1. Load Balancer: Use a load balancer (such as Nginx or HAProxy) to distribute traffic to multiple instances of Cleo, ensuring no downtime during server failures.

2. Backup Systems: Set up automatic backups of memory stores (like FAISS) to prevent data loss in case of system failures.

3. Monitoring and Alerts: Implement monitoring tools such as Prometheus or Datadog to keep track of Cleo's health and task statuses. Set up alerts for failures, timeouts, or critical errors.

---

Summary

Running Cleo in a scheduled or background execution environment ensures that agents remain operational and can process tasks autonomously. Using tools like systemd, CRON, or Supervisor, you can scale Cleo into long-running or periodic processes without manual intervention. Furthermore, the use of APScheduler allows for fine-grained control over task scheduling directly from the application.