← Back to Plugin Architecture

5.3 Queue Processing

KMP uses dereuromark/cakephp-queue for tenant-domain background jobs and a separate platform_jobs mechanism for allowlisted platform administration operations. They share a worker authority but not a data model or trust boundary.

Queue lanes

Lane	Storage/context	Processor
Default Queue plugin lane	Default application datasource	QueueDrainService::drainDefault()
Tenant Queue plugin lane	One active tenant database at a time	QueueDrainService::drainTenant() under TenantConnectionManager
Platform jobs	Platform database, allowlisted job types	PlatformJobRunner

PlatformQueueDrainService fairly drains the default and active-tenant Queue plugin datasources within bounded job/runtime budgets. PlatformWorkerService runs one bounded platform schedule and queue cycle. The production entry point is bin/cake platform worker run; platform queues run is the queue-focused command.

Enqueuing tenant work

Queue plugin jobs written while a tenant context is active land in that tenant’s queued_jobs table. Job payloads should contain stable identifiers and small immutable inputs, not serialized ORM entities, credentials, or file contents. A worker reloads the record after the correct tenant is bound.

Enqueue after the owning transaction is durable, or use an existing workflow/action pattern that guarantees the worker cannot observe rolled-back state. Make handlers idempotent because retries and partial failures are normal.

Platform operations

Tenant lifecycle changes, backups, and other privileged fleet operations are enqueued through PlatformAdminJobEnqueuer, which applies allowlisting, idempotency, conflict checks, and platform audit. Do not send these operations through an arbitrary Queue task or let a tenant payload name a shell command.

Failure and observability

Workers use bounded cycles so one tenant cannot monopolize the fleet. A failure for one tenant is reported without preventing other tenant lanes from being attempted. Error messages exposed to platform users are scrubbed; detailed exceptions remain in server logs.

Monitor platform job events/alerts and Queue plugin job state. A successful worker process only means the cycle ran; domain tests must still prove each task’s side effects and retry behavior.

Local use and verification

Queue configuration is loaded from app/config/app_queue.php and app/config/app.php. For focused local debugging, the Queue plugin’s bin/cake queue run command can drain the currently configured datasource. Multi-tenant behavior should be exercised through the platform worker services/commands.

Test idempotency, transaction ordering, retryable and terminal failures, inactive tenants, tenant context cleanup, fair traversal, runtime/job bounds, sanitized output, and the separation between Queue jobs and platform_jobs.