Draft: Topic-based routing

Refactors Worker architecture from queue-name-based to input/output type-based system using a single AMQP topic exchange. Full backwards compatibility maintained via legacy mode and dual-binding.

Key Changes

1. Worker Initialization

New (Recommended):

Worker(job_handler, input_type="pdf", output_type="text")

Legacy (Deprecated):

Worker(job_handler, queue_name="pdf-text")  # Still works, triggers deprecation warning

2. Topic Exchange Architecture

• Before: Multiple fanout exchanges ({prefix}-workers, {prefix}-jobs, {prefix}-job-results)
• After: Single topic exchange (nldoc-topics) with structured routing keys

Topic Patterns:

• Jobs: worker.pdf-text.jobs
• Results: worker.pdf-text.results.{jobid}
• Acks: worker.pdf-text.acks
• Health: health.request.pdf-text, health.heartbeat.pdf-text.{instance}

3. Configuration

New environment variable:

• NLDOC_TOPICS_EXCHANGE_NAME (optional, defaults to "nldoc-topics")

Deprecated:

• EXCHANGE (still required in legacy mode, will be removed in v6.0.0)

4. Reliability Enhancements

• Dual-Binding: Legacy workers automatically bind to both old and new exchanges for seamless migration
• Dead-Letter Exchange: All queues configured with DLX (nldoc-dlx)
• Priority Queues: All queues support priorities 0-10

Migration Strategy

Workers with derivable queue names (format: {input}-{output}) automatically enable dual-binding:

1. Bind to legacy fanout exchanges (backward compatibility)
2. Bind to new topic exchange (forward compatibility)

Migration Steps:

1. Deploy v5.0.0 (workers continue in legacy mode with dual-binding)
2. Update station to publish to new topic exchange
3. Update workers to use input_type/output_type
4. Remove EXCHANGE environment variable

Deprecations

• queue_name parameter → use input_type and output_type
• EXCHANGE environment variable → no longer required (except in legacy mode)

Both will be removed in v6.0.0.

feat!: added topic-based routing with backwards compatibility

BREAKING CHANGE: From now on, inputtype and outputtype are required parameters for Worker init. Backwards compatibility is attempted when the input/output type is derivable from the deprecated parameter exchange_name, but when this fails, the package fails to receive messages.