Pub/Sub Processor
Event-driven processor that consumes messages from Google Cloud Pub/Sub topics and sends them to external ERP systems.
Purpose
-
Polls comerzzia database for pending documents
-
Transforms entities to ERP format (JSON/XML)
-
Publishes transformed messages to Google Cloud Pub/Sub topics
-
Consumes messages from Pub/Sub topics (optional, configurable)
-
Sends data to ERP systems via HTTP
-
Supports concurrent processing with configurable parallel pull
-
Comprehensive retry logic with DLQ support
Prerequisites
This application requires an existing Google Cloud Pub/Sub setup. It does not include topic or subscription creation.
Ensure: - Google Cloud project is configured - Required topics and subscriptions are created (main and DLQ) - Service account credentials are available
Configuration
Application Settings
See application.yml for full configuration.
Key settings:
- Project ID: Google Cloud project identifier
- Consumer activation: Optional - Enable/disable via czz-to-erp.consumer.enabled in application.yml
- Topic configuration: Main topic and DLQ topic
- Subscription configuration: Main subscription with credentials
- Processing settings: Timeouts, retries, flow control
- Retry configuration: Max attempts (default: 3)
- Server configuration: Application and actuator ports, context paths
- OpenAPI/Swagger UI: Interactive API documentation for testing endpoints
- Actuator endpoints: Health checks, Prometheus metrics, and monitoring dashboard
Important Notes
-
Entity types must match those defined in
entities-config.ymlfrom Processor Common Library Module -
Flow control settings prevent message overload
Credentials Configuration
Requires service account JSON keys in credentials/ directory for Pub/Sub authentication.
Error Handling and Retry Logic
Retry Mechanism
-
First failure: Error registered in database, message republished to main topic with
retry-count=1 -
Subsequent failures: Message republished with incremented
retry-count -
Max retries reached: Message sent to DLQ topic and ACKed
