Pub/Sub Processor

Cloud-native processor that consumes messages from Google Cloud Pub/Sub subscriptions and processes them to comerzzia database.

Purpose

  • Consumes messages from Google Cloud Pub/Sub subscriptions

  • Validates and processes entity data

  • Persists to comerzzia database via integration services

  • Supports concurrent processing with configurable parallel pull

Prerequisites

This application requires an existing Google Cloud Platform project with Pub/Sub. It does not include GCP project setup or topic creation.

Ensure: - GCP project with Pub/Sub API enabled - Service accounts with roles/pubsub.subscriber and roles/pubsub.publisher roles - Required topics and subscriptions are created - Service account JSON keys available

Processing Flow

Diagram

Pub/Sub Subscription Structure

Messages consumed from configured subscription. Failed messages published to error topic.

Configuration

Application Settings

Key settings:

Spring Property Environment var Default value

erp-to-czz.project-id

ERP_TO_CZZ_PROJECT_ID

erp-to-czz

erp-to-czz.main-topic.topic-id

ERP_TO_CZZ_MAIN_TOPIC_TOPIC_ID

erp-to-czz-input

erp-to-czz.main-topic.subscription-id

ERP_TO_CZZ_MAIN_TOPIC_SUBSCRIPTION_ID

erp-to-czz-input-consumer

erp-to-czz.main-topic.credentials-file

ERP_TO_CZZ_MAIN_TOPIC_CREDENTIALS_FILE

classpath:credentials/ep-to-czz-subcription-key.json

erp-to-czz.error-topic.topic-id

ERP_TO_CZZ_ERROR_TOPIC_TOPIC_ID

erp-to-czz-input-errors

erp-to-czz.error-topic.subscription-id

ERP_TO_CZZ_ERROR_TOPIC_SUBSCRIPTION_ID

erp-to-czz-input-error-consumer

erp-to-czz.error-topic.credentials-file

ERP_TO_CZZ_ERROR_TOPIC_CREDENTIALS_FILE

classpath:credentials/erp-to-czz-publish-key.json

Important Notes

  • Separate credentials allow fine-grained IAM permissions (subscribe vs publish)

  • Parallel pull count controls concurrent message processing

Database Configuration

Requires comerzzia database configuration via comerzzia.xml in java classpath or in COMERZZIA_HOME path.

Credentials Configuration

Requires service account JSON keys in credentials/ classpath directory for Pub/Sub authentication.

Docker container execution

docker run -it --rm --name processor-pubsub $PWD:/apps/bin/lib --env-file processor-dev.env --pull=always -p 8080:8080 europe-west3-docker.pkg.dev/czz-devops/comerzzia/erp-to-czz/processor-pubsub-application:1.0
processor-dev.env file example for environment variables
ERP_TO_CZZ_PROJECT_ID=erp-to-czz-test

ERP_TO_CZZ_MAIN_TOPIC_TOPIC_ID=erp-to-czz-input
ERP_TO_CZZ_MAIN_TOPIC_SUBSCRIPTION_ID=erp-to-czz-input-consumer-test
ERP_TO_CZZ_MAIN_TOPIC_CREDENTIALS_FILE=classpath:credentials/erp-to-czz-input-consumer-test-key.son

ERP_TO_CZZ_ERROR_TOPIC_TOPIC_ID=erp-to-czz-input-errors
ERP_TO_CZZ_ERROR_TOPIC_SUBSCRIPTION_ID=erp-to-czz-input-publish-test
ERP_TO_CZZ_ERROR_TOPIC_CREDENTIALS_FILE=classpath:credentials/erp-to-czz-input-publish-test-key.son

uidActividad=MYACTIVITY
uidInstancia=MYINSTANCE

SPRING_PROFILES_ACTIVE=jsonlogs

COMERZZIA_DB_CLASS=com.mysql.jdbc.Driver
COMERZZIA_DB_DRIVER=mysql
COMERZZIA_DB_USER=comerzzia
COMERZZIA_DB_PASS=comerzzia
COMERZZIA_DB_URL=jdbc:mysql://127.0.0.1:3306/comerzzia