Kafka Input

REST API service that receives ERP data and publishes messages to Apache Kafka topics.

Purpose

  • Receives ERP data via REST endpoints

  • Validates incoming data

  • Publishes messages to Kafka topics with metadata headers

  • Provides async publishing with producer acknowledgment

Prerequisites

This application requires an existing Apache Kafka cluster. It does not include Kafka server setup or topic creation.

Ensure:

  • Kafka cluster is accessible

  • Required topics are created

Message Publishing Flow

flowchart TD
    A[ERP System]
    B[REST API POST]
    C{JSON Valid?}
    D[Add Headers]
    E[Kafka Producer<br/>Async + Retry]
    F[Kafka Topic]
    G[Error Response]

    A --> B
    B --> C
    C -->|Yes| D
    C -->|No| G
    D --> E
    E --> F
    E -->|Failed| G

    style A fill:none,stroke:#666,stroke-width:3px
    style B fill:none,stroke:#666,stroke-width:3px
    style C fill:none,stroke:#666,stroke-width:3px
    style D fill:none,stroke:#666,stroke-width:3px
    style E fill:none,stroke:#666,stroke-width:3px
    style F fill:none,stroke:#666,stroke-width:3px
    style G fill:none,stroke:#666,stroke-width:3px

Kafka Topic Structure

Messages productor to configured main topic with entity metadata headers.

Configuration

Application Settings

Key settings:

Spring Property Environment var Default value

erp-to-czz.main-topic

ERP_TO_CZZ_MAIN_TOPIC

erp-to-czz-input

spring.kafka.bootstrap-servers

SPRING_KAFKA_BOOSTRAP_SERVERS

kafka-server-svc:9092

Docker container execution

docker run -it --rm --name input-kafka --env-file input-dev.env --pull=always -p 8080:8080 europe-west3-docker.pkg.dev/czz-devops/comerzzia/erp-to-czz/input-kafka-application:1.0
input-dev.env file example for environment variables
SPRING_KAFKA_BOOSTRAP_SERVERS=127.0.0.1:9092
ERP_TO_CZZ_MAIN_TOPIC=erp-to-czz-input

SPRING_PROFILES_ACTIVE=jsonlogs

SPRING_KAFKA_BOOSTRAP_SERVERS=127.0.0.1:9092

ERP_TO_CZZ_MAIN_TOPIC=erp-to-czz-input