Kafka Implementation - Phase 0 Complete ✅

Infrastructure Setup Summary

Date: October 6, 2025 Phase: 0 - Infrastructure Setup Status: ✅ COMPLETE - Ready for Testing

✅ What We’ve Implemented

1. Docker Compose Configuration

File: infrastructure/docker/docker-compose-infrastructure.yml

Added two new services:

Kafka (Port 9092, 9094)

Image: bitnami/kafka:3.6
Mode: KRaft (No Zookeeper needed - modern approach)
Configuration:
- Auto-create topics enabled
- 3 partitions per topic (default)
- 7-day message retention
- Snappy compression
- 1GB heap memory
- Health check every 10 seconds

Kafka UI (Port 8090)

Image: provectuslabs/kafka-ui:latest
Purpose: Visual monitoring and management
Features:
- View topics and messages
- Monitor consumer lag
- Manage consumer groups
- Inspect event payloads

2. Infrastructure Scripts Updated

File: scripts/check-infrastructure.sh

Added status checks for:

Kafka broker health
Kafka UI availability
Updated service count from 4 → 6

3. Documentation Updated

File: infrastructure/docker/README.md

Added:

Kafka and Kafka UI to service list
Access points with ports
WSL2 access instructions

🚀 How to Start Kafka

Start All Infrastructure

cd /home/subramani/modern-reservation
bash infra.sh start

Check Status

bash infra.sh status

Expected Output:

┌─────────────────────┬────────────┬─────────────┬────────────────────────────────┐
│ Service             │ Status     │ Port        │ Details                        │
├─────────────────────┼────────────┼─────────────┼────────────────────────────────┤
│ kafka               │ DOCKER     │ 9092        │ Broker ready                   │
│ kafka-ui            │ DOCKER     │ 8090        │ Monitoring ready               │
└─────────────────────┴────────────┴─────────────┴────────────────────────────────┘

🔍 Access Points

Kafka UI (Monitoring Dashboard)

URL: http://localhost:8090
WSL2: http://172.27.108.197:8090
Features:
- Browse topics
- View messages
- Monitor consumer groups
- Check broker health

Kafka Broker (Programmatic Access)

Internal: kafka:9092 (within Docker network)
External: localhost:9092 or localhost:9094

✅ Verification Tests

Test 1: Check Containers Running

docker ps | grep kafka

# Should show 2 containers:
# modern-reservation-kafka
# modern-reservation-kafka-ui

Test 2: Check Kafka Broker Health

docker exec modern-reservation-kafka kafka-broker-api-versions.sh \
  --bootstrap-server localhost:9092

# Should show Kafka broker API versions

Test 3: List Topics

docker exec modern-reservation-kafka kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --list

# Initially empty (no topics yet)

Test 4: Create Test Topic

docker exec modern-reservation-kafka kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --create --topic test-topic \
  --partitions 3 \
  --replication-factor 1

# Should show: Created topic test-topic

Test 5: Describe Topic

docker exec modern-reservation-kafka kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --describe --topic test-topic

# Should show topic configuration with 3 partitions

Test 6: Send Test Message

echo "Hello Kafka!" | docker exec -i modern-reservation-kafka \
  kafka-console-producer.sh \
  --bootstrap-server localhost:9092 \
  --topic test-topic

# Message sent successfully

Test 7: Consume Test Message

docker exec modern-reservation-kafka kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic test-topic \
  --from-beginning \
  --max-messages 1

# Should display: Hello Kafka!

Test 8: View in Kafka UI

Open http://localhost:8090
Click on “modern-reservation” cluster
Go to “Topics”
Find “test-topic”
Click “Messages”
See “Hello Kafka!” message

Test 9: Delete Test Topic

docker exec modern-reservation-kafka kafka-topics.sh \
  --bootstrap-server localhost:9092 \
  --delete --topic test-topic

# Clean up test topic

📊 Infrastructure Summary

Services Now Running

Service	Container	Port	Status
Zipkin	modern-reservation-zipkin	9411	✅ Running
PostgreSQL	modern-reservation-postgres	5432	✅ Running
pgAdmin	modern-reservation-pgadmin	5050	✅ Running
Redis	modern-reservation-redis	6379	✅ Running
Kafka	modern-reservation-kafka	9092, 9094	✅ NEW
Kafka UI	modern-reservation-kafka-ui	8090	✅ NEW

Resource Usage

Kafka: ~300MB RAM (1GB heap allocated)
Kafka UI: ~150MB RAM
Total Added: ~450MB RAM

🎯 What’s Next: Phase 1

Objective: Create Shared Event Library

Timeline: Day 2 (3-6 hours)

Tasks:

Create event package structure in libs/shared/backend-utils
Create BaseEvent.java abstract class
Create EventPublisher.java utility
Create ReservationCreatedEvent.java
Build and install shared library

Location:

libs/shared/backend-utils/src/main/java/com/modernreservation/shared/
└── events/
    ├── BaseEvent.java
    ├── EventPublisher.java
    └── reservation/
        └── ReservationCreatedEvent.java

Next Command:

cd /home/subramani/modern-reservation/libs/shared/backend-utils
# Create event classes (see IMPLEMENTATION_PLAN.md for code)
mvn clean install

🔧 Troubleshooting

Kafka Container Won’t Start

# Check logs
docker logs modern-reservation-kafka

# Common fix: Remove volumes and restart
docker-compose -f infrastructure/docker/docker-compose-infrastructure.yml down -v
docker-compose -f infrastructure/docker/docker-compose-infrastructure.yml up -d

Kafka UI Not Accessible

# Check container status
docker ps | grep kafka-ui

# Check logs
docker logs modern-reservation-kafka-ui

# Verify Kafka is healthy first
docker exec modern-reservation-kafka kafka-broker-api-versions.sh \
  --bootstrap-server localhost:9092

Health Check Failing

# Wait 30 seconds for start_period
sleep 30

# Then check health
docker inspect modern-reservation-kafka | grep -A 10 Health

📝 Files Modified

Created/Modified

✅ infrastructure/docker/docker-compose-infrastructure.yml - Added Kafka + Kafka UI
✅ scripts/check-infrastructure.sh - Added Kafka health checks
✅ infrastructure/docker/README.md - Updated documentation

Ready for Next Phase

libs/shared/backend-utils/ - Create event library here
apps/backend/java-services/business-services/reservation-engine/ - Integrate events here

✅ Success Criteria - Phase 0

Kafka container added to docker-compose
Kafka UI container added to docker-compose
Health checks configured
Infrastructure scripts updated
Documentation updated
TODO: Start infrastructure and verify ← Next Step!

🎉 Ready to Proceed

Phase 0 is complete in code!

Next Action: Start the infrastructure and verify Kafka is running:

cd /home/subramani/modern-reservation
bash infra.sh start
bash infra.sh status

Then open http://localhost:8090 to see Kafka UI!

Phase: 0/4 Complete ✅ Next: Phase 1 - Shared Event Library Estimated Time to Complete Phase 1: 3-6 hours