devLap

Building an Order Processing Pipeline with Spring Integration (HTTP + File Polling)

java dev.to

If you’ve used Spring Boot REST APIs but haven’t explored Spring Integration yet, this project is a practical way to see what message-driven flow design looks like in real code.

I built a sample app that processes orders from two different entry points:

  1. HTTP API (POST /api/orders)
  2. File polling (drop CSV files into input/)

Both inputs share the same core processing logic.

👉 Source code: https://github.com/ykpraveen/spring-integration-sample

What this project does

Each order goes through this pipeline:

  1. Transform input payload into Order
  2. Validate (id, customer, total)
  3. Route by amount:
    • total <= 100 → EXPRESS
    • total > 100 → REVIEW
  4. Persist order (JPA)
  5. Fan out to:
    • archive output file
    • summary aggregation

Runtime persistence uses PostgreSQL, and tests use H2.

Tech stack

  • Java 21
  • Spring Boot 4.1
  • Spring Integration 7 (Java DSL)
  • Spring Data JPA
  • PostgreSQL (runtime)
  • H2 (test)
  • Maven

Architecture at a glance

1) HTTP flow

POST /api/orders sends raw JSON to a messaging gateway, then:

  • JsonToOrderTransformer
  • OrderValidationService
  • content-based router (EXPRESS / REVIEW)
  • OrderStore.put(...)
  • publish-subscribe to archive + summary + HTTP response message

2) File flow

The poller watches input/*.csv, then:

  • FileToStringTransformer
  • CsvToOrderTransformer
  • validation + routing + persistence
  • publish-subscribe to archive + summary

3) Error handling

  • HTTP errors return JSON via dedicated HTTP error handling.
  • File processing errors generate failure logs in output/failed/ and move bad source files to input/failed/.

Why Spring Integration here?

Using Spring Integration made these parts clean and explicit:

  • Routing rules are declarative.
  • Fan-out behavior is easy with publish-subscribe channels.
  • Retry advice can be attached per handler in the file pipeline.
  • The integration graph (/api/integration/graph) helps visualize the runtime flow.

Example requests

Submit an EXPRESS order 

curl -X POST http://localhost:8080/api/orders \
  -H "Content-Type: application/json" \
  -d '{"id":"ORD-001","customer":"Alice","description":"Book","total":25.00}'
Enter fullscreen mode Exit fullscreen mode

Submit a REVIEW order 

curl -X POST http://localhost:8080/api/orders \
  -H "Content-Type: application/json" \
  -d '{"id":"ORD-002","customer":"Bob","description":"Laptop","total":1500.00}'
Enter fullscreen mode Exit fullscreen mode

Get one order 

curl http://localhost:8080/api/orders/ORD-001
Enter fullscreen mode Exit fullscreen mode

List all orders 

curl http://localhost:8080/api/orders
Enter fullscreen mode Exit fullscreen mode

Running locally

Clone the repo first:

git clone https://github.com/ykpraveen/spring-integration-sample.git
cd spring-integration-sample
Enter fullscreen mode Exit fullscreen mode

Then run:

docker compose up -d
mvn clean package
mvn spring-boot:run
Enter fullscreen mode Exit fullscreen mode

Then test HTTP endpoints or drop a CSV file into input/.

Testing

The project currently has 38 tests (unit + integration), covering:

  • transformation and validation
  • HTTP happy paths and failure paths
  • duplicate order handling (409 Conflict)
  • file poller processing and retries
  • summary aggregation behavior via Spring Integration aggregator + writer service
  • integration graph endpoint

You can run these directly from the repo: https://github.com/ykpraveen/spring-integration-sample

Key implementation details I found useful

  1. Use correlation IDs in headers and push them into MDC for traceable logs across flow steps.
  2. Keep config split by concern (HttpIntegrationConfig, FileIntegrationConfig, shared config) instead of one huge integration config class.
  3. Treat summary aggregation as stateful logic: clear release rules (batch size vs timeout), stable keys, and append-safe file writing.
  4. Prefer explicit web path-variable lookup for GET /api/orders/{id} over indirect URL parsing.

Repo

You can clone the project and run it as a demo starter for:

  • Spring Integration basics
  • event/message-driven service design in Spring
  • hybrid ingestion patterns (HTTP + file)

If you’re learning Spring Integration, this pattern is a good stepping stone before Kafka/Rabbit-based distributed flows.

GitHub: https://github.com/ykpraveen/spring-integration-sample

Source: dev.to

arrow_back Back to Tutorials