CVE Processor is a Go application designed to download, process, and publish Common Vulnerabilities and Exposures (CVE) records to a Kafka topic. This application utilizes the CVE Record Format to normalize and enrich CVE information.

1. Downloads the CVE list and processes all JSON files in all directories.
2. CVE Services uses the CVE Record Format (view schema ReadMe), which normalizes and enriches how CVE information is presented, and adds optional data fields to CVE Records, such as: severity scores, credit for researchers, additional languages, affected product lists, additional references, ability for community contributions, etc.
3. Publishes processed CVEs to a Kafka topic named cve.
4. Exits once all CVE records have been published to the Kafka topic.

• Go 1.22.3 or later
• Docker
• Access to a Kafka cluster with 3 brokers

1. Build the Docker image:

   docker build -t cve-processor .
   

2. Run the container:

   docker run --network host cve-processor
   

1. Install dependencies:

   go mod download
   

2. Build the application:

   go build -o cve-processor ./cmd/main.go
   

3. Run the application:

   ./cve-processor
   

The application uses environment variables for configuration. Create a .env file in the project root with the following variables:

PORT=8080
# Kafka
KAFKA_USER=
KAFKA_PASSWORD=
KAFKA_BROKER_0=
KAFKA_BROKER_1=
KAFKA_BROKER_2=
URL=

Make sure to replace the placeholder values with your actual configuration.

The project includes a multi-stage Dockerfile for building and running the application in a minimal container. The final image is based on scratch for a smaller footprint.

• github.com/segmentio/kafka-go: Used for Kafka integration

Contributions are welcome! Please feel free to submit a Pull Request.

MIT