Post

Multimodal PDF Parser & Abstract Generator

Posted Updated
By
7 min read
Multimodal PDF Parser & Abstract Generator

Project

Table of contents

  1. Overview
  2. Environment Setup & Run/Stop
  3. System Flow
  4. How the Code Is Organized
  5. What Happens Step-by-Step
  6. Why Using Redis Technique
  7. How the Frontend (index.html) Links to the Backend
  8. How Redis and the Worker Work Together
  9. How pdf parser service works
  10. Error Handling & Logging
  11. Quick Tips
  12. Resources
  13. Project repository

Overview

This project parses PDFs and generates abstracts (summaries) using multiple strategies: direct text extraction and image-based OCR plus AI summarization. PDFs are not always simple—some contain selectable text, while others are scanned images—so the goal is to keep the UI fast while heavy parsing happens in the background. It handles different input modalities incluidng text-based and image-based pdfs.

  • Text-based PDFs (fast extraction with PyPDF2)
  • AI extraction + summarization (Gemini)
  • OCR for scanned PDFs (Mistral Pixtral)

Environment Setup & Run/Stop

You need the following before running:

  • Python 3.8+ (for local run)
  • Docker Desktop (for Redis and containerized run)
  • Redis (job queue)
  • Gemini API key (GEMINI_API_KEY) for Gemini parser
  • Mistral API key (MISTRAL_API_KEY) for OCR parser
  • Poppler (required for pdf2image when using Mistral OCR)

Create a .env file in the project root and set keys:

1
2

GEMINI_API_KEY=your_key_here
MISTRAL_API_KEY=your_key_here

Once the environment is ready, use these commands to start or stop the app locally:

1

docker compose up --build

1

docker compose down

Open the app at http://127.0.0.1:8000 after starting.

Use Environment Setup for first-time setup, new machines, or after changing dependencies/keys. Use the Run/Stop commands for daily development and testing once setup is complete.

System Flow

1
2
3
4
5

┌───────────┐   ┌───────────┐   ┌───────┐   ┌────────┐   ┌──────────┐
│ Browser   │ → │ FastAPI   │ → │ Redis │ → │ Worker │ → │ Parsers  │
└───────────┘   └───────────┘   └───────┘   └────────┘   └──────────┘
	▲                                          │
	└──────────────────── Results ─────────────┘

How the Code Is Organized

  • backend/main.py: API endpoints for upload, status, and results.
  • backend/workers/pdf_parser_worker.py: background worker that reads jobs from Redis and processes them.
  • backend/services/pdf_parser_service.py: parser implementations (PyPDF2, Gemini, Mistral).
  • backend/services/redis_service.py: Redis Streams and result storage helpers.
  • frontend/index.html: upload UI.

What Happens Step-by-Step

  1. Upload request arrives at FastAPI.
  2. The API saves the PDF and creates a job in Redis Stream.
  3. The worker reads the job from Redis.
  4. The worker calls the chosen parser method:
    • pypdf → quick text extraction
    • gemini → AI extraction + summary
    • mistral → OCR for scanned PDFs
  5. The worker writes the results (pages + summary) to Redis.
  6. The frontend polls the API until results are available.

Why Using Redis Technique

Redis Streams let you implement a reliable job queue:

  • FastAPI enqueues work quickly.
  • Worker processes tasks asynchronously.
  • UI stays responsive.

Pseudo flow:

1
2
3

API: enqueue job -> Redis Stream
Worker: read stream -> parse -> write results
Frontend: poll results -> render

The front page uses JavaScript fetch() calls to talk to the API:

  1. Upload request
    • The form handler builds a FormData object and sends it to:
      • POST /upload?document_id=...&parser_method=...
    • This is handled by the FastAPI endpoint in main.py:
      • @app.post("/upload")
  2. Results polling
    • After upload, the page starts polling:
      • GET /results/{document_id}
    • This maps to:
      • @app.get("/results/{document_id}")
  3. Static frontend serving
    • The backend serves the frontend file via:
      • @app.get("/") returning index.html.
    • Static files are mounted at /static so you can add assets if needed.

Because the API and frontend share the same host, the front page can call the backend using relative URLs like /upload and /results/... without extra CORS configuration.

Key Code (Frontend Upload)

1
2
3
4

const response = await fetch(`/upload?document_id=${documentId}&parser_method=${parser}`, {
	method: 'POST',
	body: formData
});

Key Code (Backend Endpoint)

1
2
3
4
5
6
7

@app.post("/upload")
async def upload_document(
		file: UploadFile = File(...),
		document_id: str = Query(None),
		parser_method: str = Query(APP_SETTINGS["default_parser"])
):
		...

How Redis and the Worker Work Together

1) Redis Connection (shared by API + worker)

Both main.py and the worker import redis_client from redis_service.py. The client is created once and reused.

1
2
3
4
5
6

redis_client = redis.Redis(
	host=os.getenv("REDIS_HOST", "localhost"),
	port=int(os.getenv("REDIS_PORT", 6379)),
	db=0,
	decode_responses=True
)

2) API Adds Jobs to a Redis Stream

When a PDF is uploaded, the API pushes a job into the stream pdf_parsing_jobs:

1
2
3
4
5
6
7
8

stream_name = "pdf_parsing_jobs"
job_data = {
	"document_id": document_id,
	"file_path": file_location,
	"parser_method": parser_method
}

redis_client.xadd(stream_name, job_data)

3) Worker Reads Jobs From the Stream

The worker uses a consumer group to read new messages (> means only new jobs):

1
2
3
4
5
6
7

return redis_client.xreadgroup(
	consumer_group,
	consumer_name,
	{stream_name: '>'},
	count=1,
	block=2000
)

After processing, it acknowledges the message so Redis knows it is done:

1

redis_client.xack(stream_name, consumer_group, message_id)

4) Worker Processes Jobs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

def process_job(job_data: dict):
	document_id = job_data.get("document_id")
	file_path = job_data.get("file_path")
	parser_method = job_data.get("parser_method", APP_SETTINGS["default_parser"])

	if parser_method == "pypdf":
		parsed_pages = parse_pdf_pypdf(file_path)
	elif parser_method == "gemini":
		parsed_pages = parse_pdf_gemini(file_path)
	elif parser_method == "mistral":
		parsed_pages = parse_pdf_mistral(file_path)
	else:
		parsed_pages = parse_pdf_pypdf(file_path)

	summary = generate_summary(parsed_pages)
	store_result(document_id, parser_method, parsed_pages, summary)

Quick Tips:

1: How main.py Connects to the Worker

main.py does not call the worker directly. Instead, it writes jobs to Redis and the worker reads jobs from Redis. Redis is the connection layer between them.

2: When Does the Worker Run?

The worker runs as a separate process (usually its own container or terminal). It should be started independently from the API server. It can be run by one of following methods:

  1. The function start_worker() is the entry point that starts the infinite job‑polling loop. It is not required to be called inside main.py.

    Typical worker entry:

    1
2

     if __name__ == "__main__":
      start_worker()

    This keeps the API process (FastAPI) and the worker process decoupled. The API handles HTTP requests; the worker handles background parsing.

  2. Run the worker in a separate terminal (this executes start_worker() via the __main__ entry point):

    1

     python -m backend.workers.pdf_parser_worker

How pdf parser service works

The file pdf_parser_service.py is where each parsing method is implemented. Below are the main libraries, API keys, and important code blocks you should learn.

1) PyPDF2 (Text PDFs)

Library: PyPDF2

Key code:

1
2
3
4

from PyPDF2 import PdfReader

pdf_reader = PdfReader(file_path)
page_text = page.extract_text()

2) Gemini (LLM Extraction + Summary)

Library: google-generativeai API Key: GEMINI_API_KEY in .env

Key setup:

1
2

GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
genai.configure(api_key=GEMINI_API_KEY)

Key code:

1
2
3

file_to_upload = genai.upload_file(file_path)
model = genai.GenerativeModel('gemini-2.5-flash-lite')
response = model.generate_content([file_to_upload, prompt])

3) Mistral Pixtral OCR (Scanned PDFs)

Libraries: mistralai, pdf2image, Pillow API Key: MISTRAL_API_KEY in .env System dependency: Poppler (for pdf2image)

Key setup:

1
2

MISTRAL_API_KEY = os.getenv("MISTRAL_API_KEY")
mistral_client = Mistral(api_key=MISTRAL_API_KEY)

Key code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

images = convert_from_path(file_path)

response = mistral_client.chat.complete(
	model="pixtral-12b-2409",
	messages=[
		{
			"role": "user",
			"content": [
				{"type": "text", "text": "Extract all text..."},
				{"type": "image_url", "image_url": f"data:image/png;base64,{img_base64}"}
			]
		}
	]
)

Error Handling & Logging

PDF parsing and external APIs can fail for many reasons (corrupt files, timeouts, invalid keys, rate limits). This project uses defensive checks and explicit error messages to keep the system stable.

The following errors already handled in this project:

  • Redis connection errors (API or worker not connected to Redis)
  • Queueing failures when adding jobs to the stream (xadd)
  • Missing API keys for Gemini or Mistral
  • File validation errors (missing file, non-PDF, empty file, too large)
  • PDF conversion failures (Poppler missing, corrupted PDF)
  • API timeouts or quota limits (Gemini / Mistral)
  • Result storage errors (failed Redis writes)

Sample Error-Handling Code Block

1
2
3
4
5
6

try:
  redis_client.xadd(stream_name, job_data)
except Exception as redis_error:
  if os.path.exists(file_location):
    os.remove(file_location)
  raise HTTPException(status_code=500, detail=f"Failed to queue job: {redis_error}")

1
2
3
4
5
6
7

# Validate file path and size
if not file_path or not os.path.exists(file_path):
  return ["[Error: Invalid or missing file path]"]

file_size = os.path.getsize(file_path)
if file_size > 50 * 1024 * 1024:
  return ["[Error: File too large]"]

Quick Tips

  • Start with main.py to understand the request/response flow.
  • Then trace the job queue and worker logic.
  • Compare parser outputs to see the difference in quality and speed.
  • Redis Streams are key for scalability and responsiveness.

Resources

Project repository

GitHub Code: PDF Parser Application

Agentic/GenAI and LLMs/LMMs with RAG/PromptEng, LLM/LMMs
LLMs multimodal PDF OCR Document AI Gemini Mistral PyPDF2 FastAPI Redis Docker API summarization abstract generation job queue Python
This post is licensed under CC BY 4.0 by the author.