Every day, millions of sensitive documents — invoices, legal contracts, meeting transcripts, academic textbooks — get uploaded to cloud AI services for processing. Most people don't think twice about it. But here's the uncomfortable truth: once your data leaves your machine, you've lost control of it.
What if you could build a complete document processing pipeline that never phones home? One that runs entirely on your hardware, processes confidential documents without any privacy concerns, and costs exactly zero dollars per API call?
That's exactly what I built. In this post, I'll walk you through five open-source tools I created for AI-powered document processing — all running locally with Gemma 3 via Ollama. No cloud APIs. No subscriptions. No data leaks.
Why Local LLMs for Document Processing?
Before diving into code, let me explain why I'm passionate about running document AI locally.
1. Confidential Documents Deserve Confidential Processing
Think about what document processing typically involves: financial invoices with bank details, legal contracts with proprietary terms, internal meeting notes with strategic discussions, student records, medical reports. These aren't cat photos — they're sensitive data with real consequences if exposed.
In my experience building search and retrieval systems, I've seen firsthand how important data privacy is. When you process documents locally, the data never leaves your machine. Period.
2. Legal Compliance Is Not Optional
GDPR, HIPAA, SOC 2, attorney-client privilege — the list of regulations governing document handling grows every year. Running a local LLM sidesteps an enormous category of compliance headaches. There's no third-party data processing agreement needed when there's no third party.
3. Cost Savings Add Up Fast
Cloud LLM APIs charge per token. If you're processing hundreds of documents daily, those costs compound quickly. A local setup with Ollama running Gemma 3 on a decent GPU costs nothing after the initial hardware investment. I've processed thousands of documents across my projects without spending a cent on API fees.
4. Offline-First Means Always Available
No internet? No problem. Your document pipeline works on an airplane, in a secure facility, or during an outage. For mission-critical workflows, this reliability is non-negotiable.
The Stack: Ollama + Gemma 3 + Python
All five tools share a common foundation:
-
Ollama — Local LLM runtime that makes running models as easy as
ollama run gemma3 - Gemma 3 — Google's powerful open-weight model, excellent for document understanding
- Python — The glue that ties everything together
- PyPDF2 / pdfplumber — PDF text extraction
- Streamlit — Clean web UI for non-technical users
Here's the base pattern every tool follows:
import requests
import json
def query_local_llm(prompt: str, model: str = "gemma3") -> str:
"""Send a prompt to the local Ollama instance."""
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": model,
"prompt": prompt,
"stream": False,
"options": {
"temperature": 0.3,
"num_predict": 2048
}
}
)
return response.json()["response"]
Low temperature (0.3) keeps the output factual and consistent — exactly what you want for document processing where hallucinations are unacceptable.
Project 1: PDF Report Generator
📄 pdf-report-generator — AI-powered PDF report generator using local Gemma 3 LLM via Ollama
This tool takes raw data or notes and generates polished, structured PDF reports. Think quarterly summaries, research briefs, or project status reports — all generated locally.
from fpdf import FPDF
import requests
def generate_report(topic: str, raw_notes: str) -> str:
prompt = f"""You are a professional report writer.
Given the following topic and raw notes, generate a well-structured
report with sections: Executive Summary, Key Findings,
Detailed Analysis, and Recommendations.
Topic: {topic}
Notes: {raw_notes}
Write the report in a formal, professional tone."""
response = requests.post(
"http://localhost:11434/api/generate",
json={"model": "gemma3", "prompt": prompt, "stream": False}
)
report_text = response.json()["response"]
# Generate PDF
pdf = FPDF()
pdf.add_page()
pdf.set_font("Arial", "B", 16)
pdf.cell(0, 10, topic, ln=True, align="C")
pdf.set_font("Arial", size=11)
pdf.multi_cell(0, 7, report_text)
output_path = f"report_{topic.replace('', '_').lower()}.pdf"
pdf.output(output_path)
return output_path
The beauty here is that your proprietary data — project metrics, financial figures, strategic notes — never touches an external server.
Project 2: Invoice Extractor
📄 invoice-extractor — AI-powered invoice data extractor using local Gemma 3 LLM via Ollama
Invoices contain some of the most sensitive financial data in any organization: vendor details, bank account numbers, tax IDs, payment amounts. This tool extracts structured data from invoice PDFs entirely offline.
import pdfplumber
import json
import requests
def extract_invoice_data(pdf_path: str) -> dict:
# Extract text from PDF
with pdfplumber.open(pdf_path) as pdf:
text = "\\n".join(page.extract_text() or "" for page in pdf.pages)
prompt = f"""Extract the following fields from this invoice text
and return them as valid JSON:
- invoice_number
- date
- vendor_name
- vendor_address
- line_items (list of description, quantity, unit_price, total)
- subtotal
- tax
- total_amount
- payment_terms
Invoice text:
{text}
Return ONLY valid JSON, no explanation."""
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "gemma3",
"prompt": prompt,
"stream": False,
"options": {"temperature": 0.1}
}
)
result = response.json()["response"]
return json.loads(result)
I set temperature to 0.1 here — even lower than usual — because invoice extraction demands precision. A wrong decimal point in a payment amount is a real problem.
Project 3: Textbook Summarizer
📄 textbook-summarizer — AI-powered textbook chapter summarizer using local Gemma 3 LLM via Ollama
Whether you're a student preparing for exams or a professional staying current with technical literature, summarizing dense textbook chapters is a time sink. This tool processes chapters into concise, structured summaries.
import pdfplumber
import requests
def summarize_chapter(pdf_path: str, chapter_start: int, chapter_end: int) -> str:
with pdfplumber.open(pdf_path) as pdf:
chapter_text = "\\n".join(
pdf.pages[i].extract_text() or ""
for i in range(chapter_start - 1, min(chapter_end, len(pdf.pages)))
)
prompt = f"""Summarize this textbook chapter. Include:
1. **Key Concepts** — Main ideas and definitions
2. **Important Formulas/Rules** — Any critical formulas or rules
3. **Summary** — A 3-5 paragraph overview of the chapter
4. **Study Questions** — 5 questions a student should be able to answer
Chapter text:
{chapter_text[:8000]}
Provide a thorough but concise summary."""
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "gemma3",
"prompt": prompt,
"stream": False,
"options": {"temperature": 0.3, "num_predict": 3000}
}
)
return response.json()["response"]
The chunking strategy (limiting to 8000 characters) is important. For longer chapters, you'd want to implement a sliding window approach that summarizes sections individually, then combines them.
Project 4: Legal Document Summarizer
📄 legal-doc-summarizer — AI-powered legal document summarizer using local Gemma 3 LLM via Ollama
Legal documents are the poster child for why local processing matters. Attorney-client privilege, confidential settlement terms, proprietary contract clauses — none of this should ever be sent to a cloud API.
import pdfplumber
import requests
def summarize_legal_document(pdf_path: str) -> dict:
with pdfplumber.open(pdf_path) as pdf:
full_text = "\\n".join(page.extract_text() or "" for page in pdf.pages)
page_count = len(pdf.pages)
prompt = f"""You are a legal document analyst. Analyze this legal
document and provide:
1. **Document Type** — Contract, NDA, agreement, filing, etc.
2. **Parties Involved** — Who are the parties?
3. **Key Terms** — Important obligations, rights, and conditions
4. **Critical Dates** — Deadlines, effective dates, expiration
5. **Financial Terms** — Payment amounts, penalties, fees
6. **Risk Factors** — Potential concerns or unusual clauses
7. **Plain Language Summary** — What this document means in simple terms
Document text:
{full_text[:10000]}
Be thorough but concise. Flag anything unusual."""
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "gemma3",
"prompt": prompt,
"stream": False,
"options": {"temperature": 0.2, "num_predict": 4000}
}
)
return {
"source_file": pdf_path,
"summary": response.json()["response"],
"pages_processed": page_count
}
This tool is specifically designed to be a first-pass analysis tool. It doesn't replace legal counsel — it helps legal professionals quickly triage and understand large volumes of documents.
Project 5: Meeting Summarizer
📄 meeting-summarizer — AI-powered meeting notes summarizer using local Gemma 3 LLM via Ollama
Meeting notes and transcripts often contain sensitive strategic discussions, personnel decisions, and financial planning. This tool turns raw meeting content into structured, actionable summaries.
import requests
def summarize_meeting(transcript: str, meeting_title: str = "") -> str:
prompt = f"""Summarize these meeting notes into a structured format:
**Meeting:** {meeting_title}
**Required sections:**
1. **Attendees** — Who was present (if mentioned)
2. **Agenda Items** — What topics were discussed
3. **Key Decisions** — Decisions that were made
4. **Action Items** — Tasks assigned, with owners and deadlines
5. **Open Questions** — Unresolved items needing follow-up
6. **Next Steps** — What happens next
Meeting transcript:
{transcript}
Focus on actionable takeaways. Be specific about who owns what."""
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "gemma3",
"prompt": prompt,
"stream": False,
"options": {"temperature": 0.3, "num_predict": 2500}
}
)
return response.json()["response"]
The structured output format — especially the action items with owners — makes this immediately useful for project management workflows.
Getting Started
Setting up the entire pipeline takes about 10 minutes:
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh
# Pull the Gemma model
ollama pull gemma3
# Clone any of the projects
git clone https://github.com/kennedyraju55/pdf-report-generator.git
cd pdf-report-generator
# Install dependencies
pip install -r requirements.txt
# Run the app
streamlit run app.py
All five projects follow this same pattern. Clone, install, run. No API keys to configure, no accounts to create, no billing to set up.
What I Learned Building These Tools
After building 116+ open-source repositories — many focused on local AI — here are my key takeaways:
Prompt engineering matters more than model size. A well-crafted prompt with Gemma 3 outperforms a lazy prompt with GPT-4 for structured extraction tasks.
Temperature is your precision dial. For document processing, keep it between 0.1-0.3. Creative writing? Crank it up. Invoice extraction? Keep it cold.
Chunking strategy is critical. Long documents need intelligent splitting — by sections, not arbitrary character counts. Respect document structure.
Local doesn't mean slow. On a modern GPU, Gemma 3 processes most documents in seconds. The overhead of an API call (network latency, rate limits, retries) often makes cloud slower in practice.
Privacy is a feature, not a limitation. The moment I stopped treating local inference as a compromise and started treating it as a feature, the use cases multiplied.
Conclusion
You don't need to send sensitive documents to the cloud to get AI-powered processing. With Ollama, Gemma 3, and a bit of Python, you can build a complete document processing pipeline that's private, free, and works offline.
All five projects are open source and ready to use. Clone them, modify them, build on top of them. That's the whole point.
About the Author
Nrk Raju Guthikonda is a Senior Software Engineer at Microsoft on the Copilot Search Infrastructure team, specializing in semantic indexing and RAG systems. With 116+ open-source repositories, he builds AI-powered tools that prioritize privacy and local-first processing.
🔗 GitHub: github.com/kennedyraju55
🔗 Dev.to: dev.to/kennedyraju55
🔗 LinkedIn: linkedin.com/in/nrk-raju-guthikonda-504066a8