Chapter 3: Handling Different File Types

Introduction

Real-world documents come in many formats, each with unique challenges. This chapter covers strategies for handling PDFs, images, tables, and structured data.

PDF Challenges

PDFs are the most common document format, but also the most challenging:

Problem Areas

Challenge	Description	Solution
Scanned PDFs	Images, not text	OCR processing
Tables	Structure lost	Specialized extractors
Multi-column	Reading order wrong	Layout analysis
Headers/Footers	Repeated noise	Filtering

PDF Extraction Options

# Option 1: Basic (PyPDF2/PyPDFLoader)
from langchain.document_loaders import PyPDFLoader
loader = PyPDFLoader("document.pdf")
# Pros: Fast, simple
# Cons: Poor with complex layouts

# Option 2: Better tables (PDFPlumber)
from langchain.document_loaders import PDFPlumberLoader
loader = PDFPlumberLoader("document.pdf")
# Pros: Better table extraction
# Cons: Slower

# Option 3: Best quality (Unstructured)
from langchain.document_loaders import UnstructuredPDFLoader
loader = UnstructuredPDFLoader(
    "document.pdf",
    mode="elements",  # Preserves structure
    strategy="hi_res"  # Best quality
)
# Pros: Best overall quality
# Cons: Requires additional dependencies

OCR for Scanned Documents

For scanned PDFs and images:

# Using Tesseract OCR
from langchain.document_loaders import UnstructuredPDFLoader

loader = UnstructuredPDFLoader(
    "scanned_document.pdf",
    strategy="ocr_only",
    ocr_languages="eng"
)
documents = loader.load()

# Using Amazon Textract
from langchain.document_loaders import AmazonTextractPDFLoader

loader = AmazonTextractPDFLoader(
    "scanned.pdf",
    client=textract_client
)

OCR Best Practices

1. Pre-process images - Deskew, enhance contrast
2. Specify language - Improves accuracy
3. Post-process text - Fix common OCR errors
4. Confidence scores - Flag low-confidence extractions

Handling Tables

Tables require special handling to preserve structure:

# Method 1: Convert to Markdown
from langchain.document_loaders import UnstructuredPDFLoader

loader = UnstructuredPDFLoader(
    "report.pdf",
    mode="elements"
)
docs = loader.load()

# Tables come as structured elements
for doc in docs:
    if doc.metadata.get("category") == "Table":
        # Process table specially
        table_markdown = convert_table_to_markdown(doc)

# Method 2: Using Camelot for PDFs
import camelot

tables = camelot.read_pdf("document.pdf", pages="all")
for table in tables:
    df = table.df
    markdown = df.to_markdown()

Table Chunking Strategy

def chunk_with_tables(documents):
    chunks = []
    
    for doc in documents:
        if is_table(doc):
            # Keep tables as single chunks
            chunks.append(doc)
        else:
            # Normal chunking for text
            text_chunks = splitter.split_documents([doc])
            chunks.extend(text_chunks)
    
    return chunks

Images in Documents

Extract and process images:

# Extract images from PDF
import fitz  # PyMuPDF

doc = fitz.open("document.pdf")
for page_num, page in enumerate(doc):
    images = page.get_images()
    for img_index, img in enumerate(images):
        # Extract image
        xref = img[0]
        base_image = doc.extract_image(xref)
        image_bytes = base_image["image"]
        
        # Process with vision model
        description = vision_model.describe(image_bytes)
        
        # Add description to document

Vision Model Integration

from langchain.chat_models import ChatOpenAI

# Use GPT-4 Vision for image understanding
vision_llm = ChatOpenAI(model="gpt-4-vision-preview")

def describe_image(image_path):
    response = vision_llm.invoke([
        {
            "type": "image_url",
            "image_url": f"data:image/jpeg;base64,{base64_image}"
        },
        {
            "type": "text",
            "text": "Describe this image in detail for document search."
        }
    ])
    return response.content

Structured Data

JSON/JSONL

from langchain.document_loaders import JSONLoader

loader = JSONLoader(
    file_path="data.json",
    jq_schema=".data[].content",
    text_content=False
)

CSV

from langchain.document_loaders import CSVLoader

loader = CSVLoader(
    file_path="data.csv",
    csv_args={"delimiter": ","},
    source_column="source"
)

# For large CSVs, process in batches
import pandas as pd

def load_csv_in_batches(file_path, batch_size=1000):
    for chunk in pd.read_csv(file_path, chunksize=batch_size):
        yield from chunk.to_dict('records')

SQL Databases

from langchain.document_loaders import SQLDatabaseLoader
from langchain.utilities import SQLDatabase

db = SQLDatabase.from_uri("postgresql://user:pass@localhost/db")

loader = SQLDatabaseLoader(
    db=db,
    query="SELECT id, content, metadata FROM documents WHERE active = true"
)

Complete Processing Pipeline

from pathlib import Path

def process_document(file_path: str):
    """Universal document processor"""
    path = Path(file_path)
    extension = path.suffix.lower()
    
    loaders = {
        ".pdf": lambda p: UnstructuredPDFLoader(p, strategy="hi_res"),
        ".docx": lambda p: Docx2txtLoader(p),
        ".txt": lambda p: TextLoader(p, encoding="utf-8"),
        ".html": lambda p: UnstructuredHTMLLoader(p),
        ".md": lambda p: UnstructuredMarkdownLoader(p),
        ".csv": lambda p: CSVLoader(p),
        ".json": lambda p: JSONLoader(p, jq_schema=".[]")
    }
    
    if extension not in loaders:
        raise ValueError(f"Unsupported file type: {extension}")
    
    loader = loaders[extension](str(path))
    documents = loader.load()
    
    # Add file metadata
    for doc in documents:
        doc.metadata["file_name"] = path.name
        doc.metadata["file_type"] = extension
        doc.metadata["file_size"] = path.stat().st_size
    
    return documents

Summary

File Type	Best Loader	Notes
PDF (text)	PDFPlumberLoader	Good table support
PDF (scanned)	UnstructuredPDFLoader + OCR	Use hi_res strategy
Word	Docx2txtLoader	Fast and reliable
HTML	UnstructuredHTMLLoader	Removes tags
CSV	CSVLoader	Batch for large files
Images	Vision models	For embedded images

Key Takeaways

1. PDFs are complex - Use the right loader for your needs
2. OCR for scanned docs - Essential for image-based PDFs
3. Tables need special handling - Don't chunk across rows
4. Images need description - Use vision models
5. Test your pipeline - With representative samples

---

Next Steps

Test your knowledge with the Module 2 Quiz!