Back to Blog
Technology2026-07-0428 min Read

The Ultimate Guide to File Conversion: Audio, Video, Images, Documents, PDFs and More

Everything you need to know about modern file conversion—from MP4 and PDF to WebP, OCR, Office documents and eBooks.

Related Solutions

The Ultimate Guide to File Conversion: Audio, Video, Images, Documents, PDFs and More

A comprehensive reference covering file conversion concepts, best practices, installation guides, supported formats, detailed example commands, and advanced workflows.

Table of Contents

  1. Introduction
  2. Installing the Conversion Toolkit
  3. Audio Conversion & Optimization
  4. Video Conversion & Optimization
  5. Image Conversion & Optimization
  6. Document Conversion
  7. PDF Operations
  8. OCR (Optical Character Recognition)
  9. eBook Conversion
  10. Archives and Compression
  11. Metadata Management
  12. Fonts and Typography
  13. CAD and 3D Files
  14. HTML Rendering and Web Content
  15. Best Practices and Advanced Tips

Introduction

In today's digital world, file conversion is essential for compatibility, optimization, and accessibility. Modern conversion platforms don't rely on a single tool but orchestrate multiple specialized applications. This guide combines conceptual knowledge with practical, ready-to-use commands for common tasks.

Whether you're a developer, content creator, or power user, understanding these tools will streamline your workflow.

Why Convert Files?

  • Compatibility: Ensure files work across devices and software (different OS, media players, browsers).
  • Optimization: Reduce file size without quality loss for faster distribution, lower bandwidth costs, and better performance.
  • Accessibility: Enable features like searchability (OCR), captions, alt text, or convert to universal formats for screen readers.
  • Archiving: Preserve originals in lossless formats for long-term storage and future-proofing.
  • Distribution: Prepare files for specific platforms (web uses WebP/MP4, mobile uses AAC/H.264, print uses CMYK/PDF).
CategoryPrimary Tool(s)Common Use Cases
Audio & VideoFFmpegTranscoding, editing, streaming
ImagesImageMagickFormat conversion, resizing
Office DocumentsLibreOfficeDOCX → PDF, spreadsheets
Markdown/HTMLPandocMulti-format publishing
PDFsGhostscript, Poppler, QPDFCompression, merging, optimization
OCRTesseract, OCRmyPDFScanned documents
MetadataExifToolViewing/editing EXIF, IPTC
eBooksCalibre (ebook-convert)EPUB, MOBI, PDF interconversion
FontsFontForgeTTF ↔ WOFF, subsetting
HTML RenderingChromium (headless)HTML to PDF

Installing the Conversion Toolkit

A robust toolkit requires installing several open-source tools. Below are platform-specific installation instructions with verification steps.

FFmpeg (Audio/Video) - v6.0+

The most critical tool for multimedia conversion. FFmpeg includes ffmpeg (conversion), ffprobe (inspection), and ffplay (playback).

Ubuntu/Debian (Latest):

bash
1# Add FFmpeg PPA for latest version
2sudo add-apt-repository ppa:ubuntuhandbook1/ffmpeg6
3sudo apt update && sudo apt install ffmpeg

macOS (Homebrew):

bash
1brew install ffmpeg
2# Optional: install with additional codecs
3brew install ffmpeg --with-libvpx --with-libvorbis

Windows (winget):

powershell
1winget install Gyan.FFmpeg
2# Verify installation (restart terminal after install)
3ffmpeg -version

Verify Installation:

bash
1ffmpeg -version
2ffprobe -version
3# Check available encoders
4ffmpeg -encoders | grep -E "libx264|libx265|libvpx|libopus"

Troubleshooting:

  • Codec missing errors: Recompile FFmpeg with specific libraries or install ffmpeg-full variant
  • Permission denied on macOS: chmod +x $(which ffmpeg)
  • WSL on Windows: Use WSL Ubuntu installation for better libx265 support

ImageMagick (Image Processing) - v7.0+

Powerful for batch image operations with support for 200+ image formats.

bash
1# Ubuntu/Debian
2sudo apt install imagemagick libmagickwand-dev
3
4# macOS
5brew install imagemagick
6
7# Windows
8winget install ImageMagick.ImageMagick
9
10# Verify
11identify -version
12magick -version

Note: ImageMagick 7+ uses magick command instead of convert. Both work, but magick is recommended.

LibreOffice (Document Conversion) - v7.0+

Essential for server-side Office document processing without requiring Microsoft Office.

bash
1# Ubuntu/Debian
2sudo apt install libreoffice
3
4# macOS
5brew install --cask libreoffice
6
7# Windows
8winget install TheDocumentFoundation.LibreOffice
9
10# Verify
11libreoffice --version

Headless Mode (for server automation):

bash
1# Test headless conversion
2libreoffice --headless --convert-to pdf test.docx

Complete Tool Installation Matrix

ToolUbuntu/DebianmacOSWindowsMin VersionPurpose
FFmpegapt install ffmpegbrew install ffmpegwinget6.0Audio/Video transcoding
ImageMagickapt install imagemagickbrew install imagemagickwinget7.0Image processing
LibreOfficeapt install libreofficebrew install --cask libreofficewinget7.0Office document conversion
Pandocapt install pandocbrew install pandocwinget3.0Document markup conversion
Ghostscriptapt install ghostscriptbrew install ghostscriptwinget10.0PDF processing
Tesseractapt install tesseract-ocrbrew install tesseractwinget5.0Optical character recognition
QPDFapt install qpdfbrew install qpdfwinget10.6PDF manipulation
ExifToolapt install libimage-exiftool-perlbrew install exiftoolwinget12.0Metadata management
Calibreapt install calibrebrew install --cask calibrewinget6.0eBook conversion
Popplerapt install poppler-utilsbrew install popplerwinget21.0PDF extraction/rendering

Automated Installation Script

For Ubuntu/Debian:

bash
1#!/bin/bash
2sudo apt update
3sudo apt install -y ffmpeg imagemagick libreoffice pandoc ghostscript \
4  tesseract-ocr poppler-utils qpdf libimage-exiftool-perl calibre
5
6echo "Verifying installations..."
7ffmpeg -version | head -1
8identify -version | head -1
9libreoffice --version

For macOS:

bash
1#!/bin/bash
2brew update
3brew install ffmpeg imagemagick pandoc ghostscript tesseract poppler qpdf exiftool
4brew install --cask libreoffice calibre
5
6echo "Verifying installations..."
7ffmpeg -version | head -1
8magick -version | head -1
9libreoffice --version

System Dependencies

Some tools require additional libraries for full functionality:

Ubuntu/Debian:

bash
1# For FFmpeg with comprehensive codec support
2sudo apt install libx264-dev libx265-dev libvpx-dev libopus-dev libvorbis-dev
3
4# For ImageMagick advanced features
5sudo apt install libjpeg-dev libpng-dev libtiff-dev libwebp-dev

macOS:

bash
1# Xcode command line tools (required for building)
2xcode-select --install
3
4# Additional development headers
5brew install pkg-config

Verification & Troubleshooting

bash
1# Test all conversions work
2echo "Testing FFmpeg..."
3ffmpeg -encoders | grep -q "libx264" && echo "✓ H.264 available" || echo "✗ H.264 missing"
4
5echo "Testing ImageMagick..."
6magick -list format | grep -q "PDF" && echo "✓ PDF support" || echo "✗ PDF missing"
7
8echo "Testing LibreOffice..."
9libreoffice --headless --proofread test.txt 2>/dev/null && echo "✓ LibreOffice headless OK"
10
11# Check disk space
12df -h | grep -E "/$|/home"

Audio Conversion & Optimization

FFmpeg is the gold standard for audio handling. It supports dozens of formats with high-quality codecs and powerful optimization features. Understanding bitrate, sample rate, and channels is crucial for optimal results.

Audio Format Reference

FormatCodecBitrate (Typical)Sample RateChannelsQualityUse CaseProsCons
MP3MPEG-1 Layer III128-320k8-48kHz1-2LossyUniversal playbackMaximum compatibilityLossy, legacy codec
AACAdvanced Audio Coding96-256k8-48kHz1-5.1LossyMobile & streamingBetter quality/sizeLess open-source
M4AMP4 Audio128-256k8-48kHz1-5.1LossyiOS/iTunesApple ecosystemProprietary feel
OpusOpus16-320k8-48kHz1-8LossyLow-bandwidth streamingBest quality at low BRLimited hardware support
FLACFree Lossless600-1200k8-192kHz1-8LosslessArchivingPerfect qualityLarge files
WAVPCM1411k (CD)44.1kHz1-2LosslessEditing/productionUncompressed, standardVery large
OGGVorbis96-320k8-48kHz1-2LossyOpen source projectsFree codecLimited device support

Bitrate Quality Guidelines

  • 32k Opus/96k AAC: Speech, podcasts—acceptable but noticeable compression
  • 64k Opus/128k AAC/MP3: Audiobooks, background music—good for casual listening
  • 192k MP3/256k AAC/128k Opus: Music streaming, general listening—transparent to most ears
  • 320k MP3/FLAC: Audiophiles, professional use—optimal for lossless or high-quality lossy

Audio Parameters Explained

text
1-b:a      Bitrate (e.g., 192k)
2-ar       Sample rate in Hz (44100, 48000, 96000)
3-ac       Audio channels (1=mono, 2=stereo, 6=5.1 surround)
4-q:a      VBR quality (0-9, lower=better, not all codecs support)
5-af       Audio filters (loudnorm, volume, bass, treble, etc.)

Comprehensive Audio Conversion Examples

bash
1# Standard MP3 conversion (128k, stereo, 44.1kHz)
2ffmpeg -i input.wav -b:a 128k -ar 44100 -ac 2 output.mp3
3
4# High-quality MP3 for archiving (320k lossless-like)
5ffmpeg -i input.wav -b:a 320k -ar 48000 output.mp3
6
7# Optimize FLAC to streaming-friendly AAC
8ffmpeg -i input.flac -c:a aac -b:a 192k -ar 44100 output.m4a
9
10# Convert to bandwidth-optimized Opus
11ffmpeg -i input.wav -c:a libopus -b:a 64k -vbr constrained output.opus
12
13# Normalize audio volume and convert to MP3
14ffmpeg -i input.mp3 -af loudnorm=I=-16:TP=-1.5:LRA=11 -b:a 192k output_normalized.mp3
15
16# Extract audio from video, downmix to mono, and optimize
17ffmpeg -i input.mp4 -vn -c:a libmp3lame -b:a 64k -ac 1 output_mono.mp3
18
19# Batch convert all WAV files to optimized MP3
20for f in *.wav; do 
21  ffmpeg -i "$f" -b:a 192k -af loudnorm "${f%.wav}.mp3"
22done
23
24# Resample audio to different sample rate (48kHz to 44.1kHz)
25ffmpeg -i input_48k.wav -ar 44100 output.wav
26
27# Add metadata to MP3
28ffmpeg -i input.mp3 -c copy -metadata artist="Artist Name" -metadata title="Track Title" output.mp3
29
30# Merge multiple audio files
31ffmpeg -f concat -safe 0 -i <(for f in *.mp3; do echo "file '$f'"; done) -c copy output.mp3
32
33# Convert stereo to 5.1 surround (upmix with filter)
34ffmpeg -i input.mp3 -af 'channelmap=channel_layout=5.1' -c:a libfdk_aac -b:a 384k output.m4a
35
36# Trim audio (first 30 seconds)
37ffmpeg -i input.mp3 -ss 0 -to 30 -c copy output.mp3
38
39# Split audio every 10 minutes
40ffmpeg -i input.mp3 -f segment -segment_time 600 -c copy output_%03d.mp3

Audio Filters for Enhancement

bash
1# Compress audio (reduce dynamic range)
2ffmpeg -i input.wav -af "compand=attacks=0:decays=0:points=-80/-80|-24/-24|0/0|20/20" output.wav
3
4# Add bass boost
5ffmpeg -i input.wav -af 'bass=g=10:f=200:w=0.3' output.wav
6
7# Remove clicks and pops (declick)
8ffmpeg -i input.wav -af 'anlmdn=s=0.001:p=0.0001' output.wav
9
10# Reverb effect (room simulation)
11ffmpeg -i input.wav -af "aecho=0.8:0.9:1000:0.3" output.wav
12
13# Fade in (3 seconds)
14ffmpeg -i input.mp3 -af "afade=t=in:st=0:d=3" -c:a copy output.mp3
15
16# Fade out (5 seconds)
17ffmpeg -i input.mp3 -af "afade=t=out:st=$(ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1:precision=0 input.mp3 | xargs expr $(date +%s) -):d=5" output.mp3

Quality Assessment & Troubleshooting

bash
1# Inspect audio properties
2ffprobe -v error -show_entries stream=codec_name,sample_rate,channels,bit_rate input.mp3
3
4# Compare file sizes and quality
5ls -lh input.* && echo "---" && \
6ffmpeg -i input.wav -b:a 128k -c:a libmp3lame temp.mp3 && ls -lh temp.mp3
7
8# Test opus quality at various bitrates
9for br in 32 64 128 256; do
10  ffmpeg -i input.wav -c:a libopus -b:a ${br}k test_opus_${br}k.opus
11  ls -lh test_opus_${br}k.opus
12done
  1. Assess source: Use ffprobe to check original bitrate and sample rate
  2. Choose target bitrate: Balance quality vs. file size (typically 128-192k for music)
  3. Normalize: Apply loudnorm filter for consistent volume across files
  4. Convert with filters: Apply any necessary audio enhancement
  5. Verify output: Spot-check converted file with audio player
  6. Batch process: Use loops for multiple files to save time

Video Conversion & Optimization

FFmpeg excels at video transcoding with hardware acceleration support. Understanding codec selection, bitrate planning, and encoding strategies is essential for efficient video processing.

Video Codec Comparison

CodecContainerBitrate (1080p/30fps)Hardware SupportQualityCompatibilityBest For
H.264/AVCMP4, MKV1-5 MbpsExcellent (NVENC, VA-API)HighUniversalWeb, streaming, universal
H.265/HEVCMP4, MKV0.5-3 MbpsGood (NVENC, QuickSync)ExcellentLimited (Apple, modern browsers)Archiving, storage, UHD
VP9WebM0.8-4 MbpsLimitedHighChrome, Firefox, EdgeYouTube, open-source web
AV1MP4, WebM, Matroska0.4-2 MbpsVery limitedExcellentLimited (recent browsers)Future-proof, very high compression
ProResMOV8-12 MbpsGood (Apple hardware)HighestApple ecosystemProfessional video editing
DNxHDMKV, MOV8-12 MbpsLimitedHighestProfessional workflowsAvid, professional editing

Encoding Presets & Quality Trade-offs

text
1Preset    Speed     Quality     File Size    Use Case
2superfast  ✓✓✓✓✓    ✗✗        Smaller     Real-time, livestreaming
3veryfast   ✓✓✓✓     ✗         Smaller     Quick conversions
4fast       ✓✓✓      ✓         Medium      Default for web
5medium     ✓✓       ✓✓        Medium      Balanced approach
6slow       ✓        ✓✓✓       Larger      Quality priority
7slower     ✗        ✓✓✓✓      Larger      Archiving, one-time

CRF (Constant Rate Factor) Quality Guide:

  • crf 18-22: High quality (near-lossless), ideal for archiving
  • crf 23-28: Balanced quality for web streaming
  • crf 29-32: Lower quality for mobile/low-bandwidth
  • crf 51: Lowest quality (rarely used)

Video Conversion Examples

Basic High-Quality MP4 (H.264):

bash
1ffmpeg -i input.mov -c:v libx264 -preset slow -crf 23 -c:a aac -b:a 192k output.mp4

Streaming-Optimized MP4 (Fast encoding, good quality):

bash
1ffmpeg -i input.mov -c:v libx264 -preset fast -crf 25 \
2  -maxrate 5000k -bufsize 10000k \
3  -c:a aac -b:a 128k output_streaming.mp4

Downscale 4K to 1080p for web:

bash
1ffmpeg -i input_4k.mp4 -vf scale=1920:1080:flags=lanczos -c:v libx264 \
2  -preset medium -crf 25 -c:a aac -b:a 128k output_1080p.mp4

High-compression H.265/HEVC (50% smaller, slower encoding):

bash
1ffmpeg -i input.mp4 -c:v libx265 -preset slow -crf 24 \
2  -c:a aac -b:a 192k output_hevc.mp4

Modern codec stack (H.265 video + AAC audio with fallback compatibility):

bash
1ffmpeg -i input.mov \
2  -c:v libx265 -preset medium -crf 25 -tag:v hvc1 \
3  -c:a aac -b:a 192k \
4  output.mp4

Web-optimized VP9/WebM (YouTube-style):

bash
1ffmpeg -i input.mp4 -c:v libvpx-vp9 -b:v 0 -crf 30 \
2  -c:a libopus -b:a 128k output.webm

Extract specific frames as images:

bash
1# Extract frame at 10 seconds
2ffmpeg -i input.mp4 -ss 00:00:10 -vframes 1 -q:v 2 frame.png
3
4# Extract every 5 seconds
5ffmpeg -i input.mp4 -vf fps=fps=0.2 frame_%04d.png
6
7# Extract specific frame range every second
8ffmpeg -i input.mp4 -vf fps=1 -ss 10 -to 30 frame_%04d.png

Two-Pass Encoding (optimal quality/size ratio):

bash
1# Pass 1: Create statistics
2ffmpeg -i input.mp4 -c:v libx264 -b:v 2M -preset medium \
3  -pass 1 -f null /dev/null
4
5# Pass 2: Encode with statistics
6ffmpeg -i input.mp4 -c:v libx264 -b:v 2M -preset medium \
7  -pass 2 -c:a aac -b:a 192k output.mp4

Batch convert multiple files with progress tracking:

bash
1#!/bin/bash
2total=$(ls *.mov | wc -l)
3count=0
4
5for input in *.mov; do
6  ((count++))
7  output="${input%.mov}.mp4"
8  echo "Converting $count/$total: $input -> $output"
9  
10  ffmpeg -i "$input" -c:v libx264 -preset fast -crf 25 \
11    -c:a aac -b:a 128k "$output"
12  
13  # Optional: remove original after successful conversion
14  # rm "$input"
15done

Hardware Acceleration Options

NVIDIA CUDA (GeForce/RTX cards):

bash
1ffmpeg -i input.mp4 -hwaccel cuda -c:v h264_cuvid -c:v h264_nvenc \
2  -preset slow -rc vbr -crf 23 output.mp4

Intel Quick Sync (integrated/dedicated Intel GPU):

bash
1ffmpeg -i input.mp4 -hwaccel qsv -c:v h264_qsv -preset 7 \
2  output.mp4

AMD VAAPI (Linux with AMD GPU):

bash
1ffmpeg -hwaccel vaapi -i input.mp4 -c:v hevc_vaapi -qp 28 \
2  output.mp4

Apple VideoToolbox (macOS):

bash
1ffmpeg -i input.mp4 -c:v h264_videotoolbox -b:v 5M output.mp4

Adaptive Bitrate Streaming (Multi-bitrate)

For streaming platforms like HLS or DASH:

bash
1# Create multiple bitrate variants
2for br in 1000k 2500k 5000k; do
3  ffmpeg -i input.mp4 -c:v libx264 -b:v $br -maxrate ${br} \
4    -bufsize $((br*2)) -preset fast -c:a aac -b:a 128k \
5    output_${br}.mp4
6done
7
8# Create HLS manifest
9ffmpeg -i output_1000k.mp4 -i output_2500k.mp4 -i output_5000k.mp4 \
10  -filter_complex "[0:v]scale=1280:720[v0];[1:v]scale=1920:1080[v1];[2:v]scale=3840:2160[v2]" \
11  -map "[v0]" -map 0:a -map "[v1]" -map 1:a -map "[v2]" -map 2:a \
12  -f hls -var_stream_select "v:0,a:0 v:1,a:1 v:2,a:2" master.m3u8

Advanced Filters & Effects

bash
1# Add watermark/overlay
2ffmpeg -i input.mp4 -i watermark.png -filter_complex \
3  "[0:v][1:v]overlay=W-w-10:H-h-10" -c:a copy output.mp4
4
5# Blur background (portrait mode effect)
6ffmpeg -i input.mp4 -vf "scale=1280:720,setsar=1,fps=30" \
7  -filter_complex "format=yuv420p,scale=640:360,scale=1280:720" \
8  output.mp4
9
10# Denoise video
11ffmpeg -i input.mp4 -vf "hqdn3d=luma_spatial=5:chroma_spatial=5" \
12  -q:v 5 output.mp4
13
14# Stabilize shaky video
15ffmpeg -i input.mp4 -vf vidstabdetect=stepsize=32:shakiness=5 \
16  -f null -
17ffmpeg -i input.mp4 -vf vidstabtransform=input='transforms.trf' \
18  -c:v libx264 output.mp4

Quality Assessment

bash
1# Get video statistics
2ffprobe -v error -show_entries stream=codec_name,profile,level,width,height,r_frame_rate,bit_rate input.mp4
3
4# Compare file sizes
5ls -lh input.mp4 output_*.mp4
6
7# Detect interlacing (for deinterlacing if needed)
8ffmpeg -i input.mp4 -vf idet -t 10 -f null -

Conversion Workflow Best Practices

  1. Inspect source: ffprobe input.mp4 to understand codec and bitrate
  2. Choose codec: H.264 for broad compatibility, H.265 for archiving/storage savings
  3. Optimize resolution: Downscale only if necessary to preserve quality
  4. Select preset: Use fast for web, medium for balanced, slow for quality
  5. Set target bitrate: 2-4 Mbps for 1080p, 1-2 Mbps for 720p
  6. Use two-pass for critical conversions (archiving, final delivery)
  7. Verify output: Spot-check quality before batch processing

Image Conversion & Optimization

ImageMagick and related tools are powerful for batch processing, resizing, optimization, and transformations. Modern image formats like WebP and AVIF offer significant compression while maintaining quality.

Image Format Comparison

FormatCompressionTransparencyAnimationQualityBrowser SupportBest ForFile Size (1000x1000)
JPEGLossyHighUniversalPhotography, complex images~100-300KB
PNGLosslessHighestUniversalGraphics, icons, screenshots~400-800KB
WebPLossy/LosslessHighChrome, Firefox, Edge, SafariWeb images, fallback legacy~50-150KB
AVIFLossy/LosslessExcellentModern browsers onlyFuture-proof web, archiving~30-80KB
HEICLossy/LosslessExcellentApple devicesiOS/macOS, mobile~60-150KB
GIFLosslessLimited (256 colors)UniversalAnimations, simple graphics~300-500KB
TIFFLossless/LossyHighestLimited (print, archiving)Professional print, archiving~1-3MB
SVGVector/LosslessScalableUniversal (text-based)Icons, logos, diagramsVariable

Image Optimization Strategy by Device

Desktop/Web (1920px width):

bash
1# Primary: WebP with JPEG fallback
2magick input.png -resize 1920x1080 -quality 85 -define webp:method=6 output.webp
3magick input.png -resize 1920x1080 -quality 85 output.jpg
4
5# Modern: AVIF for cutting-edge browsers
6magick input.png -resize 1920x1080 -quality 75 output.avif

Mobile (600px width, retina display):

bash
1# Primary: WebP lightweight
2magick input.png -resize 1200x800 -quality 80 -define webp:method=6 output_mobile.webp
3
4# Fallback: Optimized JPEG
5magick input.png -resize 1200x800 -quality 80 -strip output_mobile.jpg

Thumbnail (200px):

bash
1# Aggressive optimization for thumbnails
2magick input.png -resize 200x200 -quality 75 -define webp:lossless=false -strip output_thumb.webp

Comprehensive Image Conversion Examples

Basic format conversion:

bash
1# PNG to JPEG
2magick input.png -background white -alpha off output.jpg
3
4# JPEG to PNG (add transparency support)
5magick input.jpg -transparent white output.png
6
7# Any format to WebP
8magick input.jpg -quality 85 output.webp
9
10# Any format to AVIF
11magick input.jpg -quality 75 output.avif

Downscaling and optimization (web-ready):

bash
1# Resize and optimize in one command
2magick input.png -resize 1920x1080 -quality 85 -strip -interlace Plane output.jpg
3
4# Aggressive optimization for mobile (35% file size reduction)
5magick input.jpg -resize 1200x800 -quality 75 -strip -sampling-factor 4:2:0 -define jpeg:dct-method=float output_optimized.jpg
6
7# Convert PNG to WebP with quality preservation
8magick input.png -quality 85 -define webp:method=6 output.webp
9
10# Convert to AVIF (best compression, slower encoding)
11magick input.jpg -quality 75 -format avif output.avif

Upscaling with enhancement:

bash
1# Upscale 50% with Lanczos sharpening (quality varies)
2magick input_lowres.jpg -resize 150% -unsharp 0x1+0.5+0.05 output_upscaled.jpg
3
4# For professional upscaling: use external AI tools
5# - Topaz Gigapixel AI (best quality but paid)
6# - Real-ESRGAN (open-source, good results)
7# - waifu2x (anime-specific)

Batch processing with progress:

bash
1#!/bin/bash
2# Batch optimize all JPEGs in directory
3total=$(ls *.jpg 2>/dev/null | wc -l)
4count=0
5
6for img in *.jpg; do
7  ((count++))
8  echo "Processing $count/$total: $img"
9  
10  # Create WebP version
11  magick "$img" -resize 1920x1080 -quality 85 -define webp:method=6 "${img%.jpg}.webp"
12  
13  # Optimize original file size
14  magick "$img" -quality 85 -strip "${img%.jpg}_optimized.jpg"
15done

Responsive image generation (multiple sizes):

bash
1#!/bin/bash
2# Generate responsive image set for web
3
4input="original.jpg"
5base="${input%.*}"
6
7# Small (mobile)
8magick "$input" -resize 640x480 -quality 80 -define webp:method=6 "${base}_sm.webp"
9
10# Medium (tablet)
11magick "$input" -resize 1024x768 -quality 82 -define webp:method=6 "${base}_md.webp"
12
13# Large (desktop)
14magick "$input" -resize 1920x1440 -quality 85 -define webp:method=6 "${base}_lg.webp"
15
16# Generate HTML with responsive srcset
17cat > responsive_image.html << 'EOF'
18<picture>
19  <source 
20    srcset="image_sm.webp 640w, image_md.webp 1024w, image_lg.webp 1920w"
21    type="image/webp">
22  <source 
23    srcset="image_sm.jpg 640w, image_md.jpg 1024w, image_lg.jpg 1920w"
24    type="image/jpeg">
25  <img src="image_lg.jpg" alt="Responsive image">
26</picture>
27EOF

Color space and format-specific optimizations:

bash
1# Convert RGB to indexed color (for PNG with limited palette)
2magick input.jpg -colors 256 -depth 8 output_indexed.png
3
4# Convert to CMYK (for print)
5magick input.jpg -colorspace CMYK -density 300x300 output_print.jpg
6
7# Strip all metadata for privacy/size reduction
8magick input.jpg -strip output_clean.jpg
9
10# Preserve EXIF data specifically
11magick input.jpg -strip -set 'comment' 'Preserved comment' output.jpg

Advanced filters and effects:

bash
1# Add blur effect
2magick input.jpg -blur 0x3 output_blurred.jpg
3
4# Sharpen image
5magick input.jpg -sharpen 0x1 output_sharp.jpg
6
7# Convert to grayscale
8magick input.jpg -colorspace Gray output_bw.jpg
9
10# Increase contrast
11magick input.jpg -contrast -contrast output_high_contrast.jpg
12
13# Watermark with transparency
14magick input.jpg -gravity SouthEast label.png -composite output_watermarked.jpg
15
16# Crop to aspect ratio (16:9)
17magick input.jpg -gravity center -extent 1920x1080 output_cropped.jpg

Quality comparison tools:

bash
1# Compare two images visually
2magick compare original.jpg converted.webp difference.jpg
3
4# Get detailed image properties
5magick identify -verbose input.jpg | grep -E "Geometry|Resolution|Class|Colorspace|Depth"
6
7# Calculate PSNR (Peak Signal-to-Noise Ratio) between images
8magick compare -metric PSNR original.jpg output.jpg psnr.txt
9
10# Check file size and calculate saved space
11original_size=$(stat -f%z original.jpg 2>/dev/null || stat -c%s original.jpg)
12converted_size=$(stat -f%z output.webp 2>/dev/null || stat -c%s output.webp)
13savings=$((100 * ($original_size - $converted_size) / $original_size))
14echo "File size reduced by ${savings}% ($original_size$converted_size bytes)"

Automated web optimization workflow:

bash
1#!/bin/bash
2# Complete image optimization pipeline
3
4optimize_image() {
5  local input="$1"
6  local base="${input%.*}"
7  
8  echo "Optimizing: $input"
9  
10  # Generate WebP (primary format)
11  magick "$input" -resize 1920x1080 -quality 85 -define webp:method=6 \
12    "${base}.webp"
13  
14  # Generate JPEG fallback
15  magick "$input" -resize 1920x1080 -quality 85 -strip \
16    "${base}.jpg"
17  
18  # Generate thumbnail
19  magick "$input" -resize 200x200 -quality 75 -define webp:method=6 -strip \
20    "${base}_thumb.webp"
21  
22  # Show results
23  echo "Results:"
24  ls -lh "${base}."* | awk '{print $9, $5}'
25}
26
27# Process all images in directory
28for img in *.{jpg,png,JPG,PNG}; do
29  [ -f "$img" ] && optimize_image "$img"
30done

Troubleshooting & Performance Tips

bash
1# Check ImageMagick policy restrictions
2magick -list policy
3
4# Increase memory limit for large images
5magick -limit memory 2GB input_huge.tif output.jpg
6
7# Use parallelization for batch operations
8find . -name "*.jpg" -print0 | xargs -0 -P 4 -I {} \
9  magick {} -quality 85 -strip {}_optimized.jpg
10
11# Profile conversion speed
12time magick input.jpg -resize 1920x1080 -quality 85 output.jpg

Document Conversion

LibreOffice and Pandoc handle document conversion reliably. LibreOffice excels at Microsoft Office formats, while Pandoc specializes in markup document conversion.

Supported Format Matrix

Source FormatTarget FormatsToolQualityNotes
DOCXPDF, ODT, HTML, TXT, RTFLibreOfficeExcellentPreserves formatting, tables, images
DOCPDF, DOCX, ODTLibreOfficeGoodOlder format, may lose some formatting
XLSXCSV, ODS, PDF, HTMLLibreOfficeExcellentPreserves formulas and formatting
PPTXPDF, ODP, HTML, PNGLibreOfficeGoodConverts slides to PDF or images
MarkdownDOCX, PDF, HTML, RTFPandocExcellentMarkdown-first workflow
HTMLDOCX, PDF, MarkdownPandocGoodStrips some styling
LaTeXPDF, HTML, DOCXPandoc + xelatexExcellentMathematics support
CSVXLSX, ODSLibreOfficeGoodGood for data interchange

LibreOffice Headless Conversion

LibreOffice in headless mode enables server-side document processing without GUI.

Basic conversions:

bash
1# Convert DOCX to PDF
2libreoffice --headless --convert-to pdf input.docx
3
4# Convert XLSX to CSV
5libreoffice --headless --convert-to csv input.xlsx
6
7# Convert PPTX to PDF
8libreoffice --headless --convert-to pdf input.pptx
9
10# Convert to ODT (OpenDocument)
11libreoffice --headless --convert-to odt input.docx
12
13# Specify output directory
14libreoffice --headless --convert-to pdf --outdir /path/to/output/ input.docx

Batch document conversion:

bash
1# Convert all DOCX files to PDF
2for file in *.docx; do
3  libreoffice --headless --convert-to pdf "$file"
4done
5
6# Convert all PPTX to PDF with progress
7total=$(ls *.pptx 2>/dev/null | wc -l)
8count=0
9for file in *.pptx; do
10  ((count++))
11  echo "Converting $count/$total: $file"
12  libreoffice --headless --convert-to pdf "$file"
13  # Optional: remove original
14  # rm "$file"
15done

Advanced LibreOffice options:

bash
1# Enable macro execution (security risk - be cautious)
2libreoffice --headless --convert-to pdf --norestore input.docx
3
4# Custom filter options (XLSX to CSV with specific delimiter)
5libreoffice --headless --convert-to csv:Text - csv -1 input.xlsx
6
7# Convert and apply style (Writer documents)
8libreoffice --headless --convert-to pdf --outdir /tmp/ \
9  --printer-name "Print to File" input.docx

Pandoc - Markup-First Document Conversion

Pandoc handles markdown, LaTeX, HTML, and more with excellent control over formatting.

Basic Pandoc conversions:

bash
1# Markdown to DOCX
2pandoc input.md -o output.docx
3
4# Markdown to PDF (requires xelatex or wkhtmltopdf)
5pandoc input.md -o output.pdf --pdf-engine=xelatex
6
7# DOCX to Markdown (extract from Word)
8pandoc input.docx -o output.md
9
10# HTML to Markdown
11pandoc input.html -o output.md
12
13# PowerPoint to Markdown
14pandoc input.pptx -o output.md

PDF generation with Pandoc:

bash
1# Basic PDF from Markdown (using xelatex for LaTeX support)
2pandoc input.md -o output.pdf --pdf-engine=xelatex
3
4# PDF with custom styling and fonts
5pandoc input.md -o output.pdf \
6  --pdf-engine=xelatex \
7  --variable fontsize=12pt \
8  --variable geometry:margin=1in
9
10# PDF with table of contents
11pandoc input.md -o output.pdf --toc --pdf-engine=xelatex
12
13# PDF with metadata
14pandoc input.md -o output.pdf \
15  -M author="Author Name" \
16  -M title="Document Title" \
17  -M date="$(date)" \
18  --pdf-engine=xelatex

Advanced format preservation:

bash
1# DOCX to PDF with maximum formatting preservation
2pandoc input.docx -o output.pdf --pdf-engine=xelatex \
3  --variable linkcolor=blue \
4  --variable geometry:margin=1in
5
6# HTML to DOCX with embedded styles
7pandoc input.html -o output.docx --embed-resources --self-contained
8
9# Markdown to DOCX with custom reference document
10pandoc input.md -o output.docx --reference-doc=custom_style.docx
11
12# DOCX to clean HTML (strip styles)
13pandoc input.docx -o output.html -t html5 --strip-comments
14
15# DOCX to HTML with CSS preservation
16pandoc input.docx -o output.html --css=styles.css

Batch conversion with Pandoc:

bash
1#!/bin/bash
2# Convert all markdown files to DOCX with metadata
3
4for md_file in *.md; do
5  docx_file="${md_file%.md}.docx"
6  echo "Converting: $md_file$docx_file"
7  
8  pandoc "$md_file" -o "$docx_file" \
9    -M author="Your Organization" \
10    --reference-doc=template.docx
11done

Extract text from complex documents:

bash
1# Extract text from DOCX
2pandoc input.docx -t plain -o output.txt
3
4# Extract text from PDF (requires pdftotext)
5pdftotext input.pdf output.txt
6
7# Extract structured content as Markdown
8pandoc input.docx -t markdown -o output.md

LibreOffice vs. Pandoc - Decision Matrix

ScenarioRecommended ToolReason
DOCX/XLSX/PPTX → PDFLibreOfficePreserves Office formatting perfectly
Markdown → DOCX/PDFPandocMarkdown-first workflow, consistent output
DOCX → Markdown text extractionPandocClean text extraction
Batch Office conversionLibreOfficeFaster for multiple Office files
Complex PDF with embedded fontsPandoc + xelatexBetter LaTeX/font control
Server-side automationLibreOfficeStable for production
Scientific documents/mathPandoc + LaTeXLaTeX math support
Heavy table/chart preservationLibreOfficeBetter table formatting

Quality Considerations

Formatting preservation issues:

  • Complex tables: LibreOffice generally better
  • Embedded graphics: Both tools preserve, check output
  • Header/footer: May not convert perfectly
  • Complex styles: Can be lost in markdown conversions
  • Track changes: Typically not preserved in conversion

Troubleshooting common issues:

bash
1# If fonts are missing in PDF
2libreoffice --headless --convert-to pdf --outdir /tmp/ input.docx
3# Check if output is readable; if not, try embedding fonts
4
5# For better encoding support
6LANG=en_US.UTF-8 pandoc input.md -o output.pdf
7
8# Verify conversion success
9file output.pdf  # Should show: PDF document
10pdfinfo output.pdf | head -5  # Show basic info
11
12# Compare file sizes before and after
13echo "Original: $(du -h input.docx | cut -f1)"
14echo "Converted: $(du -h output.pdf | cut -f1)"

Integration with automation pipelines

bash
1#!/bin/bash
2# Complete document pipeline: DOCX → Multiple formats
3
4convert_document() {
5  local input="$1"
6  local base="${input%.*}"
7  
8  echo "Converting: $input"
9  
10  # PDF (primary distribution)
11  libreoffice --headless --convert-to pdf "$input"
12  
13  # Markdown (archiving, version control)
14  pandoc "$input" -o "${base}.md"
15  
16  # HTML (web viewing)
17  pandoc "$input" -o "${base}.html" -t html5
18  
19  # Text (indexing, search)
20  pandoc "$input" -o "${base}.txt" -t plain
21  
22  echo "Completed conversions:"
23  ls -lh "${base}."*
24}
25
26# Process document
27convert_document "my_document.docx"

PDF Operations

PDFs are universal, but they can be large and complex. Multiple tools exist for compression, manipulation, security, and optimization.

PDF Tool Comparison

ToolPurposeStrengthLimitation
GhostscriptCompression, conversionBest compression levelsComplex options
QPDFMerging, splitting, linearizationFull PDF manipulationLower compression than Ghostscript
Poppler (pdfunite, pdfseparate, pdftotext)Extraction, merging, textReliable, simple commandsLimited compression
ImageMagickPDF to images, simple editingEasy image extractionRasterizes content
pdftkPage manipulation, rotationSimple syntaxDeprecated in newer versions

PDF Compression & Optimization

Ghostscript compression (best compression ratio):

bash
1# Maximum compression for ebook-style distribution
2gs -sDEVICE=pdfwrite \
3   -dPDFSETTINGS=/ebook \
4   -dCompatibilityLevel=1.4 \
5   -dNOPAUSE -dQUIET -dBATCH \
6   -sOutputFile=compressed.pdf \
7   input.pdf
8
9# Settings comparison:
10# /screen    - Lowest quality, smallest size (for screen viewing)
11# /ebook     - Low-medium quality, small size (typical compression)
12# /printer   - High quality, medium size (print-ready)
13# /prepress  - Maximum quality, largest size (publication)

QPDF linearization (web-optimized PDFs):

bash
1# Linearize for web streaming (fast first-page viewing)
2qpdf --linearize input.pdf output_web.pdf
3
4# Compress while linearizing
5qpdf --linearize --compress-streams=y input.pdf output_optimized.pdf
6
7# Remove unused objects (cleanup)
8qpdf --compress-streams=y --object-streams=generate input.pdf output_clean.pdf

Combine compression strategies:

bash
1# First compress with Ghostscript, then optimize with QPDF
2gs -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook \
3   -sOutputFile=temp.pdf input.pdf && \
4qpdf --linearize temp.pdf output_final.pdf && \
5rm temp.pdf

PDF Merging and Splitting

Merge multiple PDFs:

bash
1# Simple concatenation (preserves page order)
2pdfunite page1.pdf page2.pdf page3.pdf merged.pdf
3
4# Merge with compression
5gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER \
6   -sOutputFile=merged.pdf \
7   page1.pdf page2.pdf page3.pdf
8
9# Merge with ordering
10qpdf --empty --pages file1.pdf 1-3 file2.pdf 1-5 -- output.pdf

Split PDFs:

bash
1# Extract specific page range (pages 1-5)
2pdfseparate -f 1 -l 5 input.pdf output_page_%d.pdf
3
4# Split every N pages (useful for large documents)
5gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER \
6   -dFirstPage=1 -dLastPage=10 -sOutputFile=part1.pdf input.pdf
7
8# Extract single page
9qpdf input.pdf --pages . 5 -- page5.pdf
10
11# Split into individual pages
12pdfseparate input.pdf page_%d.pdf

PDF Security & Encryption

Add password protection:

bash
1# Encrypt with owner password (restrict printing, copying)
2qpdf --encrypt owner_password user_password 40 -- input.pdf output_secure.pdf
3
4# Strong 256-bit encryption (newer standard)
5qpdf --encrypt user_password owner_password 256 -- input.pdf output_secure.pdf
6
7# Restrict permissions (no printing, no copying, no modification)
8qpdf --encrypt user_pass owner_pass 256 \
9   --restrictions=-print,-copy,-modify -- input.pdf restricted.pdf

Remove passwords:

bash
1# Remove user password (if unencrypted for content)
2qpdf --password=user_password --empty --pages input.pdf -- output_unencrypted.pdf
3
4# Decrypt and re-encrypt with different permissions
5qpdf --password=old_pass input.pdf --encrypt new_pass user_pass 256 -- output.pdf

PDF Rotation & Reordering

Rotate pages:

bash
1# Rotate all pages 90 degrees clockwise
2pdftk input.pdf cat 1-endsouth output rotated.pdf
3
4# Rotate specific pages
5pdftk input.pdf cat 1-5south 6-end output rotated_custom.pdf
6# South = 90° clockwise, North = 90° counter-clockwise
7
8# Using QPDF (more modern)
9qpdf input.pdf --rotate=90 -- output.pdf

Reorder pages:

bash
1# Reverse page order
2pdftk input.pdf cat end-1south output reversed.pdf
3
4# Extract and rearrange (pages 5,3,1,2,4)
5pdftk input.pdf cat 5 3 1 2 4 output reordered.pdf
6
7# Extract front and back pages separately, then recombine
8pdftk input.pdf cat odd-page-number-down even-page-number-down output two_sided.pdf

PDF Text Extraction & Analysis

Extract text:

bash
1# Simple text extraction
2pdftotext input.pdf output.txt
3
4# Preserve layout
5pdftotext -layout input.pdf output.txt
6
7# Extract with coordinates (useful for analysis)
8pdftotext -bbox input.pdf output.txt
9
10# Extract from specific pages
11pdftotext -f 1 -l 5 input.pdf output.txt

Search and analyze:

bash
1# Search for text in PDF
2pdftotext input.pdf - | grep -i "search term"
3
4# Count pages
5pdfinfo input.pdf | grep Pages
6
7# Get metadata
8pdfinfo input.pdf
9
10# Extract document properties
11exiftool input.pdf | grep -E "Title|Author|Creator|CreationDate"

PDF Repair & Validation

Check PDF integrity:

bash
1# Validate PDF structure
2qpdf --check input.pdf
3
4# Get detailed validation output
5pdfinfo input.pdf 2>&1 | head -20
6
7# Check for corruption
8file input.pdf

Repair broken PDFs:

bash
1# Attempt repair with Ghostscript
2gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER \
3   -sOutputFile=repaired.pdf input.pdf
4
5# Alternative: use QPDF to reconstruct
6qpdf --deterministic-id input.pdf repaired.pdf
7
8# Validate after repair
9qpdf --check repaired.pdf

PDF Watermarking & Annotation

Add watermark:

bash
1# Add text watermark using GhostScript
2gs -sDEVICE=pdfwrite -q -dNOPAUSE -dBATCH -dSAFER \
3   -sOutputFile=watermarked.pdf input.pdf
4
5# Using ImageMagick (rasterizes to images first)
6magick -density 150 input.pdf -pointsize 48 -fill rgba(255,0,0,0.1) \
7   -gravity center -annotate 0 'DRAFT' -density 150 watermarked.pdf

Extract or remove annotations:

bash
1# Remove annotations/markup with QPDF
2qpdf input.pdf --coalesce-contents --empty -- output_clean.pdf
3
4# List annotations (if pdftotext supports it)
5pdftotext -enc UTF-8 input.pdf - | grep -i "annotation"

Batch PDF Optimization Workflow

bash
1#!/bin/bash
2# Complete PDF optimization pipeline
3
4optimize_pdf() {
5  local input="$1"
6  local base="${input%.*}"
7  
8  echo "Optimizing: $input"
9  
10  # First pass: Ghostscript compression
11  gs -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook \
12     -dNOPAUSE -dQUIET -dBATCH \
13     -sOutputFile="${base}_compressed.pdf" "$input"
14  
15  # Second pass: QPDF linearization
16  qpdf --linearize "${base}_compressed.pdf" "${base}_optimized.pdf"
17  
18  # Show results
19  echo "Results:"
20  echo "Original: $(du -h $input | cut -f1)"
21  echo "Optimized: $(du -h ${base}_optimized.pdf | cut -f1)"
22  
23  # Optional: replace original with optimized
24  # mv "${base}_optimized.pdf" "$input"
25}
26
27# Process all PDFs in directory
28for pdf in *.pdf; do
29  optimize_pdf "$pdf"
30done

PDF Accessibility & Compliance

bash
1# Tag PDF for accessibility (using existing commercial tools recommended)
2# Alternative: use OCR to make searchable (see OCR section)
3
4# Ensure text is selectable (not image-only)
5# Check if text layer exists:
6pdftotext input.pdf - | head -10
7
8# If empty, PDF is image-only (needs OCR - see OCR section)

OCR (Optical Character Recognition)

Convert scanned documents and images to searchable, editable text. OCR quality depends on image quality, resolution, and language configuration.

OCR Tool Comparison

ToolAccuracyLanguagesSpeedSupported FormatsBest For
TesseractGood (80-95%)100+FastImage files, PDFBatch processing, CLI usage
OCRmyPDFExcellentMultipleMediumPDFPDF-specific, preserves layout
Google Cloud VisionExcellent (95%+)50+FastImages, PDFsHigh accuracy needed, API-based
Amazon TextractExcellentMultipleMediumImages, PDFsAWS integration, documents
EasyOCRGood-Excellent80+MediumCommon image formatsModern deep learning approach

Tesseract Basics

Simple OCR:

bash
1# Extract text from image
2tesseract input.png output.txt
3
4# Specify language (default: English)
5tesseract input.png output.txt -l eng
6
7# Output multiple formats
8tesseract input.png output pdf txt hocr
9  # Creates: output.pdf, output.txt, output.hocr

Available languages:

bash
1# List installed languages
2tesseract --list-langs
3
4# Install additional languages (Ubuntu example)
5sudo apt install tesseract-ocr-fra  # French
6sudo apt install tesseract-ocr-deu  # German
7sudo apt install tesseract-ocr-chi-sim  # Simplified Chinese

Multilingual OCR:

bash
1# Mix multiple languages (e.g., English + French)
2tesseract input.png output.txt -l eng+fra
3
4# For mixed scripts
5tesseract input.png output.txt -l eng+ara+chi-sim

OCRmyPDF - PDF Specialist

OCRmyPDF adds OCR to PDFs while preserving the original layout, making it searchable.

Basic usage:

bash
1# Make PDF searchable while preserving layout
2ocrmypdf input.pdf output_searchable.pdf
3
4# Deskew and fix rotation automatically
5ocrmypdf --deskew --rotate-pages auto input.pdf output.pdf
6
7# Specify language
8ocrmypdf -l fra input.pdf output.pdf  # French
9
10# Multiple languages
11ocrmypdf -l eng+fra input.pdf output.pdf

Advanced OCRmyPDF options:

bash
1# Complete optimization pipeline
2ocrmypdf --deskew \
3         --rotate-pages auto \
4         --remove-background \
5         --optimize 3 \
6         -l eng+fra \
7         input.pdf output_optimized.pdf
8
9# Force OCR even if text already exists (re-OCR)
10ocrmypdf --force-ocr input.pdf output.pdf
11
12# Keep intermediate files for debugging
13ocrmypdf --keep-temporary-files input.pdf output.pdf
14
15# Specify output PDF compression
16ocrmypdf --output-type pdf --compress-streams=y input.pdf output.pdf
17
18# Generate searchable with image-only fallback for poor quality areas
19ocrmypdf --image-dpi 200 --rotate-pages=skip input.pdf output.pdf

Image Preprocessing for Better OCR Accuracy

High-quality images dramatically improve OCR accuracy.

Preprocess with ImageMagick:

bash
1# Basic enhancement (increase contrast, sharpen)
2magick input.png -contrast-stretch 0 -sharpen 0x1 enhanced.png
3
4# Convert to high-contrast black & white
5magick input.png -threshold 40% bw.png
6
7# Increase DPI/resolution for small text
8magick input.png -resize 200% upscaled.png
9
10# Combine preprocessing + OCR
11magick input.png -contrast-stretch 0 -sharpen 0x1 preprocessed.png && \
12tesseract preprocessed.png output.txt

Batch preprocess images:

bash
1#!/bin/bash
2# Preprocess all images before OCR
3
4for img in *.png; do
5  echo "Processing: $img"
6  
7  # Enhance with contrast and sharpening
8  magick "$img" -contrast-stretch 0 -sharpen 0x1 "${img%.*}_enhanced.png"
9  
10  # Run OCR on enhanced image
11  tesseract "${img%.*}_enhanced.png" "${img%.*}_ocr.txt"
12done

OCR Accuracy Optimization

Quality metrics and testing:

bash
1# Compare OCR accuracy (manual verification needed)
2# 1. Run OCR on test image
3tesseract test.png output.txt
4
5# 2. Manually review output.txt for errors
6# 3. Adjust preprocessing and retry
7
8# For PDF OCR - check confidence
9ocrmypdf --pdf-renderer hocr input.pdf output_hocr.pdf
10# Extract confidence scores from HOCR output

Improve accuracy:

bash
1# High-DPI preprocessing + OCR
2magick input.png -density 300 -depth 8 enhanced_300dpi.png && \
3tesseract enhanced_300dpi.png output.txt --psm 6
4
5# PSM (Page Segmentation Mode) options:
6# 0 = Auto
7# 3 = Fully automatic page segmentation (default)
8# 6 = Assume single uniform block of text
9# 11 = Sparse text; find as much text as possible
10# 13 = Raw line; treat image as a single text line

Batch OCR with quality control:

bash
1#!/bin/bash
2# Batch OCR with preprocessing and validation
3
4for pdf in *.pdf; do
5  base="${pdf%.*}"
6  
7  echo "Processing: $pdf"
8  
9  # Add OCR with auto-rotate
10  ocrmypdf --deskew \
11           --rotate-pages auto \
12           --remove-background \
13           -l eng \
14           "$pdf" "${base}_ocr.pdf"
15  
16  # Validate output is searchable
17  if pdftotext "${base}_ocr.pdf" - | wc -w | grep -q "[0-9]"; then
18    echo "✓ OCR successful for $base"
19    # Optional: remove original
20    # rm "$pdf"
21  else
22    echo "✗ OCR failed for $base"
23  fi
24done

Scanned Document Workflow

Complete pipeline for digitizing paper documents:

bash
1#!/bin/bash
2# Complete scanned document pipeline
3
4process_scanned_document() {
5  local input="$1"
6  local base="${input%.*}"
7  
8  echo "Processing scanned document: $input"
9  
10  # Step 1: Preprocess image (increase DPI, enhance contrast)
11  echo "Preprocessing..."
12  magick "$input" \
13    -density 300 \
14    -contrast-stretch 0 \
15    -sharpen 0x1 \
16    "${base}_preprocessed.png"
17  
18  # Step 2: Run OCR
19  echo "Running OCR..."
20  tesseract "${base}_preprocessed.png" "${base}_text.txt" \
21    -l eng --psm 6
22  
23  # Step 3: Create searchable PDF if original is image
24  if [[ "$input" == *.png ]] || [[ "$input" == *.jpg ]]; then
25    magick "${base}_preprocessed.png" "${base}_scanned.pdf"
26    ocrmypdf "${base}_scanned.pdf" "${base}_searchable.pdf"
27  fi
28  
29  echo "Output files:"
30  ls -lh "${base}"_*.{txt,pdf} 2>/dev/null
31}
32
33# Process document
34process_scanned_document "scanned_page.png"

Handling Multiple Pages

Convert multi-page PDF to images, then OCR:

bash
1# Extract each page as image
2pdfimages input.pdf page_%d.png
3
4# Run OCR on each page
5for img in page_*.png; do
6  tesseract "$img" "${img%.*}_ocr.txt"
7done
8
9# Combine results
10cat page_*_ocr.txt > complete_document.txt

OCRmyPDF for multi-page PDFs:

bash
1# Process entire multi-page PDF at once
2ocrmypdf --language eng \
3         --deskew \
4         --rotate-pages auto \
5         input.pdf output_searchable.pdf

Advanced OCR: Cloud-Based Options

For production systems needing higher accuracy:

bash
1# Google Cloud Vision OCR (requires gcloud CLI and credentials)
2gcloud vision document-text-detect input.png > ocr_result.json
3
4# Extract text from response
5jq '.responses[0].fullTextAnnotation.text' ocr_result.json
6
7# AWS Textract (requires AWS CLI)
8aws textract detect-document-text --document S3Object=\{Bucket=my-bucket,Name=input.pdf\} > result.json
9
10# Extract text
11jq '.Blocks[] | select(.BlockType=="LINE") | .Text' result.json

Troubleshooting OCR Issues

bash
1# Poor OCR results - check image quality
2identify -verbose input.png | grep Resolution
3
4# Insufficient resolution - upscale first
5magick input.png -density 150 -resample 300 upscaled.png && \
6tesseract upscaled.png output.txt
7
8# Mixed languages causing errors - specify explicitly
9tesseract input.png output.txt -l eng+fra
10
11# PDF appears text-based but has no OCR - check with pdftotext
12pdftotext input.pdf - | wc -w  # Returns 0 if no text layer
13
14# Re-OCR existing PDF
15ocrmypdf --force-ocr input.pdf output_reocred.pdf

eBook Conversion

Calibre's ebook-convert tool is the industry standard for converting between ebook formats while preserving metadata and formatting.

eBook Format Comparison

FormatContainerDevice SupportQualityMetadataBest For
EPUBZip archiveMost readers (Kobo, Apple, Android)GoodExcellentUniversal ebook standard
EPUB3Zip archiveModern readersExcellentExcellentModern features, fixed layout
MOBIProprietaryKindle (older)GoodLimitedOlder Kindles (deprecated)
AZW3ProprietaryKindle (modern)ExcellentGoodModern Kindle devices
PDFBinaryUniversalExcellentVariableFixed layout, archiving
HTMLTextWeb browsersGoodLimitedWeb viewing
TXTTextAll devicesPoorNonePlain text

Basic eBook Conversions

Standard conversions:

bash
1# EPUB to PDF (for printing)
2ebook-convert input.epub output.pdf
3
4# PDF to EPUB
5ebook-convert input.pdf output.epub
6
7# EPUB to Kindle format
8ebook-convert input.epub output.azw3
9
10# MOBI (legacy) to AZW3
11ebook-convert input.mobi output.azw3

Preserve metadata during conversion:

bash
1# View metadata before conversion
2ebook-meta input.epub
3
4# Convert while preserving metadata
5ebook-convert input.epub output.pdf --preserve-cover-aspect-ratio
6
7# Set metadata directly
8ebook-meta input.epub --title="New Title" --author="Author Name"

Advanced eBook Conversion

Optimize for specific devices:

bash
1# Kindle-optimized conversion (AZW3)
2ebook-convert input.epub output.azw3 \
3  --paper-size a6 \
4  --margin-left 16 --margin-right 16 \
5  --margin-top 16 --margin-bottom 16 \
6  --font-size-mapping table:20,
7
8 monospace:20
9
10# Kobo-optimized EPUB
11ebook-convert input.pdf output.epub \
12  --no-default-epub-cover \
13  --paper-size a6 \
14  --margin-left 10 --margin-right 10
15
16# Apple Books (iBooks) format
17ebook-convert input.pdf output.epub \
18  --enable-heuristics \
19  --change-justification=left

PDF to EPUB with enhanced heuristics:

bash
1# Use heuristics to improve layout detection
2ebook-convert input.pdf output.epub \
3  --enable-heuristics \
4  --paper-size a4 \
5  --margin-left 20 --margin-right 20 \
6  --margin-top 20 --margin-bottom 20

Extract from files and convert:

bash
1# Convert from HTML/web content to EPUB
2ebook-convert input.html output.epub --paper-size a5
3
4# Create EPUB from markdown
5ebook-convert input.md output.epub
6
7# Convert and add cover image
8ebook-convert input.txt output.epub --cover=cover.jpg

Batch eBook Conversion

bash
1#!/bin/bash
2# Batch convert all eBooks to multiple formats
3
4for input in *.epub; do
5  base="${input%.*}"
6  
7  echo "Converting: $input"
8  
9  # PDF for printing
10  ebook-convert "$input" "${base}.pdf" --paper-size a4
11  
12  # AZW3 for Kindle
13  ebook-convert "$input" "${base}.azw3"
14  
15  # MOBI for older Kindles
16  ebook-convert "$input" "${base}.mobi"
17  
18  echo "Created: ${base}.pdf, ${base}.azw3, ${base}.mobi"
19done

eBook Quality & Validation

bash
1# Validate EPUB structure
2ebook-convert input.epub output.epub --check-only
3
4# Extract text from eBook
5ebook-convert input.epub output.txt
6
7# Check for common issues
8ebook-meta input.epub | grep -E "Title|Author|Series"
9
10# Compare file sizes
11ls -lh *.epub *.pdf *.azw3 | sort -k5 -h

Archives and Compression

Archive and compress files for distribution, backup, and storage optimization. Different algorithms provide various compression/speed trade-offs.

Compression Algorithm Comparison

FormatAlgorithmCompressionSpeedCompatibilityEncryptionBest For
ZIPDeflateMedium (50-70%)Very FastUniversalOptionalDocument sharing, archives
7zLZMA2Excellent (70-85%)SlowLimitedYesMaximum compression, archiving
TAR+GZIPLZ77Good (50-70%)FastUnix/LinuxNo (separate)Unix/Linux systems
TAR+BZIP2Burrows-WheelerVery Good (60-80%)MediumUnix/LinuxNo (separate)Better compression than gzip
TAR+XZLZMAExcellent (70-85%)Very SlowLimitedNo (separate)Best compression for archives
RARProprietaryGood (50-75%)FastLimitedYesLegacy systems, multi-volume
GZIPDeflateMedium (50-70%)Very FastUniversalNoSingle file compression
BZIP2Burrows-WheelerGood (60-80%)MediumGoodNoBetter than gzip for large files

ZIP Archives

Create ZIP archives:

bash
1# Basic ZIP
2zip -r archive.zip folder/file1.txt file2.jpg
3
4# ZIP with compression level (9=maximum)
5zip -r -9 archive.zip folder/
6
7# ZIP excluding certain files
8zip -r archive.zip folder/ -x "*.log" "*.tmp"
9
10# Encrypt ZIP with password
11zip -e -r archive_secure.zip folder/
12
13# ZIP with better compression (store vs. deflate)
14zip -r -Z bzip2 archive.zip folder/  # Note: not all systems support bzip2 in ZIP

Extract ZIP:

bash
1# Basic extraction
2unzip archive.zip
3
4# Extract to specific directory
5unzip archive.zip -d /path/to/extract/
6
7# List ZIP contents without extracting
8unzip -l archive.zip
9
10# Extract specific files
11unzip archive.zip "path/to/file.txt"

7-Zip (Best Compression)

Create 7z archives:

bash
1# Maximum compression
27z a -t7z -m0=lzma2 -mx=9 -mfb=64 -md=32m -ms=on archive.7z folder/
3
4# Fast compression
57z a -t7z -m0=lzma2 -mx=5 archive.7z folder/
6
7# Encrypt with password
87z a -t7z -p archive_secure.7z folder/
9
10# Split into volumes (for storage/distribution)
117z a -t7z -v1g archive.7z folder/  # Creates 1GB segments

Extract 7z:

bash
1# Basic extraction
27z x archive.7z
3
4# Extract to directory
57z x archive.7z -o/path/to/extract/
6
7# List contents
87z l archive.7z
9
10# Test archive integrity
117z t archive.7z

TAR with Compression

Create TAR archives:

bash
1# TAR + GZIP (fastest, good for web)
2tar czf archive.tar.gz folder/
3
4# TAR + BZIP2 (slower, better compression)
5tar cjf archive.tar.bz2 folder/
6
7# TAR + XZ (slowest, best compression)
8tar cJf archive.tar.xz folder/
9
10# TAR without compression (for piping)
11tar cf archive.tar folder/
12
13# Exclude files
14tar czf archive.tar.gz --exclude="*.log" --exclude=".DS_Store" folder/
15
16# Create with specific ownership/permissions
17tar czf archive.tar.gz --owner=0 --group=0 --mode=755 folder/

Extract TAR archives:

bash
1# Auto-detect compression and extract
2tar xf archive.tar.gz  # or .tar.bz2, .tar.xz
3
4# Extract to directory
5tar -C /path/to/extract/ -xzf archive.tar.gz
6
7# List contents
8tar tzf archive.tar.gz
9
10# Extract specific file
11tar xzf archive.tar.gz path/to/file.txt

Batch Archiving

bash
1#!/bin/bash
2# Archive multiple directories with compression levels
3
4archive_folder() {
5  local folder="$1"
6  local date=$(date +%Y%m%d_%H%M%S)
7  
8  echo "Archiving: $folder"
9  
10  # Create compressed archive with timestamp
11  tar czf "${folder}_${date}.tar.gz" "$folder"
12  
13  # Show file sizes
14  echo "Original size: $(du -sh $folder | cut -f1)"
15  echo "Archive size: $(du -sh ${folder}_${date}.tar.gz | cut -f1)"
16  
17  # Optional: verify archive
18  tar tzf "${folder}_${date}.tar.gz" > /dev/null && echo "✓ Archive verified"
19}
20
21# Archive each directory
22for dir in project1 project2 project3; do
23  archive_folder "$dir"
24done

Archive Encryption & Security

bash
1# Encrypt TAR with GPG
2tar czf - folder/ | gpg --encrypt --output archive.tar.gz.gpg
3
4# Decrypt and extract
5gpg --decrypt archive.tar.gz.gpg | tar xz
6
7# Password-protected ZIP
8zip -e -r archive.zip folder/
9
10# Password-protected 7z
117z a -p archive.7z folder/

Archive Integrity & Verification

bash
1# Test ZIP archive
2unzip -t archive.zip
3
4# Test 7z archive
57z t archive.7z
6
7# Test TAR archive
8tar tzf archive.tar.gz > /dev/null
9
10# Calculate and verify checksums
11sha256sum archive.tar.gz > archive.tar.gz.sha256
12sha256sum -c archive.tar.gz.sha256  # Verify later

Metadata Management

Metadata contains information about files (photos, documents) like creation date, author, location, and copyright. ExifTool is the universal solution for reading and writing metadata across formats.

Metadata Types

Metadata TypeUsed InExamplesPrivacy Risk
EXIFImage files (JPEG, TIFF)Camera model, GPS location, timestampsHIGH (location tracking)
IPTCImages, DocumentsKeywords, copyright, creatorMEDIUM
XMPImages, PDF, DocumentsCustom metadata, color profilesMEDIUM
METADATA (PDF)PDF documentsAuthor, title, creation dateMEDIUM
ID3Audio files (MP3, FLAC)Artist, album, track numberLOW (non-sensitive info)
Vorbis CommentsOGG, FLACArtist, album, track informationLOW

Reading Metadata

View all metadata:

bash
1# All metadata for image
2exiftool image.jpg
3
4# Detailed, verbose output
5exiftool -a -G1 image.jpg  # Group by metadata type
6
7# Show only specific type
8exiftool -EXIF image.jpg
9exiftool -IPTC image.jpg
10exiftool -XMP image.jpg

Extract specific metadata fields:

bash
1# Camera information
2exiftool -model -make -datetimeoriginal image.jpg
3
4# GPS location (if available)
5exiftool -gps* image.jpg
6
7# Copyright and author
8exiftool -creator -copyright image.jpg
9
10# Extract from PDF
11exiftool -Title -Author -Subject document.pdf
12
13# Extract and format as CSV
14exiftool -csv -filename -model -datetimeoriginal *.jpg

Writing Metadata

Add or modify metadata:

bash
1# Add copyright notice
2exiftool -Copyright="© 2026 My Company" image.jpg
3
4# Set creator/artist
5exiftool -Creator="Photographer Name" image.jpg
6exiftool -Artist="Artist Name" -Album="Album Name" song.mp3
7
8# Add keywords (searchable metadata)
9exiftool -Keywords="travel,sunset,landscape" image.jpg
10
11# Add GPS coordinates
12exiftool -GPSLatitude=40.7128 -GPSLongitude=-74.0060 image.jpg
13
14# Date-stamp image from file modification time
15exiftool -datetimeoriginal\<filemodifydate image.jpg

Batch update metadata:

bash
1#!/bin/bash
2# Add copyright to all images
3
4for image in *.jpg; do
5  echo "Processing: $image"
6  exiftool -overwrite_original \
7    -Copyright="© 2026 My Company" \
8    -Creator="Author Name" \
9    "$image"
10done

Remove Metadata (Privacy)

Remove all metadata:

bash
1# Remove all EXIF data (dangerous photos)
2exiftool -all= image.jpg  # Creates backup (.jpg_original)
3
4# Remove without backup
5exiftool -all= -overwrite_original image.jpg
6
7# Remove specific sensitive data
8exiftool -GPS* -datetimeoriginal= image.jpg
9
10# For PDF documents
11exiftool -all= document.pdf

Batch remove sensitive metadata:

bash
1#!/bin/bash
2# Remove EXIF from all images (batch)
3
4for image in *.jpg *.png; do
5  exiftool -all= -overwrite_original "$image" && \
6    echo "Cleaned: $image"
7done

Remove specific metadata types:

bash
1# Remove GPS location only
2exiftool -gps*= image.jpg
3
4# Remove all dates
5exiftool -datetimeoriginal= -createdate= -modifydate= image.jpg
6
7# Remove camera information
8exiftool -model= -make= -lensmodel= image.jpg

Metadata for Different File Types

Audio files (ID3 tags):

bash
1# View ID3 metadata
2exiftool -ID3:* song.mp3
3
4# Add ID3 v2.4 tags
5exiftool -Artist="Artist Name" \
6         -Album="Album Title" \
7         -Title="Track Name" \
8         -Track=1 \
9         -AlbumArtist "Album Artist" \
10         song.mp3

PDF documents:

bash
1# View PDF metadata
2exiftool document.pdf
3
4# Set PDF metadata
5exiftool -Title="Document Title" \
6         -Author="Author Name" \
7         -Subject="Document Subject" \
8         -Keywords="keyword1, keyword2" \
9         document.pdf

Video files:

bash
1# FFmpeg can edit video metadata too
2ffmpeg -i input.mp4 \
3       -metadata title="Video Title" \
4       -metadata artist="Creator Name" \
5       -metadata date="2026-01-01" \
6       -codec copy output.mp4

Workflow: Batch Metadata Processing

bash
1#!/bin/bash
2# Complete metadata processing pipeline
3
4process_photo_batch() {
5  local source_dir="$1"
6  local copyright="$2"
7  
8  echo "Processing photos in: $source_dir"
9  
10  # Count files
11  count=$(find "$source_dir" -name "*.jpg" -o -name "*.png" | wc -l)
12  echo "Found $count images"
13  
14  # Get date from first image
15  date=$(exiftool -datetimeoriginal -s -s -s "$source_dir"/*.jpg | head -1)
16  echo "Image date: $date"
17  
18  # Add consistent metadata
19  for image in "$source_dir"/*.{jpg,png}; do
20    [ -f "$image" ] || continue
21    
22    # Preserve original, add metadata
23    exiftool -overwrite_original \
24      -Copyright="$copyright" \
25      -Creator="photographer" \
26      -Keywords="batch,archive" \
27      "$image" && echo "✓ $image"
28  done
29}
30
31# Usage
32process_photo_batch "./raw_photos" "© 2026 My Company"

Metadata Validation & Cleanup

bash
1# Find images with problematic metadata
2for image in *.jpg; do
3  size=$(exiftool -filesize -s -s -s "$image")
4  if [ ${size%MB} -gt 5 ]; then  # > 5MB potentially has large metadata
5    echo "Large metadata in: $image"
6    exiftool -a "$image" | head -20
7  fi
8done
9
10# Validate metadata consistency
11exiftool -csv -filename -model -creator *.jpg > metadata_report.csv
12
13# Strip and rebuild metadata (clean slate)
14for image in *.jpg; do
15  exiftool -all= "$image" && \
16  exiftool -Creator="Photographer" -Copyright="© 2026" "$image"
17done

Fonts and Typography

Convert and optimize fonts for web, print, and various platforms. Font subsetting reduces file size significantly for web delivery.

Font Format Comparison

FormatExtensionPlatform SupportFile SizeCompressionBest For
TrueType.ttfUniversal (desktop)MediumVariableDesktop applications, printing
OpenType.otfUniversal (desktop)MediumVariableProfessional typography, advanced features
Web Open Font.woffAll modern browsersSmallGzip compressedWeb delivery (primary format)
Web Open Font 2.woff2Modern browsersVery SmallSuperiorModern web (recommended)
Embedded OpenType.eotLegacy IESmallMicrosoft formatLegacy Windows IE support
SVG Font.svgOlder mobile browsersVariableVector-basedLegacy mobile, fallback

Font Conversion with FontForge

Basic font conversion:

bash
1# TTF to OTF
2fontforge -c 'Open($1); Generate($2);' input.ttf output.otf
3
4# TTF to WOFF (web-optimized)
5fontforge -c 'Open($1); Generate($2);' input.ttf output.woff
6
7# TTF to WOFF2 (best compression)
8fontforge -c 'Open($1); Generate($2);' input.ttf output.woff2
9
10# OTF to TTF
11fontforge -c 'Open($1); Generate($2);' input.otf output.ttf

Batch font conversion:

bash
1#!/bin/bash
2# Convert all TTF fonts to WOFF2
3
4for font in *.ttf; do
5  base="${font%.*}"
6  echo "Converting: $font"
7  fontforge -c "Open('$font'); Generate('${base}.woff2');"
8done

Web Font Optimization

Create web font stacks with subsetting:

bash
1# Using sfnt2woff-zopfli (maximum compression)
2# Install: npm install -g sfnt2woff-zopfli
3
4for font in *.ttf; do
5  base="${font%.*}"
6  
7  # Convert to WOFF (modern browsers)
8  fontforge -c "Open('$font'); Generate('${base}.woff');"
9  
10  # Create WOFF2 for cutting-edge browsers
11  sfnt2woff-zopfli "$font" -o "${base}.woff2"
12done

Font subsetting (reduce file size):

bash
1# Using FontTools (Python-based, highly recommended)
2# pip install fonttools
3
4# Subset for Latin characters only
5pyftsubset input.ttf --unicodes="U+0-U+17F"
6
7# Subset for specific text
8pyftsubset input.ttf --text="Hello World"
9
10# Subset for web-safe characters
11pyftsubset input.ttf --unicodes="U+0020-U+007F,U+00A0-U+017F" \
12  --output-file="output_subset.ttf"

Advanced subsetting with character range:

bash
1# Common subsetting ranges:
2# U+0-U+7F        → ASCII
3# U+0-U+17F       → Latin Extended-A
4# U+0-U+24F       → Latin Extended-B
5# U+0400-U+04FF   → Cyrillic
6# U+4E00-U+9FFF   → CJK Unified Ideographs (Chinese/Japanese/Korean)
7
8pyftsubset myfont.ttf --unicodes="U+0-U+7F,U+00A0-U+017F" \
9  --output-file="myfont_latin.ttf"

Web Font CSS & Delivery

Generate CSS with multiple formats (progressive enhancement):

css
1@font-face {
2  font-family: 'MyFont';
3  src: url('myfont.woff2') format('woff2'),
4       url('myfont.woff') format('woff'),
5       url('myfont.ttf') format('truetype');
6  font-display: swap;  /* Show fallback while loading */
7  font-weight: normal;
8  font-style: normal;
9}
10
11body {
12  font-family: 'MyFont', sans-serif;
13}

Self-host fonts efficiently:

bash
1#!/bin/bash
2# Prepare fonts for web delivery
3
4prepare_web_fonts() {
5  local font=$1
6  local base="${font%.*}"
7  
8  # Convert to WOFF2 (primary)
9  fontforge -c "Open('$font'); Generate('${base}.woff2');"
10  
11  # Convert to WOFF (fallback)
12  fontforge -c "Open('$font'); Generate('${base}.woff');"
13  
14  # Show compression achieved
15  echo "Original: $(du -h $font)"
16  echo "WOFF: $(du -h ${base}.woff)"
17  echo "WOFF2: $(du -h ${base}.woff2)"
18}
19
20# Generate all formats
21prepare_web_fonts

For print/publication:

bash
1# Ensure OpenType with embedded metrics
2fontforge -c 'Open($1); SetOS2Value("fsType", 0); Generate($2);' input.ttf output_print.otf
3
4# Create embedded PDF fonts
5# Use xelatex or pdflatex with font inclusion

Font Properties & Inspection

bash
1# List font properties
2fontforge -c 'Open($1); foreach glyph in selection Print($2, FontEm); done' myfont.ttf
3
4# Check character coverage
5fontforge -c 'Open($1); ranges=GetFontOrder(); foreach r in ranges Print(r); done' myfont.ttf
6
7# Inspect with identify (for font files)
8identify myfont.ttf
9
10# Get font name and metadata
11fontforge -c 'Open($1); Print(fontname, ", Weight: ", weight, ", Width: ", width);' myfont.ttf

Font Subsetting for Specific Use Cases

bash
1# Sports/Games - emoji and symbols subset
2pyftsubset font.ttf --text="🎮🏀⚽🎯" --output-file="emoji_subset.ttf"
3
4# CJK languages (large file, consider variable fonts)
5pyftsubset fontcjk.ttf --unicodes="U+4E00-U+9FFF" --output-file="cjk_subset.ttf"
6
7# Icons only (strip all text)
8pyftsubset icons.ttf --unicodes="U+E000-U+F8FF" --output-file="icons_subset.ttf"

CAD and 3D Files

Convert between CAD formats (architectural, engineering) and 3D model formats (game engines, 3D printing).

CAD Format Comparison

FormatEcosystemComplexityFile SizeCompatibilityBest For
DWGAutodesk AutoCADMediumMediumIndustry standardArchitectural, engineering drawings
DXFAutodesk (open subset)Low-MediumMediumGood open support2D exchange, simpler CAD
STEPISO standardHighVariableUniversal 3D CADEngineering, 3D CAD interchange
IGESLegacy ISOMediumLargeLimited modern supportLegacy CAD systems
STL3D printingLowLarge (binary)Universal 3D printing3D printing, mesh export
OBJWavefront (open)Low-MediumMediumUniversal 3D graphics3D modeling, game engines, web
FBXAutodeskHighMediumGame/animation industryGame engines (Unity, Unreal), animation
glTFKhronos (modern)Low-MediumSmall (with compression)Modern 3D webWeb 3D, efficient delivery
BLENDBlenderHighLargeBlender proprietaryComplex scenes, VFX

2D CAD Conversion (DWG ↔ DXF)

Using LibreCAD:

bash
1# DWG to DXF
2librecad input.dwg -export dxf output.dxf
3
4# DXF to DWG (for import into AutoCAD)
5librecad input.dxf -export dwg output.dwg

Batch convert CAD files:

bash
1#!/bin/bash
2# Convert all DWG to DXF
3
4for dwg in *.dwg; do
5  dxf="${dwg%.*}.dxf"
6  echo "Converting: $dwg$dxf"
7  libreoffice --headless --convert-to dxf "$dwg" --outdir .
8done

3D Model Conversion (Blender)

Install Blender (headless):

bash
1# macOS
2brew install blender
3
4# Ubuntu
5sudo apt install blender
6
7# Verify
8blender --version

Basic 3D format conversions:

bash
1# STL to OBJ
2blender --background --python convert.py -- input.stl output.obj
3
4# FBX to glTF (efficient web format)
5blender --background --python convert.py -- input.fbx output.gltf
6
7# Create conversion Python script (convert.py)
8cat > convert.py << 'EOF'
9import sys
10import bpy
11
12# Arguments: blender --python convert.py -- input output
13input_file = sys.argv[-2]
14output_file = sys.argv[-1]
15
16# Load file
17bpy.ops.import_scene.gltf(filepath=input_file) if input_file.endswith('.gltf') else None
18bpy.ops.import_mesh.stl(filepath=input_file) if input_file.endswith('.stl') else None
19bpy.ops.import_scene.fbx(filepath=input_file) if input_file.endswith('.fbx') else None
20
21# Export
22bpy.ops.export_scene.gltf(filepath=output_file, use_format='GLTF_SEPARATE') if output_file.endswith('.gltf') else None
23bpy.ops.export_mesh.stl(filepath=output_file) if output_file.endswith('.stl') else None
24EOF

Optimized conversion for web (glTF):

bash
1# Use Blender to convert and optimize for web
2blender --background model.fbx --python << 'PYTHON'
3import bpy
4import sys
5
6# Import FBX
7bpy.ops.import_scene.fbx(filepath=sys.argv[-3])
8
9# Optimize mesh (reduce polygons if needed)
10for obj in bpy.context.scene.objects:
11    if obj.type == 'MESH':
12        # Apply modifiers
13        bpy.context.view_layer.objects.active = obj
14        bpy.ops.object.modifier_apply(modifier="Subdivision")
15
16# Export optimized glTF
17bpy.ops.export_scene.gltf(
18    filepath=sys.argv[-2],
19    use_format='GLTF_EMBEDDED',
20    use_draco_mesh_compression=True
21)
22PYTHON output.gltf

STL for 3D Printing

Prepare STL for 3D printing:

bash
1# Check mesh integrity
2# Use Blender to validate
3
4# Scale STL to printing dimensions
5# Units typically in millimeters for 3D printers
6blender --background model.stl --python << 'PYTHON'
7import bpy
8
9# Scale model to 100mm size
10bpy.ops.transform.scale(value=100)
11
12# Export
13bpy.ops.export_mesh.stl(filepath='model_scaled.stl')
14PYTHON

OBJ for Game Engines

Export to OBJ for game engines:

bash
1# High-poly model for detailed graphics
2blender --background model.blend --python << 'PYTHON'
3import bpy
4
5# Select all mesh objects
6for obj in bpy.data.objects:
7    if obj.type == 'MESH':
8        obj.select_set(True)
9
10# Export
11bpy.ops.export_scene.obj(
12    filepath='model_highpoly.obj',
13    use_normals=True,
14    use_uv_coords=True,
15    use_materials=False
16)
17PYTHON

Batch convert multiple models:

bash
1#!/bin/bash
2# Convert multiple FBX to OBJ for game engine
3
4for fbx in *.fbx; do
5  obj="${fbx%.fbx}.obj"
6  echo "Converting: $fbx$obj"
7  
8  blender --background "$fbx" -o "$obj" -f STILL
9done

STEP Files (Engineering CAD)

Convert STEP to other formats (requires specialized tools):

bash
1# Using FreeCAD CLI
2freecad --batch << 'PYTHON'
3import Part
4Part.open('input.step')
5Part.exportSTL('output.stl')
6PYTHON
7
8# Convert STEP to IGES
9freecad --batch << 'PYTHON'
10import Part
11shape = Part.open('input.step').read()
12Part.exporter('iges')('output.iges', shape)
13PYTHON

HTML Rendering and Web Content

Convert web content (HTML/CSS) to static documents (PDF, images). Chromium headless provides the most accurate rendering.

Rendering Engines Compared

ToolEnginePDF QualityImage QualityJavaScript SupportCSS SupportBest For
Chromium --headlessBlinkExcellentExcellentFullExcellentModern web pages, complex layouts
wkhtmltopdfQtWebKitGoodN/ALimitedGoodOlder but widely deployed
PuppeteerChromium + Node.jsExcellentExcellentFullExcellentProgrammatic rendering, automation
Headless FirefoxGeckoExcellentExcellentFullExcellentFirefox rendering, open-source
LibreOfficeIntegratedGoodN/ANoLimitedDocument-oriented content

PDF from HTML (Chromium)

Basic HTML to PDF:

bash
1# Simple conversion
2chromium --headless --disable-gpu --print-to-pdf=output.pdf input.html
3
4# With custom paper size
5chromium --headless --disable-gpu \
6  --print-to-pdf=output.pdf \
7  --print-to-pdf-margin-top=10 \
8  --print-to-pdf-margin-bottom=10 \
9  --print-to-pdf-margin-left=10 \
10  --print-to-pdf-margin-right=10 \
11  input.html

Advanced Page Rendering Options:

bash
1# A4 paper size, 96 DPI
2chromium --headless --disable-gpu \
3  --virtual-time-budget=5000 \
4  --disable-features=TranslateUI \
5  --print-to-pdf=output.pdf \
6  --print-to-pdf-margin-top=0.5in \
7  --print-to-pdf-margin-bottom=0.5in \
8  --print-to-pdf-margin-left=0.75in \
9  --print-to-pdf-margin-right=0.75in \
10  input.html

HTML with CSS and images to PDF:

bash
1# Render with CSS processed
2chromium --headless --disable-gpu \
3  --default-background-color=16777215 \
4  --print-to-pdf=output.pdf \
5  file:///absolute/path/to/input.html

Screenshot (Web to Image)

Using Chromium headless:

bash
1# Take webpage screenshot
2chromium --headless --disable-gpu \
3  --screenshot \
4  --window-size=1920x1080 \
5  input.html
6
7# Specify output file
8chromium --headless --disable-gpu \
9  --screenshot=output.png \
10  --window-size=1920x1080 \
11  input.html

Programmatic Web Rendering with Puppeteer

Node.js-based rendering (more control):

bash
1# Install Puppeteer
2npm install puppeteer

Create PDF from HTML programmatically:

javascript
1// render.js
2const puppeteer = require('puppeteer');
3
4(async () => {
5  const browser = await puppeteer.launch();
6  const page = await browser.newPage();
7  
8  // Set viewport/paper size
9  await page.setViewport({ width: 1920, height: 1080 });
10  
11  // Load HTML
12  await page.goto('file:///path/to/input.html', {
13    waitUntil: 'networkidle2'
14  });
15  
16  // Generate PDF with options
17  await page.pdf({
18    path: 'output.pdf',
19    format: 'A4',
20    margin: {
21      top: '0.5in',
22      bottom: '0.5in',
23      left: '0.75in',
24      right: '0.75in'
25    },
26    printBackground: true
27  });
28  
29  // Take screenshot
30  await page.screenshot({ path: 'output.png', fullPage: true });
31  
32  await browser.close();
33})();

Run Puppeteer script:

bash
1node render.js

Render Remote URLs

bash
1# Render website from URL to PDF
2chromium --headless --disable-gpu \
3  --print-to-pdf=website.pdf \
4  https://example.com
5
6# With authentication headers
7chromium --headless --disable-gpu \
8  --print-to-pdf=website.pdf \
9  https://example.com

Advanced Rendering Scenarios

Render with JavaScript execution delay:

bash
1# Wait for JavaScript to complete (virtual time)
2chromium --headless --disable-gpu \
3  --virtual-time-budget=5000 \
4  --print-to-pdf=output.pdf \
5  input.html
6
7# Using Puppeteer with wait conditions
8const page = await browser.newPage();
9await page.goto(url, { waitUntil: 'networkidle2' });
10await page.waitForSelector('.dynamic-content');  // Wait for element
11await page.pdf({ path: 'output.pdf' });

Batch convert HTML files:

bash
1#!/bin/bash
2# Convert all HTML to PDF
3
4for html in *.html; do
5  pdf="${html%.html}.pdf"
6  echo "Converting: $html$pdf"
7  
8  chromium --headless --disable-gpu \
9    --print-to-pdf="$pdf" \
10    "file://$(pwd)/$html"
11done

Render with custom CSS injection:

javascript
1// Puppeteer: inject CSS before rendering
2const page = await browser.newPage();
3await page.goto('file:///input.html');
4
5// Inject custom CSS (for print styling)
6await page.addStyleTag({
7  content: `
8    body { font-size: 12pt; line-height: 1.5; }
9    .no-print { display: none; }
10    @media print { a { color: black; text-decoration: none; } }
11  `
12});
13
14await page.pdf({ path: 'output.pdf' });

Performance & Optimization

bash
1# Disable unnecessary features for faster rendering
2chromium --headless --disable-gpu \
3  --disable-web-resources \
4  --disable-default-apps \
5  --disable-extensions \
6  --no-sandbox \
7  --print-to-pdf=output.pdf \
8  input.html
9
10# Parallel rendering (multiple PDFs)
11#!/bin/bash
12for i in {1..4}; do
13  chromium --headless --disable-gpu \
14    --print-to-pdf="output_$i.pdf" \
15    "page_$i.html" &
16done
17wait  # Wait for all background jobs

PDF vs Screenshot Format Selection

Use CaseFormatRendering ToolReasoning
Print-ready documentsPDFChromium, PuppeteerVector format, file size, typography control
Web page previewPNG/JPGChromium --screenshotGood for thumbnails, sharing
Archive/backupPDFPuppeteer with embedSearchable, accessible, long-term
Email-friendlyJPGChromium --screenshotCompatibility, inline viewing
Complex layoutsPDFPuppeteerPreserves spacing, multi-page formatting

Best Practices and Advanced Tips

Development Workflow Best Practices

1. Always Backup Originals

bash
1# Before any conversion, backup the original
2cp original.mp4 original.mp4.backup
3
4# Or create a deduplicated backup archive
5tar czf backups/$(date +%Y%m%d_%H%M%S)_backup.tar.gz original.*

2. Test Conversions First

bash
1# Test on small sample before batch processing
2ffmpeg -i input.mp4 -t 10 -c:v libx264 -crf 25 test_output.mp4
3# Preview to ensure quality/settings are acceptable before full conversion

3. Use Version Control for Scripts

bash
1# Store conversion scripts in git for reproducibility
2git init conversion-scripts
3git add *.sh
4git commit -m "Initial conversion tooling"

Quality Assessment Workflows

Audio Quality Metrics:

bash
1# ABX testing framework (perceptual blind testing)
2# For critical audio comparisons
3
4# PESQ (Perceptual Evaluation of Speech Quality)
5# Requires PESQ binary, compares against reference
6pesq +16000 reference.wav converted.wav
7
8# Spectral analysis with FFmpeg
9ffmpeg -i original.mp3 -vf showspectrum -t 5 spectrum_orig.mp4
10ffmpeg -i converted.mp3 -vf showspectrum -t 5 spectrum_conv.mp4

Video Quality Metrics:

bash
1# PSNR (Peak Signal-to-Noise Ratio) - measure distortion
2ffmpeg -i reference.mp4 -i converted.mp4 -filter_complex "[0][1]psnr=stats_file=psnr.txt" \
3  -f null -
4grep 'PSNR' psnr.txt | tail -1
5
6# SSIM (Structural Similarity Index) - perceptual quality
7ffmpeg -i reference.mp4 -i converted.mp4 -filter_complex "[0][1]ssim=stats_file=ssim.txt" \
8  -f null -
9grep 'SSIM' ssim.txt | tail -1

Image Quality Comparison:

bash
1# PSNR and SSIM for images
2magick compare -metric PSNR original.jpg converted.webp psnr.txt
3magick compare -metric SSIM original.jpg converted.webp ssim.txt
4
5# Visual diff
6magick compare original.jpg converted.webp -compose Src difference.png

Production Automation Patterns

Queue-based conversion system:

bash
1#!/bin/bash
2# conversion-queue.sh - Redis/file-based job queue
3
4QUEUE_DIR="/var/data/conversion_queue"
5RESULTS_DIR="/var/data/conversion_results"
6LOG_FILE="/var/log/conversion.log"
7
8process_queue() {
9  while true; do
10    # Check for pending jobs
11    if [ -f "$QUEUE_DIR/pending.txt" ]; then
12      job=$(head -1 "$QUEUE_DIR/pending.txt")
13      
14      # Extract job parameters
15      input=$(echo $job | cut -d'|' -f1)
16      output=$(echo $job | cut -d'|' -f2)
17      format=$(echo $job | cut -d'|' -f3)
18      
19      echo "[$(date)] Processing: $job" >> "$LOG_FILE"
20      
21      # Execute conversion with timeout
22      timeout 3600 convert_file "$input" "$output" "$format"
23      
24      if [ $? -eq 0 ]; then
25        echo "SUCCESS: $job" >> "$LOG_FILE"
26        mv "$job" "$QUEUE_DIR/completed.txt"
27      else
28        echo "FAILED: $job" >> "$LOG_FILE"
29        # Retry logic
30        retry_count=$(grep -c "$job" "$QUEUE_DIR/failed.txt" 2>/dev/null || echo 0)
31        if [ $retry_count -lt 3 ]; then
32          echo "$job" >> "$QUEUE_DIR/pending.txt"
33        fi
34      fi
35      
36      # Remove from pending
37      tail -n +2 "$QUEUE_DIR/pending.txt" > "$QUEUE_DIR/pending.txt.tmp"
38      mv "$QUEUE_DIR/pending.txt.tmp" "$QUEUE_DIR/pending.txt"
39    fi
40    
41    sleep 1
42  done
43}
44
45process_queue

Error handling and retry logic:

bash
1#!/bin/bash
2# Conversion with automatic retry and fallback
3
4convert_with_fallback() {
5  local input="$1"
6  local output="$2"
7  local max_retries=3
8  
9  for ((i=1; i<=max_retries; i++)); do
10    echo "Attempt $i/$max_retries: Converting $input"
11    
12    # Try primary conversion method
13    if ffmpeg -i "$input" -c:v libx264 -crf 25 "$output" -y 2>/dev/null; then
14      echo "✓ Success with primary method"
15      return 0
16    fi
17    
18    # Fallback 1: Lower quality settings
19    if [ $i -eq 2 ]; then
20      echo "Attempting with lowered quality..."
21      if ffmpeg -i "$input" -c:v libx264 -crf 30 -preset fast "$output" -y 2>/dev/null; then
22        echo "✓ Success with fallback settings"
23        return 0
24      fi
25    fi
26    
27    # Fallback 2: Different codec
28    if [ $i -eq 3 ]; then
29      echo "Attempting with alternative codec..."
30      if ffmpeg -i "$input" -c:v mpeg4 -q:v 5 "$output" -y 2>/dev/null; then
31        echo "✓ Success with alternative codec"
32        return 0
33      fi
34    fi
35    
36    # Wait before retry
37    sleep $((i * 5))
38  done
39  
40  echo "✗ Conversion failed after $max_retries attempts"
41  return 1
42}
43
44convert_with_fallback "$1" "$2"

Parallel batch processing with rate limiting:

bash
1#!/bin/bash
2# Parallel conversion with system resource monitoring
3
4MAX_PARALLEL=4
5JOBS_RUNNING=0
6
7convert_file_async() {
8  local input="$1"
9  local output="$2"
10  
11  # Wait if max parallel jobs reached
12  while [ $(jobs -r | wc -l) -ge $MAX_PARALLEL ]; do
13    sleep 1
14  done
15  
16  # Run conversion in background
17  {
18    echo "Converting: $input"
19    ffmpeg -i "$input" -c:v libx264 -crf 25 "$output"
20    echo "Completed: $input"
21  } &
22}
23
24# Process all files
25for input in *.mp4; do
26  output="${input%.*}_converted.mp4"
27  convert_file_async "$input" "$output"
28done
29
30# Wait for all background jobs
31wait
32echo "All conversions complete!"

Monitoring & Logging

Comprehensive logging setup:

bash
1#!/bin/bash
2# conversion-with-logging.sh
3
4LOG_FILE="conversion_$(date +%Y%m%d_%H%M%S).log"
5ERROR_LOG="conversion_errors_$(date +%Y%m%d).log"
6
7log() {
8  echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
9}
10
11log_error() {
12  echo "[$(date '+%Y-%m-%d %H:%M:%S')] ERROR: $1" | tee -a "$ERROR_LOG"
13}
14
15convert_with_logging() {
16  local input="$1"
17  local output="$2"
18  
19  log "START: Converting $input"
20  
21  # Capture both stdout and stderr
22  if ffmpeg -i "$input" -c:v libx264 -crf 25 "$output" >> "$LOG_FILE" 2>&1; then
23    log "SUCCESS: $output created ($(du -h $output | cut -f1))"
24  else
25    log_error "Conversion failed for $input"
26    return 1
27  fi
28}
29
30# Process files
31for file in *.mp4; do
32  convert_with_logging "$file" "${file%.mp4}_converted.mp4"
33done
34
35log "All conversions completed"

Container-Based Deployment

Docker for consistent conversion environment:

dockerfile
1# Dockerfile for file conversion toolkit
2
3FROM ubuntu:22.04
4
5# Install all conversion tools
6RUN apt-get update && apt-get install -y \
7    ffmpeg \
8    imagemagick \
9    libreoffice \
10    pandoc \
11    ghostscript \
12    tesseract-ocr \
13    poppler-utils \
14    qpdf \
15    exiftool \
16    calibre
17
18# Create working directory
19WORKDIR /conversions
20
21# Health check
22HEALTHCHECK --interval=30s CMD ffmpeg -version
23
24ENTRYPOINT ["/bin/bash"]

Build and run:

bash
1docker build -t conversion-toolkit .
2docker run -v $(pwd):/conversions conversion-toolkit convert.sh

Complex Ad-Hoc Conversions

Multi-format web image generation:

bash
1#!/bin/bash
2# Generate responsive image set with fallbacks
3
4generate_image_set() {
5  local input="$1"
6  local base="${input%.*}"
7  
8  # Define breakpoints
9  declare -a sizes=(320 640 1024 1920)
10  declare -a formats=(webp jpg avif)
11  
12  for size in "${sizes[@]}"; do
13    for format in "${formats[@]}"; do
14      output="${base}_${size}w.${format}"
15      
16      echo "Generating: $output"
17      magick "$input" -resize "${size}x${size}" \
18        -quality 85 -define "${format}:method=6" \
19        "$output"
20    done
21  done
22}
23
24generate_image_set "hero.png"

Concatenate media files with format conversion:

bash
1#!/bin/bash
2# Join multiple videos with format sync
3
4concat_videos() {
5  local output="$1"
6  shift
7  local files=("$@")
8  
9  # Create concat demuxer file
10  for file in "${files[@]}"; do
11    echo "file '$file'" >> concat.txt
12  done
13  
14  # Concatenate with format conversion
15  ffmpeg -f concat -safe 0 -i concat.txt \
16    -c:v libx264 -crf 25 -c:a aac -b:a 192k \
17    "$output"
18  
19  rm concat.txt
20}
21
22concat_videos "merged.mp4" video1.mp4 video2.mkv video3.mov

Security Considerations

Sanitize filenames for batch processing:

bash
1#!/bin/bash
2# Prevent path traversal and injection attacks
3
4sanitize_filename() {
5  local filename="$1"
6  # Remove path components, special characters
7  echo "$filename" | sed 's/[^a-zA-Z0-9._-]//g' | head -c 255
8}
9
10for file in *; do
11  safe_name=$(sanitize_filename "$file")
12  if [ "$file" != "$safe_name" ]; then
13    echo "Renaming: $file$safe_name"
14    mv "$file" "$safe_name"
15  fi
16done

Limit resource usage:

bash
1# Prevent runaway processes from consuming system resources
2
3# Set resource limits for conversion process
4ulimit -v 4194304  # 4GB memory limit
5ulimit -t 3600     # 1 hour CPU time limit
6
7# Run FFmpeg with memory/time constraints
8ffmpeg -i input.mp4 -c:v libx264 -crf 25 output.mp4

Performance Tuning

Optimal FFmpeg settings by use case:

Use CaseCodecPresetCRFBitrateSpeed
LivestreamH.264fastN/A3-5 MbpsReal-time
Web streamingH.264medium252-4 MbpsFast
ArchiveH.265slower221.5-3 MbpsSlow
MobileH.264fast281-2 MbpsVery fast
Print stillsPNGN/AN/ALosslessMedium

Troubleshooting Guide

bash
1# Check tool versions
2ffmpeg -version | head -1
3magick -version | head -1
4libreoffice --version
5
6# Verify codec/format support
7ffmpeg -codecs | grep -i h264
8ffmpeg -formats | grep -i mp4
9magick -list format | grep -i pdf
10
11# Debug conversion issues
12# Enable verbose output
13ffmpeg -v debug -i input.mp4 output.mp4 2>&1 | grep -i error
14
15# Check system resources during conversion
16watch -n 1 'ps aux | grep ffmpeg | grep -v grep'
17
18# Monitor disk space
19watch -n 5 'df -h /path/to/working/dir'

Summary Checklist

  • ✓ Backup originals before any conversion
  • ✓ Test on small sample first
  • ✓ Monitor quality with appropriate metrics
  • ✓ Use version-controlled, documented scripts
  • ✓ Implement error handling and retry logic
  • ✓ Log all operations for troubleshooting
  • ✓ Parallelize where possible, but respect system limits
  • ✓ Strip sensitive metadata before distribution
  • ✓ Test output on target platforms
  • ✓ Document conversion parameters for reproducibility
  • ✓ Keep tools updated for bug fixes and new features
  • ✓ Use containers for consistent environments
  • ✓ Monitor storage and cleanup old conversions

This guide serves as a living reference. Expand sections as needed with tool-specific documentation for edge cases and production requirements.

Connect

Let's build together

Currently accepting new projects for remote full-stack development and architecture.
Phone / WhatsApp+34 627 191 675
LocationCosta Blanca, Spain (Remote Worldwide) · Remote worldwide
Available — Responds in 24h
Message

Prefer a call? Message on WhatsApp