ocr-cpp

About

OCR Service in C++.

Web service to perform optical character recognition on images using Tesseract API with Leptonica.

There is currently experimental support to optionally use a local LLM such as GLM OCR with llama.cpp to recognize i.e. handwritten text which Tesseract is not designed for.

When using a suitable model such as Gemma 4 E4B, the service can also transcribe speech in text, and describe the contents of images and video clips. Also specialized models such as MedGemma 1.5 4B can be used.

The web service is using cpp-httplib, providing multithreading (using at least 8 threads or the number of available logical CPUs minus one), and json to structure recognized text in a Json array. Plain text, hOCR and TSV output is supported as well.

Since a Tesseract API instance can not be used concurrently, one instance is created per thread and reused for all requests handled by that thread.

LLM recognitions are possible only one at a time; concurrent requests using LLM are blocked until an ongoing process completes. Recognitions using Tesseract are however handled concurrently, also while the LLM is busy.

Images are converted to JPEG, so all image formats supported by libvips can be used for LLM recognitions, including multipage TIFF and PDF. If necessary, images are scaled down to a reasonable size to improve performance.

As a little by-product, the service can also generate thumbnails very efficiently.

Limitations

• LLM recognition currently only returns plain text/Markdown

Building

A C++ compiler supporting modules, CMake and the dependent libraries are required to build and test the project. Building works fine on Debian 13 with the following toolchain and libraries installed from Debian repository with 'apt':

• CMake
• Ninja
• GNU Make (for Eclipse CDT LSP - clangd BMI)
• g++ 15.2.0
• Doxygen
• Catch2
• libtesseract5
• libleptonica6
• libcpp-httplib0.41
• nlohmann-json3
• libicu78
• libvips
• libmagic

The following command should install all that is needed to build the project:

sudo apt install build-essential cmake ninja-build doxygen catch2 \
libtesseract-dev libleptonica-dev libcpp-httplib-dev \
nlohmann-json3-dev libicu-dev libvips-dev libmagic-dev

Since this project is currently using the internal API, it is probably easiest to download the source from llama.cpp, build it and copy the needed headers to i.e. /usr/local/include/llama and the shared libraries to /usr/local/lib/llama.

Once all dependencies are satisfied the project can be built by running:

cmake -B build -G Ninja
ninja -C build

This compiles the executables build/ocr and build/ocr-test.

Testing

To run the tests:

ninja -C build test

Running

It might be necessary to install libtesseract5 and language files, and other libraries:

sudo apt install libtesseract5 tesseract-ocr-deu tesseract-ocr-eng \
tesseract-ocr-fra libleptonica6 libcpp-httplib0.41 libicu78 \
libvips42t64 libopenblas0 libvulkan1

The service is run with for example:

cd build
./ocr 0.0.0.0 8080 /path/to/GLM-OCR-Q8_0.gguf /path/to/mmproj-GLM-OCR-Q8_0.gguf

Available parameters are:

• bind address: i.e. '0.0.0.0' or a hostname/FQDN
• http port: i.e. '8080'
• model path: the path to the model, i.e. '/path/to/GLM-OCR-Q8_0.gguf'
• mmproj path: the path to the multimodal projection, i.e. '/path/to/mmproj-GLM-OCR-Q8_0.gguf'

Environment variables that can be set:

• OCR_GPU_LAYERS: number of GPU layers in VRAM. '-1' is auto (default), <= '-2' is all
• OCR_NO_GPU: if set to any value, the GPU is not used for multimodal
• OMP_THREAD_LIMIT: max. mumber of CPU threads

An effort is made to clean up before exiting on CTRL-C.

Using

OCR

Images can be PUT'ed to the REST endpoint /ocr with the following query parameters:

• llm: 'true' for using the LLM, 'false' or absent to use Tesseract.
• lang: i.e. 'lang=en', applies only to Tesseract. The matching language file must be available.
• format: can be one of: 'text', 'json', 'hocr', 'tsv'. Ignored by the LLM option.

Process Media Content

Media content can be PUT'ed to /media. No parameters need to be given.

Content is processed using a prompt based on the mime type:

• 'audio': Transcribe speech in text
• 'image': Describe the content
• 'video': Describe what is happening

The header X-Prompt can be used to override these default prompts.

The /media endpoint passes the media content as-is to llama, without doing any conversion like the /ocr endpoint does.

Generate thumbnails

Thumbnails can be generated by PUT'ing an image to the /thumbs endpoint.

For multipage images, the X-Pages header indicates the number of thumbnails returned. The response contains concatenated JPEG data which can be split using JPEG magic bytes.

Examples

Recognize text in 'res/eng.png' with Tesseract and return it as plain text:

curl --request PUT --url 'http://localhost:8080/ocr?lang=en' \
--data-binary @res/eng.png

Recognize script in 'res/scribble.png' with llama and return it as plain text:

curl --request PUT --url 'http://localhost:8080/ocr?llm=true' \
--data-binary @res/scribble.png

Transcribe speech in 'res/JFK.mp3' with llama and return it as plain text:

curl --request PUT --url 'http://localhost:8080/media' \
--data-binary @res/JFK.mp3

Analyze video 'res/bear.mkv' with llama:

curl --request PUT --url 'http://localhost:8080/media' \
--data-binary @res/bear.mkv

Generate thumbnails (JPEG):

curl --request PUT --url 'http://localhost:8080/thumbs' \
--data-binary @res/eng.png --output thumb.jpg

Container

Building a container image currently requires some manual work:

• Copy the llama shared libraries into the build context, i.e.: mkdir -p llama && cp -r /usr/local/lib/llama/* llama/
• Copy the model and multimodal projection to: models/model.gguf and models/mmproj.gguf
• Run ninja -C build image

To run the container:

docker run --privileged --rm -p 8080:8080 gitdode/ocr-cpp

Running the container in privileged mode is necessary to give access to the graphics device.

Documentation

To update the documentation in the directory doc, run:

ninja -C build doc

Clang/Eclipse CDT LSP Editor

Once initially, the BMI for libstdc++ needs to be precompiled:

make bmi

Precompiling the module interfaces *.cppm when they were modified is done the same way, and for the changes to be reflected in the LSP editor it is enough to edit an affected file.

TODO

• Write more tests (always)
• See issues