ocr-cpp

About

OCR Service in C++.

Web service to perform optical character recognition on images using Tesseract API with Leptonica.

There is currently experimental support to optionally use a local LLM such as GLM OCR with llama.cpp to recognize i.e. handwritten text which Tesseract is not designed for.

Speech recognition from an audio file is also possible, when using a model with audio support such as Gemma 4 E4B.

The web service is using cpp-httplib, providing multithreading (using at least 8 threads or the number of available logical CPUs minus one), and json to structure recognized text in a Json array. Plain text, hOCR and TSV output is supported as well.

Since a Tesseract API instance can not be used concurrently, one instance is created per thread and reused for all requests handled by that thread.

LLM recognitions are possible only one at a time; concurrent requests using LLM are blocked until an ongoing process completes. Recognitions using Tesseract are however handled concurrently, also while the LLM is busy.

Images are converted to JPEG, so all image formats supported by libvips can be used for LLM recognitions, including multipage TIFF and PDF. If necessary, images are scaled down to a reasonable size to improve performance.

As a little by-product, the service can also generate thumbnails very efficiently.

Limitations

• LLM recognition currently only returns plain text/Markdown

Building

A C++ compiler supporting modules, CMake and the dependent libraries are required to build and test the project. Building works fine on Debian 13 with the following toolchain and libraries installed from Debian repository with 'apt':

• CMake
• Ninja
• GNU Make (for Eclipse CDT LSP - clangd BMI)
• g++ 15.2.0
• Doxygen
• Catch2
• libtesseract5
• libleptonica6
• libcpp-httplib0.41
• nlohmann-json3
• libicu78
• libvips

The following command should install all that is needed to build the project:

sudo apt install build-essential cmake ninja-build doxygen catch2 \
libtesseract-dev libleptonica-dev libcpp-httplib-dev \
nlohmann-json3-dev libicu-dev libvips-dev

Since this project is currently using the internal API, it is probably easiest to download the source from llama.cpp, build it and copy the needed headers to i.e. /usr/local/include/llama and the shared libraries to /usr/local/lib/llama.

Once all dependencies are satisfied the project can be built by running:

cmake -B build -G Ninja
ninja -C build

This compiles the executables build/ocr and build/ocr-test.

Testing

To run the tests:

ninja -C build run_tests

Running

It might be necessary to install libtesseract5 and language files, and other libraries:

sudo apt install libtesseract5 tesseract-ocr-deu tesseract-ocr-eng \
tesseract-ocr-fra libleptonica6 libcpp-httplib0.41 libicu78 \
libvips42t64 libopenblas0 libvulkan1

The service is run with for example:

cd build
./ocr 0.0.0.0 8080 /path/to/GLM-OCR-Q8_0.gguf /path/to/mmproj-GLM-OCR-Q8_0.gguf

Available parameters are:

• bind address: i.e. '0.0.0.0' or a hostname/FQDN
• http port: i.e. '8080'
• model path: the path to the model, i.e. '/path/to/GLM-OCR-Q8_0.gguf'
• mmproj path: the path to the multimodal projection, i.e. '/path/to/mmproj-GLM-OCR-Q8_0.gguf'

Environment variables that can be set:

• OCR_GPU_LAYERS: number of GPU layers in VRAM. '-1' is auto (default), <= '-2' is all
• OCR_NO_GPU: if set to any value, the GPU is not used for multimodal
• OMP_THREAD_LIMIT: max. mumber of CPU threads

An effort is made to clean up before exiting on CTRL-C.

Using

Images can be PUT'ed to the REST endpoint /ocr with the following query parameters:

• llm: 'true' for using the LLM, 'false' or absent to use Tesseract.
• lang: i.e. 'lang=en', applies only to Tesseract. The matching language file must be available.
• format: can be one of: 'text', 'json', 'hocr', 'tsv'. Ignored by the LLM option.

Just for fun, the header X-Prompt can be used to override the default prompt with for example "Could you describe the image for me?"

For multipage images, the X-Pages header indicates the number of thumbnails returned. The response contains concatenated JPEG data which can be split using JPEG magic bytes.

Examples

Recognize text in 'res/eng.png' with Tesseract and return it as plain text:

curl --request PUT --url 'http://localhost:8080/ocr?lang=en' \
--data-binary @res/eng.png

Recognize script in 'res/scribble.png' with llama and return it as plain text:

curl --request PUT --url 'http://localhost:8080/ocr?llm=true' \
--data-binary @res/scribble.png

Recognize speech in 'res/blabla.mp3' with llama and return it as plain text:

curl --request PUT --url 'http://localhost:8080/speech' \
--data-binary @res/blabla.mp3

Generate thumbnails (JPEG):

curl --request PUT --url 'http://localhost:8080/thumbs' \
--data-binary @res/eng.png --output thumb.jpg

Container

Building a container image currently requires some manual work:

• Copy the llama shared libraries into the build context, i.e.: mkdir -p llama && cp -r /usr/local/lib/llama/* llama/
• Copy the model and multimodal projection to: models/model.gguf and models/mmproj.gguf
• Run ninja -C build image

To run the container:

docker run --privileged --rm -p 8080:8080 gitdode/ocr-cpp

Running the container in privileged mode is necessary to give access to the graphics device.

Documentation

To update the documentation in the directory doc, run:

ninja -C build doc

Clang/Eclipse CDT LSP Editor

Once initially, the BMI for libstdc++ needs to be precompiled:

make bmi

Precompiling the module interfaces *.cppm when they were modified is done the same way, and for the changes to be reflected in the LSP editor it is enough to edit an affected file.

TODO

• Write more tests (always)
• See issues