Create README.md

README.md

@@ -0,0 +1,132 @@
+---
+license: openrail
+library_name: transformers
+tags:
+- ocr
+- vlm
+---
+
+# Chandra
+
+Chandra is an OCR model that outputs markdown, HTML, and JSON.  It is highly accurate at extracting text from images and PDFs, while preserving layout information.
+
+You can try Chandra in the free playground [here](https://www.datalab.to/playground), or at a hosted API [here](https://www.datalab.to/).
+
+## Features
+
+- Convert documents to markdown, html, or json with detailed layout information
+- Good handwriting support
+- Reconstructs forms accurately, including checkboxes
+- Good support for tables, math, and complex layouts
+- Extracts images and diagrams, with captions and structured data
+- Support for 40+ languages
+
+## Quickstart
+
+The easiest way to start is with the CLI tools:
+
+```shell
+pip install chandra-ocr
+
+# With VLLM
+chandra_vllm
+chandra input.pdf ./output
+
+# With HuggingFace
+chandra input.pdf ./output --method hf
+
+# Interactive streamlit app
+chandra_app
+```
+
+## Benchmarks
+
+| **Model** |  ArXiv   | Old Scans Math |  Tables  | Old Scans | Headers and Footers | Multi column | Long tiny text |   Base   |    Overall     |
+|:----------|:--------:|:--------------:|:--------:|:---------:|:-------------------:|:------------:|:--------------:|:--------:|:--------------:|
+| Datalab Chandra v0.1.0 |   81.4   |    **80.3**    | **89.4** | **50.0**  |        88.3         |   **81.0**   |    **91.6**    | **99.9** | **82.7 ± 0.9** |
+| Datalab Marker v1.10.0 | **83.8** |      69.7      |   74.8   |   32.3    |        86.6         |     79.4     |      85.7      |   99.6   |   76.5 ± 1.0   |
+| Mistral OCR API |   77.2   |      67.5      |   60.6   |   29.3    |        93.6         |     71.3     |      77.1      |   99.4   |   72.0 ± 1.1   |
+| Deepseek OCR |   75.2   |      67.9      |   79.1   |   32.9    |        96.1         |     66.3     |      78.5      |   97.7   |   74.2 ± 1.0   |
+| GPT-4o (Anchored) |   53.5   |      74.5      |   70.0   |   40.7    |        93.8         |     69.3     |      60.6      |   96.8   |   69.9 ± 1.1   |
+| Gemini Flash 2 (Anchored) |   54.5   |      56.1      |   72.1   |   34.2    |        64.7         |     61.5     |      71.5      |   95.6   |   63.8 ± 1.2   |
+| Qwen 3 VL |   70.2   |      75.1      |   45.6   |   37.5    |        89.1         |     62.1     |      43.0      |   94.3   |   64.6 ± 1.1   |
+| olmOCR v0.3.0 |   78.6   |      79.9      |   72.9   |   43.9    |      **95.1**       |     77.3     |      81.2      |   98.9   |   78.5 ± 1.1   |
+
+## Examples
+
+| Type | Name | Link |
+|------|------|------|
+| Tables | Water Damage Form | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/tables/water_damage.png) |
+| Tables | 10K Filing | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/tables/10k.png) |
+| Forms | Handwritten Form | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/forms/handwritten_form.png) |
+| Forms | Lease Agreement | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/forms/lease.png) |
+| Handwriting | Doctor Note | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/handwriting/doctor_note.png) |
+| Handwriting | Math Homework | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/handwriting/math_hw.png) |
+| Books | Geography Textbook | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/books/geo_textbook_page.png) |
+| Books | Exercise Problems | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/books/exercises.png) |
+| Math | Attention Diagram | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/attn_all.png) |
+| Math | Worksheet | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/worksheet.png) |
+| Math | EGA Page | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/math/ega.png) |
+| Newspapers | New York Times | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/newspapers/nyt.png) |
+| Newspapers | LA Times | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/newspapers/la_times.png) |
+| Other | Transcript | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/other/transcript.png) |
+| Other | Flowchart | [View](https://github.com/datalab-to/chandra/blob/master/assets/examples/other/flowchart.png) |
+
+## Usage
+
+### Installation
+
+```shell
+pip install chandra-ocr
+```
+
+### From code
+
+```python
+
+from chandra.model import InferenceManager
+from chandra.model.schema import BatchInputItem
+
+# Run chandra_vllm to start a vLLM server first if you pass vllm, else pass hf
+# you can also start your own vllm server with the datalab-to/chandra model
+manager = InferenceManager(method="vllm")
+batch = [
+    BatchInputItem(
+        image=PIL_IMAGE,
+        prompt_type="ocr_layout"
+    )
+]
+result = manager.generate(batch)[0]
+print(result.markdown)
+```
+
+### With transformers
+
+```python
+from transformers import AutoModel, AutoProcessor
+from chandra.model.hf import generate_hf
+from chandra.model.schema import BatchInputItem
+from chandra.output import parse_markdown
+
+model = AutoModel.from_pretrained("datalab-to/chandra").cuda()
+model.processor = AutoProcessor.from_pretrained("datalab-to/chandra")
+
+batch = [
+    BatchInputItem(
+        image=PIL_IMAGE,
+        prompt_type="ocr_layout"
+    )
+]
+
+result = generate_hf(batch, model)[0]
+markdown = parse_markdown(result.raw)
+```
+
+# Credits
+
+Thank you to the following open source projects:
+
+- [Huggingface Transformers](https://github.com/huggingface/transformers)
+- [VLLM](https://github.com/vllm-project/vllm)
+- [olmocr](github.com/allenai/olmocr)
+- [Qwen 3 VL](https://github.com/QwenLM/Qwen3)