publishA research assistant can quote the right document and still read the wrong table. The damage can happen before a language model receives a single word.
In a public agricultural-statistics PDF, the Balasore district row has separate columns for total rice requirement and seasonal production. Microsoft’s MarkItDown 0.1.9 turns the requirement value 3.47 and the next production value 2.78 into one Markdown cell. A population heading comes out backwards. The conversion command finishes successfully.
If you prepare reports for an assistant, this is a consequential distinction: a file that opens is not necessarily a faithful account of what its source says. Here is the observed failure, the limited repair that works, and a way to retain a usable result without asking an AI to guess the missing structure.
The numbers survived. Their relationships did not
The input is a public one-page PDF attached to a pdfplumber discussion. Its printed page number is 87, and its table describes district rice surplus and deficit during 2012–13. These are historical sample data, not current agricultural estimates or a trading signal.
The following comparison comes from opening the rendered page and executing the current stable MarkItDown release on November 3, 2026:
| Item | Visible source | Observed converted text |
|---|---|---|
| Population heading | Projected Population | noitalupoP detcejorP |
| Seasonal heading | Kharif | firahK |
| Balasore requirement and following production cell | 3.47, then 2.78 in separate columns | 3.47 2.78 in one cell |
| Balasore next production cells | 0.86, then 3.64 in separate columns | 0.86 3.64 in one cell |
Markdown is ordinary text with simple conventions for headings, links and tables. A vertical bar separates table cells. When two source columns become one cell, the text no longer supplies the same mapping between a number and its label.
The finished result a researcher actually needs is smaller and clearer. For this row, a source-checked extract is:
| District | Total requirement | Kharif rice production | Rabi rice production | Total rice production | Rice surplus |
|---|---|---|---|---|---|
| Balasore | 3.47 | 2.78 | 0.86 | 3.64 | 0.17 |
All five quantities are in lakh tonnes, meaning units of 100,000 tonnes. Labels and values were checked against the rendered source. This is a manually reconstructed selection, not the converter’s output.
For one row, opening the PDF and copying the values is sufficient. Installing a converter becomes useful when you repeatedly need searchable text from documents. It does not remove the work of checking consequential tables.
Why this is worth checking now
Microsoft released MarkItDown 0.1.9 on October 21. Its changes include real conversion regression coverage and dependency updates. That makes a fresh run more informative than repeating an old warning that PDF tables are never supported.
Two distinct recent reports identify more specific problems. A September 28 report about rotated headers supplies this agricultural PDF and describes reversed words and merged cells in version 0.1.8. A separate October 2 reading-order report describes two-column prose becoming interleaved when another page triggers the converter’s form-processing route.
Those are user reports, not a representative failure-rate study. This investigation reproduced the agricultural sample on 0.1.9; it did not independently reproduce the second report’s mixed-document experiment. Neither report establishes how often your documents will fail.
The reader question is practical: what should you verify before letting an assistant answer questions from converted PDFs? A universal claim that extraction is broken would be as unhelpful as assuming that a successful command proves accuracy.
A PDF contains positions, not your interpretation
People see the agricultural table’s borders, grouped headings and vertical labels together. A program has to recover those relationships from the page’s stored content and positions.
The 0.1.9 PDF converter source uses pdfplumber for form-like content and pdfminer for a fallback extraction path. The implementation includes table-handling logic. Calling it simply a plain-text extractor would miss the current mechanism, while calling its output a verified table would overstate the result.
This matters for an assistant because readable-looking text can hide a structural error. A number may be present and searchable while its unit, column or qualifier is missing. A citation back to the PDF identifies the document. It does not establish that the extracted claim matches the page.
No language model was connected or evaluated here. The observed error belongs to document conversion. Whether a particular model would notice, correct or amplify it remains unmeasured.
The repair that works, and where it stops
The issue author suggested changing the direction used to read rotated characters. Running pdfplumber with char_dir_rotated="btt", meaning bottom to top, recovered both “Projected Population” and “Kharif” on this page.
That is a real improvement with a narrow scope. It is not a repaired MarkItDown table. The alternate extraction returns words, and readable words do not establish correct column boundaries, grouped headers or units. A document containing text rotated in another direction could need a different setting.
For the Balasore question, the dependable repair is to return to the rendered page and record the specific fields you need. Keep the original beside the extract. Do not globally reverse every strange word or ask an assistant to infer all the columns from the damaged Markdown.
You can copy this editable result into a text file named checked-row.csv:
district,total_requirement_lakh_tonnes,kharif_rice_lakh_tonnes,rabi_rice_lakh_tonnes,total_rice_lakh_tonnes,rice_surplus_lakh_tonnes
Balasore,3.47,2.78,0.86,3.64,0.17
It deliberately excludes population and paddy surplus, which use different meanings or units. Check that 2.78 plus 0.86 equals 3.64, and 3.64 minus 3.47 equals 0.17. Arithmetic is a useful second check, but the page establishes which quantities belong in those calculations.
Check your next document before asking for a summary
You can follow this route with a browser and a text editor, without Python, a ZIP or a paid model account:
- Open the source and choose one question whose answer depends on a table. For this sample: what is Balasore’s rice surplus, and how does it relate to production and requirement?
- Read the complete heading and unit on the page. Trace each needed value through its column, including any grouped heading.
- Compare those labels, values and relationships with the converted text. For the sample, reject the merged
3.47 2.78cell as an adequate structured answer. - Save the small checked extract above, with the source link, printed page 87 and historical period. Mark it as manually reviewed. A text file records your review; it does not perform one.
- If you later use an assistant, supply the checked extract and ask it to explain the calculation while retaining the source and units. Review the answer before sharing it.
You are finished when you can locate every selected value on the page, explain the two arithmetic checks and distinguish your corrected extract from raw conversion. For a larger document, repeat this on the actual tables that support the conclusion, rather than assuming one successful row validates the whole file.
Optional: reproduce the actual converter result
This terminal replay was executed with Python 3.12.14 on Linux. It needs an internet connection for installation and the public sample download. Runtime conversion uses a local file, disabled plugins and no broker, wallet, model or cloud-service credentials. Installing packages still uses the network. No graphical application is taught, so the evidence here is actual text output rather than an interface screenshot.
Create a clean folder and install the pinned packages:
mkdir pdf-conversion-check
cd pdf-conversion-check
python3 -m venv .venv
.venv/bin/pip install 'markitdown[pdf]==0.1.9' pdfplumber==0.11.10 pdfminer.six==20260107
curl --fail --location https://github.com/jsvine/pdfplumber/files/12135557/agstat.pdf --output agstat.pdf
Then run this complete diagnostic. The checksum identifies the exact public sample bytes; it does not certify the statistics. A changed file stops the check before conversion.
env -i PATH="$PWD/.venv/bin:/usr/bin:/bin" .venv/bin/python -I - <<'PY'
from pathlib import Path
from hashlib import sha256
from markitdown import MarkItDown
import pdfplumber
source = Path('agstat.pdf')
expected = '9c0a9e9afc63ffb01b80f57a72939c743f9e1ab138a38a07017de6fc80568650'
assert sha256(source.read_bytes()).hexdigest() == expected, 'Source bytes changed'
raw = MarkItDown(enable_plugins=False).convert_local(str(source)).markdown
assert 'noitalupoP detcejorP' in raw and 'firahK' in raw
assert '3.47 2.78' in raw and '0.86 3.64' in raw
Path('raw.md').write_text(raw, encoding='utf-8')
with pdfplumber.open(source) as pdf:
words = pdf.pages[0].extract_words(char_dir_rotated='btt')
readable = ' '.join(word['text'] for word in words)
assert 'Projected Population' in readable and 'Kharif' in readable
Path('rotated-words.txt').write_text(readable, encoding='utf-8')
print('SOURCE checksum=MATCH')
print('MARKITDOWN reversed_headers=OBSERVED merged_cells=OBSERVED')
print('DIRECTION readable_headers=YES table_reconstruction=NOT_VERIFIED')
print('FILES raw.md rotated-words.txt')
PY
Complete verified stdout:
SOURCE checksum=MATCH
MARKITDOWN reversed_headers=OBSERVED merged_cells=OBSERVED
DIRECTION readable_headers=YES table_reconstruction=NOT_VERIFIED
FILES raw.md rotated-words.txt
Run it again to repeat the observation. These two diagnostic files are overwritten on repeat; the source and your separate checked-row.csv are untouched. If a future converter fixes the sample, the assertions should fail rather than keep printing an obsolete finding. Do not remove them merely to obtain a passing message.
If installation reports a missing PDF dependency, confirm that the quoted [pdf] extra was installed into this environment. If the checksum differs, inspect the downloaded file and source URL. If output is empty on another PDF, check whether its pages are images: reading embedded text and optical character recognition, which recognizes text from images, are different operations. Paid OCR and cloud conversion options were not tested here.
What to carry into an assistant pipeline
MarkItDown’s current security guidance also distinguishes local files from its broader URL-capable conversion API. Restrict inputs and process permissions when building a service. The local replay is not a security audit or proof about hosted integrations.
This was one public page, one pinned converter run repeated cleanly, and a visual source check by the same AI authoring session. There was no independent beginner trial, model comparison, measured time saving or general accuracy estimate. Setup, review, repair and future rechecking remain part of the work even when no model API is billed.
For the next experiment, keep the same source and question, then compare another converter’s complete row-to-heading mapping against the page. Count repairs and missing units as well as successful text extraction. The useful result is a trustworthy answer to a document question, not the existence of a Markdown file.