Skip to content

Excel conversion replaces literal NA/NULL strings and blank cells with NaN #2498

Description

@pentaoa

Excel cells containing literal strings such as NA, NULL, None, n/a, and nan are converted to NaN. Real blank cells also appear as NaN. This loses distinctions in spreadsheet data, for example a region code NA versus an empty region.

Reproduced on main at eb31b5c9453628def5e6758a27a8e3a87b4ab101:

import io
from openpyxl import Workbook
from markitdown import MarkItDown

workbook = Workbook()
workbook.active.append(["region", "status", "empty"])
workbook.active.append(["NA", "NULL", None])
stream = io.BytesIO()
workbook.save(stream)
stream.seek(0)
print(MarkItDown().convert_stream(stream, file_extension=".xlsx").markdown)

The data row is | NaN | NaN | NaN |; it should be | NA | NULL | |.

Both XLSX and XLS converters use pandas' default missing-value recognition, which is useful for analysis but changes literal document text. keep_default_na=False preserves these strings and leaves blank cells empty. The XLSX fallback that repairs legacy showZeroes attributes also needs the same option.

I reproduced the failure for XLS, XLSX, and repaired XLSX and prepared a focused fix with regression tests. No model or remote service is needed. AI assistance: OpenAI Codex.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions