[cyberspace-cs]

What

WordNumberFilter guards every cell with if text:. That guard is False for None, but True for NaN, because bool(float("nan")) is True. So a single null cell raises

AttributeError: 'float' object has no attribute 'split'

and aborts the whole run.

Why this is reachable in normal use

A null cell in the text column is routine:

• an upstream generator that returns nothing for a row,
• a left join / merge that does not match,
• a jsonl written from a None field and read back as NaN.

import pandas as pd
pd.DataFrame({"text": ["one two three", None, float("nan"), "four five"]})["text"]
# -> a str column containing NaN; iterating yields "one two three", nan, nan, "four five"

The traceback points at the tqdm loop, so the offending row is not obvious.

Fix

Count any non-string cell as zero words. That is already what the else branch intended (word_counts.append(0)) — the bug is only that NaN never reaches it. The counting is factored into _count_words() so the contract can be tested on its own, and run() keeps the same behaviour for every string cell.

While in the file I also aligned the get_desc() defaults with the constructor: the docs said min_words=5 / max_words=100, the constructor uses 20 / 100000.

Tests

test/cpu_only/test_word_number_filter_missing_values.py — 10 cases, all passing:

• None / NaN cells are counted as 0 and the run completes,
• rows outside the range are dropped and the original order is preserved,
• _count_words on None, NaN, pd.NA, 42, "", " ", "one", "one two three".

$ pytest test/cpu_only/test_word_number_filter_missing_values.py -q
10 passed

I confirmed the two integration cases fail on main with the AttributeError above and pass with the patch.

Scope note

BlocklistFilter.run() and the ~19 loops in rule_based_filter.py use the same if text: guard, so they fail the same way on a NaN cell. I kept this PR to one operator to keep review small — happy to send the same treatment as a follow-up, or to widen this one if you prefer a single sweep.

One more thing I noticed but did not change, because it is a behaviour change rather than a crash: min_words is inclusive (<=) while max_words is exclusive (<), so max_words=100 drops a text with exactly 100 words. Worth aligning if you agree it is unintended.