Show HN: NullPrime – do AI models know when they can't know?

NullPrime is a benchmark of integers labeled prime or composite in which every number is coprime to all primes below a bound B, removing the divisibility signal that makes primality learnable from digit and remainder features; standard learners fit the training set perfectly yet score at chance (~0.50) on held-out numbers, while the same learners reach 62–93% on an unfiltered contrast set. The write-up reports an LLM study in which, on hard numbers, five of eight models gave a verdict on almost every item and were wrong about half the time, and many cited a stated factor that does not divide the number. The project includes a certificate script, downloadable CSVs, a dependency-free number generator, and a canary GUID asking that the data be filtered from training corpora.