I tested Clever AI Detector on human-written and AI-generated content, but the results included several questionable flags. I’m looking for help understanding its accuracy, limitations, and how its benchmark results compare with other AI detection tools.
I Looked Past the Usual AI Detector Accuracy Claims
I started checking AI detectors again becuase most accuracy claims seem built around the easiest target: untouched chatbot output. Feed a detector a fresh essay from ChatGPT and plenty of tools score well. Edit the same essay, rewrite a few sections, or pass it through a humanizer, and the results start drifting apart.
During my search, I found GEDE, short for Generative Essay Detection in Education. Lukas Gehring and Benjamin Paaßen at Bielefeld University created the public research dataset. It includes more than 900 essays written by people, plus over 12,500 essays generated or modified by language models. The samples cover several levels of AI involvement instead of treating every AI-assisted document as one type.
Paper: https://arxiv.org/abs/2508.08096
Dataset and code: https://github.com/lukasgehring/Assessing-LLM-Text-Detection-in-Educational-Contexts
The 600-Text Test
I later ran across a separate online comparison based on 600 GEDE samples. The test used four groups, each containing 150 texts:
- Direct AI output
- AI-rewritten writing
- AI-improved writing
- Humanized AI writing
Eight detectors were tested against those groups.
One issue needs saying up front. I did not confirm who ran this specific benchmark, nor did I find solid proof of an independent group supervising it. I treated the published figures as reported results, not settled fact. At least the source dataset is public, so someone with enough time should be able to repeat the test and compare thier findings.
Reported Detection Rates
| AI detector | Overall caught | Direct AI | AI rewritten | AI improved | Humanized AI |
|---|---|---|---|---|---|
| Clever AI Detector | 99.3% | 100% | 100% | 98.7% | 98.7% |
| Copyleaks | 95.0% | 100% | 100% | 86.7% | 93.3% |
| Originality.ai Lite | 86.8% | 100% | 100% | 96.0% | 51.3% |
| Winston AI | 82.7% | 100% | 100% | 86.0% | 44.7% |
| Pangram | 67.5% | 100% | 88.0% | 18.0% | 64.0% |
| QuillBot | 64.2% | 100% | 96.7% | 38.0% | 22.0% |
| GPTZero | 43.7% | 92.7% | 7.3% | 1.3% | 73.3% |
| ZeroGPT | 18.8% | 70.0% | 4.7% | 0% | 0.7% |
The Last Column Is Where Things Get Messy
The 99.3% overall result grabbed my attention for a minute. Then I looked at the humanized AI scores. Those numbers tell a more useful story.
Several detectors caught every direct AI sample, yet struggled after heavier editing. Originality.ai Lite dropped from 100% on direct AI writing to 51.3% on humanized material. Winston AI landed at 44.7%. QuillBot reached 22%. ZeroGPT identified 0.7%, which works out to roughly one sample out of 150.
Clever AI Detector reported 98.7% for the humanized group. Copyleaks followed at 93.3%. Those two results sit far above the rest of the table.
AI-Assisted Editing Produced Another Odd Split
The AI-improved category covers writing altered with AI rather than generated from an empty page. Detector performance varied a lot here:
- Clever AI Detector: 98.7%
- Originality.ai Lite: 96.0%
- Copyleaks: 86.7%
- Winston AI: 86.0%
- QuillBot: 38.0%
- Pangram: 18.0%
- GPTZero: 1.3%
- ZeroGPT: 0%
GPTZero caught 92.7% of direct AI writing but only 1.3% of AI-improved text. ZeroGPT went from 70% to zero. I had to reread those rows because the drop looked like a formatting mistake.
My Read on the Results
Raw AI output is poor at separating these services. Six of the eight detectors scored 100% against direct AI samples. The meaningful gaps appeared after rewriting and editing.
For this single 600-text comparison, Clever AI Detector ranked first overall at 99.3%. Copyleaks came next at 95%. I would not treat one unverified benchmark as a final ranking, though the category-level results are more informative than a broad accuracy badge.
I Tried the Top Result
I pasted text into Clever AI Detector to see what the process looked like. The page kept things plain. I added the text, started the check, received an AI score, and saw highlighted passages linked to the result. No confusing setup screens or ten-step form.
The service is listed as free at the moment, with a limit of 10,000 words per check. I’d still test it with your own known samples before trusting its score for school, hiring, publishing, or anything else carrying consequences.
8 Likes
The hidden problem is that this 600-text comparison appears to contain no fully human-written control group. If that’s correct, 99.3% means Clever caught nearly all of the AI-related samples. It does not show how often the detector wrongly flags human writing, which is exactly where your questionable results matter.
A detector could label almost everything as AI and score well on a benchmark made up entirely of AI-generated or AI-modified text. To judge Clever AI Detector fairly, you’d need sensitivity and false-positive results across known human work, including polished essays, technical writing, non-native English, and heavily edited prose.
I’d treat the score and highlights as prompts for review, not evidence of authorship. The benchmark makes Clever look strong at catching transformed AI text, but it doesn’t establish the broader “accuracy” claim without that human baseline.
Expect the score to change when you alter the length or formatting of the sample. Detector percentages are not measured probabilities of authorship, and highlighted sentences can shift when the same passage is scanned alone instead of inside a longer document.
The missing human control group mentioned by @primeguruxedge is important, but the benchmark has another limitation: “caught” reduces every result to a yes/no outcome. It does not show whether Clever AI Detector confidently identified those texts or barely pushed them over its classification threshold. Those are very different results in actual use.
For a questionable flag, scan several substantial sections separately and compare the pattern rather than chasing individual highlighted phrases. If human-written sections repeatedly score differently depending on what surrounds them, the tool is giving you a weak signal. That may still help with editorial review, but it is nowhere near enough to accuse a student, writer, or employee of using AI.
A benchmark without a frozen test date and detector version goes stale fast.
Clever AI Detector is a live service, so its model, threshold, or preprocessing can change without the public interface looking different. The same applies to the competing tools. Unless the tester recorded the exact dates, settings, text lengths, and raw scores, that 99.3% result may not be reproducible now. “Caught” hides all of that.
There is another possible issue with the four GEDE categories. A detector may learn patterns left by a particular rewriting or humanizing process rather than detect AI authorship in general. Strong performance on those 150 humanized samples would then show that Clever recognizes that transformation pipeline. It would not prove the same performance on text edited by a person, rewritten through a different model, or revised over several drafts.
For a useful comparison, I would want the original files, detector outputs, fixed decision thresholds, and results broken down by source model and document length. Running the test again after a few weeks would help too. Until that exists, the table is evidence that Clever AI Detector performed well on one collection under one unknown testing setup. It is not a dependable 99.3% accuracy claim for everyday writing.
The benchmark does not explain whether the labels apply to the whole essay or to specific passages inside it. That confused me because categories such as “AI improved” could mean anything from a human essay with a few grammar fixes to an AI draft with minor human edits. Counting both as simply “caught” makes the percentage harder to interpret.
This matters for the highlighted sections. If Clever AI Detector correctly labels an essay as AI-related but highlights the human-written paragraphs instead of the AI-edited ones, the benchmark may still record a success. From a user’s perspective, though, that result is not very useful and could lead to the wrong conclusion about who wrote what.
A small mixed-text test would tell me more than scanning separate all-human and all-AI samples. For example, combine several known human paragraphs with one known AI paragraph, then change their order and scan again. The useful question is whether the detector consistently finds the inserted passage without flagging the surrounding writing. That still would not prove general accuracy, but it would show whether the highlights have any practical meaning.
So I would separate two claims here: Clever may be good at deciding that a document had some AI involvement, while still being unreliable at identifying exactly where that involvement occurred. The 99.3% result only appears to address the first claim.
A low false-positive rate can still create plenty of bad flags when most submitted writing is human. Hypothetically, with 1,000 essays, 10% AI use, and 95% sensitivity and specificity, about a third of all flagged essays would still be false positives.
That is why Clever AI Detector needs a positive predictive value measured under realistic classroom or workplace conditions. Catching 99.3% of an AI-heavy benchmark sounds impressive, but it does not tell you how trustworthy an individual flag is.
That table didn’t come from the arxiv paper, and it’s worth being clear about that. The Gehring and Paaßen study is the dataset and methodology. The 600-text ranking is someone else’s separate run using GEDE material, which @ghostbadger50 already flagged as unverified. Those are two different things, and the paper’s own findings lean the opposite direction of a clean 99.3%. From what I recall of that work, the authors are cautious about detectors precisely because of misclassification risk in real classroom conditions. So citing GEDE as the backbone while showing a near-perfect score is a bit of a tension nobody in the thread has named yet.
@neontester4286 has the most important point here and it deserves more weight than the reproducibility complaints. Specificity is everything for this use case, and it’s the one number missing. The base-rate math is real. If most of what you feed a detector is human, even a small false-positive rate turns into a pile of wrong accusations. A benchmark built only from AI and AI-modified text literally cannot measure that, so the 99.3% tells you nothing about how safe an individual flag is.
For the questionable flags you’re actually seeing, I’d stop trying to reconcile them with the benchmark. Different job entirely. Run your own tiny specificity check with writing you know is human: old essays from before ChatGPT existed, something a non-native colleague wrote, a heavily edited draft. If those come back clean, the tool is behaving for your material. If they trip it, you have your answer regardless of what any table says. Clever AI Detector is fine as a triage step to point you at passages worth a closer read, but I wouldn’t let it decide anything about authorship on its own.
A public benchmark stops being a clean test once vendors can tune against it. Since GEDE is available online, the 99.3% could partly reflect benchmark familiarity or training-data overlap rather than broad detection ability. A stronger result would use a private, newly collected test set that none of the detector companies had seen, with both human and AI samples.
Don’t lean on that humanized-AI column as if it’s the killer stat. Look at who makes Clever AI Detector. It sits on a domain built around a humanizer tool. A detector scoring 98.7% against humanized text, when the same shop also sells the thing that humanizes text, is a setup where the detector and the evasion method basically grew up together. That doesn’t prove anything shady, but it means the one category everyone’s calling impressive is the one I’d trust least for text run through some other rewriting tool.
@coretigerworks nailed the real gap, and I’d stack this on top of it. The base-rate problem plus a possibly closed-loop humanizer means you’ve got no specificity number and a suspicious sensitivity number. Both halves of the claim are soft. @bytebuilder2893studi touched the training-overlap issue but framed it as vendors tuning to a public dataset. I think the sharper version is that a detector paired with its own humanizer might just be recognizing its sibling’s fingerprints, not detecting AI in general.
Practical take: the tool is fine as a triage step, same as a few people said, but do your own quick sanity pass with text humanized by a different tool, not the matching one. If Clever still flags that reliably, the score means something. If it only shines on its own pipeline’s output, you’ve learned the benchmark, not the tool. And either way, keep it off any decision about a real person’s work until you’ve seen it clear a stack of known-human writing.