AI Thesis Review — faster, fairer academic evaluation

Grading a thesis is slow, and it is rarely as consistent as everyone pretends. Two evaluators read the same document and land on different scores; a single evaluator scores the fortieth submission differently than the fourth. AI Thesis Review is an AI-enhanced submission and evaluation platform for universities that attacks both problems at once — it makes the read faster and it makes the scoring evidence-backed. You can try the live system at the-pace-system.vercel.app.

The problem

Thesis evaluation is a high-stakes, low-throughput task. An evaluator has to read a long document, judge it against a rubric, check whether the contribution is actually novel relative to prior work, and then justify a score. In practice that means hours per thesis, and the justification step — the part that makes grading fair — is usually the first thing to get compressed under a deadline.

Two failure modes fall out of that. First, speed: departments sit on backlogs because reading is serial and human. Second, consistency: without a shared, referenceable basis for each score, the same work gets graded differently depending on who reads it and when. I wanted the system to address the second problem while solving the first — faster review that is also more defensible, not faster review that cuts corners.

Architecture

The core is a pipeline. A thesis comes in, gets chunked and embedded, each chunk is matched against a corpus of prior submissions and published work using vector search, and those neighbors become the evidence an LLM uses to score the thesis against the rubric. The output is a structured report an evaluator reads and signs off on — the human stays in the loop, but the reading and the citation legwork are done for them.

Submission intake

Intake is a Django service backed by PostgreSQL. It handles upload, parsing, and normalization — turning a raw document into clean text and the metadata the rest of the pipeline needs. Every thesis and its evaluation state lives in Postgres, which keeps the system of record boring and auditable and leaves the vector store to do only what it is good at.

Embedding and similarity search

Each submission is split into chunks and embedded, then queried against a vector index of prior work. I used Pinecone for the managed, always-on index and FAISS where a local, in-process index made more sense — same embeddings, two backends for two deployment shapes. The point of this stage is retrieval: given a section of a new thesis, pull back the closest passages from everything the system has already seen.

Rubric scoring

This is where retrieval pays off. Instead of asking an LLM to grade a thesis cold, I hand it the rubric criteria plus the retrieved neighbors, and score criterion by criterion. Because the model is reasoning over concrete prior work rather than its own vague priors, a novelty judgment — for example — is tied to the actual passages it resembles. Every score comes back with the evidence that produced it, which is what makes the result something an evaluator can defend rather than just accept.

for criterion in rubric.criteria:
    neighbors = index.query(embed(thesis_chunks), top_k=8)
    result = llm.score(
        criterion=criterion,
        thesis=thesis_text,
        evidence=neighbors,          # retrieved prior work
    )
    report.add(criterion, result.score, result.rationale, neighbors)

Evaluator report

The pipeline ends in a report, not a verdict. For each rubric criterion the evaluator sees the suggested score, the rationale, and the specific prior work that grounds it — served through the Next.js front end. They can accept, adjust, or override. The system does the reading and the cross-referencing; the human keeps the authority. That division is the whole design: it is what lets the tool be fast without pretending it is the final judge.

Outcomes

The headline result is throughput. Review and evaluation time dropped by roughly 93% — the hours-long serial read collapses into a report an evaluator reviews and signs off on. Just as important, scoring got more consistent: because every score is anchored to retrieved evidence and a shared rubric, two evaluators are working from the same basis instead of their own gut, and the justification that used to get squeezed out under deadline is now produced by default.

• ~93% less review/evaluation time per thesis.
• Evidence-backed scores — every rubric judgment cites the prior work it rests on.
• More consistent grading across evaluators, driven by a shared rubric and shared evidence.
• Human-in-the-loop by design: the evaluator always has final say.

Stack

• Next.js — evaluator-facing front end and report UI.
• Django — submission intake, pipeline orchestration, and APIs.
• Pinecone — managed vector index for similarity search.
• FAISS — local vector search where an in-process index fit better.
• LLMs — rubric-based scoring and rationale generation over retrieved evidence.
• PostgreSQL — system of record for theses and evaluation state.