# AI-Examiner — Requirements Specification | Item | Value | |---|---| | Product | AI-Examiner (ShikkhaNet Technologies) | | Document version | 2.1 (supersedes 2.0) | | Date | 19 October 2026 | | Release target | v1.0 (school-level SaaS pilot), with v1.1 items identified | | Status | Ready for solution architecture | > **How to use this document.** This is a **requirements specification**. It states *what* the system must do, the rules it must follow, the data it must keep, and how success is measured. It does **not** prescribe architecture, frameworks, cloud vendors or AI models; decisions left to the solution architect are marked **[Architect decision]**. Keywords: **MUST** (mandatory for the release stated), **SHOULD** (expected unless there is a justified reason not to), **MAY** (optional). Every requirement has an ID. Each requirement is tagged **v1.0** or **v1.1**; untagged requirements are v1.0. ### Changes from v2.0 - Added an explicit v1.0 / v1.1 release split (§0). - Removed the Local Pre-processing Agent; image optimization happens in the browser at upload (§8). - Script identification simplified to one PDF (or image set) per student, named by roll number (§9). - Page processing is one AI pass per page (classification, extraction, question labeling, rough-work detection, legibility rating) (§10). - Scoring is one AI call per script by default (all answers, all rubrics, script feedback), not one call per scoring unit (§12). - Art elements simplified: LaTeX for equations; description + bounding box for diagrams, graphs and tables; no stored crop images (§10.2). - Position data at line level only; PDF annotations are line-keyed with remarks in the margin (§10.3, §14). - Rubric authoring: upload-and-structure of the teacher's question paper and model answers is the primary path; ShikkhaNet shared question bank added (§7). - Biddaloy integration reduced for v1.0 to SSO and student roster; catalog seeded; results via CSV export; API push in v1.1 (§6). - Mark step default 0.5 (0.25 configurable); fabrication/show-through "zero tolerance" restated as acceptance-test criteria (§17.2). - Added AI cost ceiling (§17.1), operations and monitoring (§17.9), batch analytics (§13.3), notifications (§13.4). - Deferred to v1.1: calibration anchors, outlier detection, consistency re-scoring sample, per-answer handwriting rating, detailed-reasoning PDF appendix, AI drafting of rubrics from scratch, results API push, image-set uploads beyond PDF. --- ## 0. Release Scope | Area | v1.0 (launch) | v1.1 (follow-on) | |---|---|---| | Tenancy & access | School = tenant; SSO from Biddaloy; scope enforced | — | | Biddaloy data | Student roster per class/section; seeded academic catalog | Live catalog sync; results API push; exam list sync | | Results to Biddaloy | CSV/Excel export in Biddaloy's import format | API push with acknowledgment | | Rubric authoring | Upload question paper + model answers → AI structures → teacher approves; manual editor; copy from ShikkhaNet question bank or own earlier set | AI drafting of model answers from scratch; spreadsheet import | | Upload | One PDF per student named by roll; browser-side compression | Image sets (JPEG/PNG) per student | | Extraction | One pass per page: classification, verbatim text, labels, equations (LaTeX), descriptions of figures, legibility rating, line-level boxes | Word-level boxes (if a reliable method is proven) | | Scoring | One call per script; five factors; guardrails in code; brief + detailed reasoning; script strengths/weaknesses/suggestions | Calibration anchors; outlier detection; consistency re-scoring sample; per-answer handwriting rating | | Review | Flags queue; corrections; overrides (optional reason); finalize/reopen; full audit | — | | PDF | Cover summary + annotated pages (line-level ticks/crosses/circles, margin remarks, per-answer marks); Provisional watermark | Detailed-reasoning appendix | | Analytics | Per-batch question averages, most-missed key points, common errors | Cross-batch and cross-year trends | | Credits | Metering, estimate, reservation, usage reports | — | | Operations | Monitoring, alerts, processing dashboard, ShikkhaNet quality sampling | — | --- ## 1. Purpose and Objective AI-Examiner reads scanned handwritten answer scripts, scores each answer against teacher-provided questions, model answers and marking rules, explains every score, and produces an annotated PDF of each script that looks like it was marked by a human examiner. Version 1 is used by **individual schools**. Each school is a tenant of a SaaS platform. **Biddaloy** (ShikkhaNet's school management platform) remains the system of record for users, permissions, students and result processing. | # | Job | Summary | |---|---|---| | J1 | Extraction | Read every page; extract text **exactly as written** plus equations and descriptions of figures; label content by question; store each answer against its scoring unit. | | J2 | Scoring | From extracted content and the reference context, score each answer under the configured rules, give a brief and a detailed reason, and produce strengths, weaknesses and improvement suggestions for the script. | | J3 | Annotated PDF | One PDF per script: original pages with errors marked, examiner remarks and marks, plus a summary cover page. | --- ## 2. Glossary | Term | Meaning | |---|---| | Tenant | One school. All data is isolated per tenant. | | Academic Catalog | Master list of Education System → Class → Subject → Syllabus (books/chapters), owned by ShikkhaNet/Biddaloy, seeded into AI-Examiner. | | Exam | An examination event (e.g., "Half-Yearly 2026"), named by the teacher in v1.0 (synced from Biddaloy in v1.1). | | Question Set | The full question paper for one Exam + Class + Subject: sections, questions, sub-questions, model answers, marking rules. | | Scoring Unit | The smallest item that receives a mark: a question without sub-parts, or a sub-question (e.g., creative question part গ). | | Exam Batch | One upload job: all scripts of one Question Set (optionally limited to sections). Up to 500 scripts. | | Script | All pages written by one student for one Exam Batch. | | Page | One image of one side of one sheet. | | Element | One extracted item on a page: a text line, an equation, a figure description, a table, a label. | | Answer Segment | The elements (possibly across pages) that answer one Scoring Unit. | | Factor | One of five scoring dimensions: Correctness, Completeness, Spelling, Grammar, Handwriting. | | Rubric | Model answer + key points + marking instructions + factor weights for a Scoring Unit. | | Step | Smallest allowed mark increment: 0.5 (default), 1 or 0.25. | | Credit | Billing unit consumed by processing (§15). | | Override | A teacher change to an AI-produced value. | | Finalized | A script whose scores have been approved and released for export/push. | --- ## 3. Scope ### 3.1 In scope - Multi-tenant SaaS, each school a tenant. - Biddaloy SSO and student roster (§6). - Reference context management (§7), including the ShikkhaNet shared question bank. - Browser upload with compression (§8); script identification (§9). - Single-pass extraction with question labeling (§10); answer assembly and mapping (§11). - Scoring with reasoning and script feedback (§12). - Review queue, overrides, finalization, audit, batch analytics, notifications (§13). - Annotated PDF (§14). Credit metering (§15). Operations (§17.9). ### 3.2 Out of scope - User roles, user management and permissions (Biddaloy provides them). - Result processing: grades, GPA, pass/fail, merit lists, report cards. - Grading systems; AI-Examiner outputs **marks only**. - MCQ / OMR questions. - Scanning hardware, scanning software and the scanning process. - Any locally installed software at the school. - Student or guardian access. - Payment collection and invoicing. - National/board-scale volumes; the design SHOULD NOT prevent future scale-up, but v1 need not meet it. - Plagiarism/copying detection. --- ## 4. Actors Roles are not designed here; Biddaloy decides which users map to which actor. | Actor | Does | |---|---| | Platform Operator (ShikkhaNet) | Maintains catalog seed, question bank, global defaults, AI configuration, credit rates, quality sampling. | | School Admin | Tenant settings (feedback language, review mode, retention, PDF options). | | Teacher | Creates Question Sets, uploads scripts, reviews flags, overrides, finalizes, downloads PDFs and exports. | | Biddaloy | Identity provider, student roster source, results consumer. | --- ## 5. Reference Context Hierarchy 1. **Education System** — BISE (General/National Curriculum); English Medium (CAIE, Pearson Edexcel, IB, Oxford International AQA); Madrasha Education Board (Dakhil/Alim, all boards in the catalog); Technical Education Board. 2. **Class / Level** — Class 5 to 12 or equivalent level names. 3. **Subject** — e.g., "Bangla 1st Paper", each mapped to a subject category (FW-04). 4. **Syllabus** — books and chapters covered. 5. **Question Set** — sections, choice rules, max marks, total target marks, pass mark (informational). 6. **Scoring Unit** — question/sub-question text, stimulus, model answer, rubric, max mark, factor weights. - **CTX-01** The complete relevant context (levels 1–6) MUST be available to scoring so that expected depth and standard match the education system and class. - **CTX-02** The Syllabus selection MUST be available to scoring. --- ## 6. Biddaloy Integration Mechanism is an **[Architect decision]**; data and behavior are mandatory. - **INT-01** Users reach AI-Examiner through Biddaloy (SSO/token hand-off). AI-Examiner MUST NOT keep its own passwords. - **INT-02** Biddaloy supplies per session: tenant ID, user ID, display name, and permitted scope (classes/subjects). AI-Examiner MUST enforce the scope and record the user ID on every audited action. - **INT-03** All data access MUST be tenant-isolated at every layer, UI and API. - **INT-04** **Student roster:** AI-Examiner MUST fetch students of a class/section (student ID, roll, name, class, section, group). Roll MUST be unique within Class + Section (+ Group); otherwise Biddaloy supplies a unique identifier used in file names. - **INT-05** **Academic Catalog (v1.0):** seeded into AI-Examiner from a ShikkhaNet-provided export; editable only by the Platform Operator. **(v1.1)** Live sync from Biddaloy. - **INT-06** **Exams (v1.0):** created by the teacher inside AI-Examiner (name, academic year). **(v1.1)** Synced from Biddaloy with Biddaloy exam IDs. - **INT-07** **Results hand-off (v1.0):** export of finalized results per batch as CSV/Excel in the layout Biddaloy's result import accepts: student ID, roll, exam, class, section, subject, per-question marks, total obtained, total target, override indicator, finalized date. **(v1.1)** API push, retried until acknowledged, idempotent, with revisions on reopen. - **INT-08** AI-Examiner provides its own web UI (opened from Biddaloy) and an API covering the same functions. - **INT-09** UI in Bengali and English. --- ## 7. Reference Context Management ### 7.1 Question Set - **QS-01** Teacher creates a Question Set for Exam + Class + Subject, selecting Syllabus books/chapters. - **QS-02** Holds: total questions, sections, per-question max mark, total target marks, pass mark (informational), Step (default 0.5; 1 or 0.25 allowed), rounding mode, content-gate flag, choice-rule mode. - **QS-03** Sections with "answer any N of M" rules MUST be supported. - **QS-04** Validation: maximum attainable total (compulsory questions + N highest max marks per choice section) MUST equal total target marks before status "Ready". - **QS-05** Versioned. Once scoring starts on any script the version is locked; later changes create a new version and require re-scoring of affected Scoring Units. Extraction is never repeated because of a rubric change. - **QS-06** Copy from: the teacher's own earlier Question Sets, or the **ShikkhaNet question bank** (QS-10). ### 7.2 Authoring paths - **QS-07** **Primary path — upload and structure:** the teacher uploads the question paper and the model answers/marking scheme as PDF, images or Word. The system MUST structure them into sections, questions, sub-questions, max marks and model answers, present the result for approval, and let the teacher edit any item. Unresolved items (e.g., a question without a model answer) MUST block "Ready". - **QS-08** **Manual editor:** full create/edit of every field, including LaTeX and images for model answers. - **QS-09** **(v1.1)** AI drafting of model answers/key points from scratch (clearly labeled, teacher approval required). **(v1.1)** Spreadsheet import template. - **QS-10** **ShikkhaNet question bank:** the Platform Operator MUST be able to publish Question Sets (e.g., board model questions, common school exam papers) visible to all tenants of the same education system. Tenants copy a published set into their own tenant and may edit the copy; the published original is never changed by tenants. Tenants MAY submit their sets to ShikkhaNet for publishing (manual approval). ### 7.3 Question structure Supported types (no MCQ): creative/structured (stimulus + sub-parts ক/খ/গ/ঘ, Appendix A), short answer, essay/composition, translation, grammar exercise, mathematical/numerical (method marks), proof/derivation, diagram/graph/figure-based, chemical equations, Arabic/religious text. - **QS-11** Each question stores: printed label, alternative labels students may write (teacher-entered or system-generated and reviewed), type, text, optional stimulus (text/image), sub-parts (one level), max mark per Scoring Unit, expected answer language(s), optional cognitive level. ### 7.4 Rubric per Scoring Unit - **RB-01** Model answer MUST exist (text, LaTeX, optional reference image). - **RB-02** Key-point list MAY exist (description, mark/weight, mandatory flag); for numerical questions key points are method steps and final answer. - **RB-03** Free-text marking instructions MAY exist. - **RB-04** Acceptable alternatives and common mistakes MAY be listed. - **RB-05** With only a model answer, scoring MUST note in reasoning that the rubric was minimal. ### 7.5 Factor weights - **FW-01** Five Factors, percentage weights summing to 100%; 0% disables a factor. - **FW-02** Levels, lower wins: platform default per subject category → Question Set → Scoring Unit. - **FW-03** Platform defaults: | Subject category | Correctness | Completeness | Spelling | Grammar | Handwriting | |---|---|---|---|---|---| | Language & literature | 50% | 25% | 10% | 10% | 5% | | Humanities, social science, religion, science theory | 60% | 25% | 5% | 5% | 5% | | Mathematics and numerical | 70% | 25% | 0% | 0% | 5% | - **FW-04** Every Subject in the catalog maps to a subject category (Platform Operator). --- ## 8. Upload and Image Optimization (browser-side) No software is installed at the school. Scanning is out of scope; the school produces files with its scanner. - **UP-01** Supported inputs: **PDF** (one per student; v1.0), and **(v1.1)** JPEG/PNG image sets. Maximum 40 pages per script, 500 scripts per batch (Platform Operator configurable). - **UP-02** The browser MUST compress pages before upload to a configurable target (default about 300 KB per page) while keeping a minimum effective resolution (default 200 DPI for A4; never below 150 DPI). Readability has priority over size. Compression MUST be deterministic and non-AI. Where browser capability is insufficient (very large files), server-side optimization MUST apply the same rules. - **UP-03** Upload MUST support multi-file selection and drag-and-drop of up to 500 files, resume after interruption, per-file progress, and server-side checksum verification. - **UP-04** Duplicate uploads (same checksum within a batch) MUST be rejected with a message. - **UP-05** Page order is the order inside the PDF (or natural filename sort for image sets). Teachers can reorder (UP-10). --- ## 9. Script Identification and Page Classification - **UP-06** **Identification:** file name = `` or `_`, e.g., `23.pdf` or `২৩_রহিম.pdf`. Bengali and ASCII digits are equivalent. - **UP-07** Rolls MUST be matched to the roster for the batch's class/section. The upload screen MUST show: matched, unmatched files, duplicate rolls, and roster students with no file (recorded "Not uploaded", not scored). - **UP-08** If a cover page is detected, the roll read from it (§10) is compared with the file name; a mismatch raises a flag. The file name is used until the teacher resolves it. - **UP-09** Page classification (part of the single extraction pass, §10): Cover, Answer, Supplementary, Rough work, Blank, Unreadable. - **UP-10** Teacher MUST be able to reassign a script to a student, reorder/move/delete pages, add missed pages, and reclassify pages. All audited. - **UP-11** Blank detection MUST be conservative (any faint writing ⇒ not blank). Blank pages are not charged and get no further AI processing. **[Architect decision]** whether a cheap non-AI blank check runs before the AI pass. - **UP-12** Rough work is extracted but excluded from scoring by default (teacher may include). - **UP-13** Unreadable pages are flagged for re-upload and never silently scored as "not attempted". --- ## 10. Extraction (J1) — single pass per page ### 10.1 One pass - **EX-00** For each non-blank page, one AI pass MUST produce all of: page classification; verbatim text by line; question/sub-question labels found; equations; figure/table descriptions; struck/inserted flags; rough-work detection; uncertain/illegible markers; cover-page fields (roll, registration, subject code) where present; a page legibility rating (Appendix B); language tags; line bounding boxes; confidence. Separate AI calls for these outputs are not permitted in v1.0. **[Architect decision]** whether the pass is given the previous page's tail for continuity. ### 10.2 Fidelity - **EX-01** Bengali, English, Arabic, mixed within page, line or word sequence. - **EX-02** **Verbatim:** exactly as written, errors preserved; no correction, normalization or completion. Bengali conjuncts, vowel signs, hasanta and Arabic diacritics preserved as written. - **EX-03** **Digits:** highest accuracy, including similar characters (১/২, ৪/৮, ০/0/O, Arabic-Indic digits); stored in the script the student used. A misread number is a critical defect. - **EX-04** **Isolated labels** (ক, খ, গ, ঘ, ঝ, ট, ঠ, A, B, x, y …) in figures, geometry and sub-question headings MUST be separate label elements, never merged into words. - **EX-05** Struck-out text: extracted with "struck" flag, excluded from scoring by default. - **EX-06** Insertions (caret/arrow): placed at intended position, flagged "inserted". - **EX-07** Arabic/RTL stored in logical order; mixed runs ordered correctly. - **EX-08** Faint writing, inconsistent letterforms, smudges, ruled lines and show-through handled; show-through MUST NOT be extracted as content. - **EX-09** **Uncertainty:** never present a guess as certain. Mark spans uncertain (up to three candidate readings) or illegible (with position). Markers are visible to scoring and teacher. - **EX-10** Printed booklet content (headers, instructions) excluded from answer content. ### 10.3 Art elements (simplified) | Element | Stored as | |---|---| | Equation / mathematical expression | LaTeX plus verbatim linear text; student errors preserved | | Chemical equation / formula | Text with LaTeX where helpful (no structure parsing) | | Diagram / drawing / figure / construction / graph / chart | Bounding box + textual description of what is actually drawn: components, labels present (as label elements), connections, axes/units/scale, measurements stated | | Table | Rows/columns with verbatim cells | | Student's own underline/box | Attribute on the text line | - **EX-11** Descriptions MUST describe what the student drew, not the correct figure; missing/wrong labels reported as observed. No crop images are stored in v1.0; the PDF stage and the review UI render regions from the page image using the bounding box. ### 10.4 Position, order, confidence - **EX-12** Every element stores: page, **line-level** bounding box (normalized 0–1), reading order, language tag(s), type, confidence, flags, candidate readings. Word-level boxes are **not** required (v1.1 only if a reliable method is proven). - **EX-13** Original AI extraction is immutable; teacher corrections are a separate layer with user and time. - **EX-14** Provider, model, version and tokens recorded per page. ### 10.5 Review mode - **EX-15** Tenant setting: **Automatic** (default; scoring runs, low-confidence items flagged) or **Review required** (scoring waits per script until flags are cleared). - **EX-16** Flagging confidence threshold configurable (platform default from pilot calibration). --- ## 11. Answer Assembly and Mapping (part of J1, non-AI where possible) - **SG-01** Using the labels produced by the extraction pass and the Question Set labels/alternatives, the system MUST assemble Answer Segments per Scoring Unit. This step SHOULD be deterministic code; an AI call is permitted only for scripts where labels are ambiguous (flagged by confidence). - **SG-02** Answers may be in any order, span pages, and be interrupted and resumed ("continued on page 9"). - **SG-03** Missing sub-labels under a clear question heading MAY be mapped with recorded confidence; low confidence flagged. - **SG-04** Same Scoring Unit answered twice: use the non-struck attempt; otherwise flag, score both, keep the higher by default (configurable: first attempt). - **SG-05** Unmappable content is "Unmapped" and flagged; teacher assigns. - **SG-06** Choice rules: more than N answered ⇒ all scored, best N counted by default (configurable: first N). Not-counted answers shown as such in PDF and reasoning. - **SG-07** No content ⇒ "Not attempted", score 0, no AI scoring. - **SG-08** Output per Scoring Unit: ordered element IDs, assembled verbatim content, mapping confidence, flags. --- ## 12. Scoring (J2) — one call per script ### 12.1 Call granularity - **SC-00** Default: **one AI call per script** receives all Answer Segments of the script with the full Question Set rubric and returns scores for every Scoring Unit plus the script-level feedback (SC-19). Configurable alternative: one call per question (all its sub-parts). Per-Scoring-Unit calls are not permitted except for re-scoring a single unit after an override/correction. **[Architect decision]** on splitting very long scripts into two calls while keeping each question's parts together. ### 12.2 Inputs - **SC-01** Extracted content (text, LaTeX, figure descriptions), context hierarchy, rubric, factor weights, Step, feedback language, page legibility ratings. - **SC-02** Scoring works from extracted content, not images. Exception: for diagram/graph/figure-based Scoring Units the page region (from the bounding box) MAY be attached. - **SC-03** Personal identifiers MUST NOT be sent to AI scoring; scripts use an anonymous ID; cover-page regions are never sent. - **SC-04** Uncertain/illegible spans: student not penalized for unreadable content; unit flagged when uncertainty could move the score by one Step or more. ### 12.3 Factor ratings (0–1 per Scoring Unit) | Factor | Definition | Method | |---|---|---| | Correctness | Accuracy of what the student stated/did vs rubric | AI judgment; with key points, weighted share correct, reduced by incorrect content | | Completeness | Coverage of key points, steps, parts, depth for class, labels, units | AI judgment; with key points, weighted share present; missing mandatory points listed | | Spelling | Spelling in expected language(s) | Error count; rating = max(0, 1 − errors × penalty), default 0.1; repeated identical misspelling counts once | | Grammar | Grammar, structure, punctuation, সাধু-চলিত consistency where required | Error count; same formula, default 0.1 | | Handwriting | Legibility and neatness | **Script-level** rating = average of page legibility ratings from extraction (0–10 → 0–1), applied to every Scoring Unit. No separate AI call. **(v1.1)** per-answer rating | - **SC-05** Spelling/grammar errors listed individually (as written, correction, line reference, type). Rubric exceptions (proper nouns, terms) not counted. - **SC-06** Text in a non-expected language is not a spelling error unless the rubric says so; the rubric MAY reduce Correctness for it. - **SC-07** Bengali, ASCII and Arabic-Indic digits numerically equivalent unless rubric says otherwise. ### 12.4 Computation and guardrails - **SC-08** Raw score = Max mark × Σ (weightᵢ × ratingᵢ). - **SC-09** Content gate (default ON, per Question Set): Correctness = 0 ⇒ score 0. - **SC-10** Missing mandatory key points MUST be reflected in ratings and named in reasoning. - **SC-11** Rounded to Step; rounding mode: nearest (default, ties up), down, up. - **SC-12** **Guardrails in code (not AI):** 0 ≤ score ≤ unit max; question ≤ question max; script total ≤ target total; enforced on every AI result and every override. - **SC-13** Ratings, counts, weights, raw and final scores all stored for recomputation. ### 12.5 Reasoning - **SC-14** **Brief explanation** ≤ 150 user-perceived characters (grapheme clusters); used as the PDF margin remark. - **SC-15** **Detailed explanation:** correct content; wrong content; missing key points/steps; spelling/grammar notes; handwriting (if weighted); how factors combined; rules applied. - **SC-16** Reasoning references the student's actual content; never claims content not in the extraction. - **SC-17** **Error items:** type (spelling, grammar, factual, wrong method/step, calculation, missing key point, missing unit, wrong/missing label, irrelevant, not counted), **line reference**, remark ≤ 60 characters, optional correction. - **SC-18** **Positive items** (correct key points/steps) with line references for ticks. ### 12.6 Script-level feedback (same call) - **SC-19** Strengths (3–5), weaknesses (3–5), suggestions (3–5, concrete, tied to syllabus chapters where relevant), overall summary ≤ 300 characters. - **SC-20** All reasoning in the tenant feedback language (Bengali or English); quoted student content and language-subject corrections stay in their original language. ### 12.7 Consistency - **SC-21** Same rubric applied identically across the batch; consistent criteria and wording. - **SC-22** Determinism target: re-scoring identical input yields the same mark in ≥ 95% of units, within one Step otherwise (validated per §19). - **SC-23 (v1.1)** Consistency re-scoring sample (default off; when on, 5% of units, min 10, flag > one Step difference). - **SC-24 (v1.1)** Calibration anchor scripts. **SC-25 (v1.1)** Outlier detection within a batch. ### 12.8 Outputs and re-scoring - **SC-26** Per Scoring Unit: score, max, ratings, weights, counts, error items, positive items, brief, detailed, flags, model info, Question Set version, timestamps, share of script tokens. Example: Appendix C. - **SC-27** Re-score a unit, question, script or batch reusing stored extraction; history kept. - **SC-28** Overridden units are not overwritten by re-scoring unless the teacher chooses. --- ## 13. Review, Override, Finalization, Audit, Analytics, Notifications ### 13.1 Review and override - **RV-01** Review queue per batch: unidentified/mismatched scripts, unreadable pages, low-confidence extraction, uncertain spans affecting score, unmapped content, duplicate answers, failed processing. Each flag: type, severity, link to page/line, status. - **RV-02** Side-by-side view: page image with line overlays, extracted content, scoring results. - **RV-03** Teacher can correct extraction, reassign segments, override scores (respecting Step and guardrails), edit explanations/feedback, add/edit/remove annotation items. **Override reason is optional.** - **RV-04** Original AI values preserved; latest teacher value is displayed, used in PDF and export. - **RV-05** PDF regenerated after any affecting change (new version number). ### 13.2 Finalization and audit - **RV-06** Finalize per script or batch; blocked while blocking flags are open (default blocking: unidentified script, unreadable page, unmapped content). Finalized scripts are read-only until reopened; reopening and later changes are audited and included in the next export (v1.1: resent via API). - **RV-07** **Audit log (append-only):** every create/update/delete/override/re-score/finalize/reopen/export/download and every AI call: tenant, user or system, time, entity, before/after, optional reason, Question Set version, and for AI calls provider/model/version/tokens/outcome. - **RV-08** Full history of any Scoring Unit viewable in UI. ### 13.3 Batch analytics - **AN-01** After scoring, per batch: average and distribution per question and Scoring Unit; attempt rate; most-missed key points; most frequent spelling/grammar errors; count of flags and overrides. Exportable as CSV/Excel. Computed from stored results, no AI. ### 13.4 Notifications - **NT-01** In-app and email notification when: upload complete with identification issues; batch scoring complete; flags awaiting review; processing failure; credits below a threshold. Email delivery mechanism **[Architect decision]**. --- ## 14. Annotated PDF (J3) - **PDF-01** One PDF per script; bulk ZIP per batch. Naming `____.pdf` (configurable; unsafe characters replaced). - **PDF-02** **Cover page:** school, exam, class/section, subject, student name and roll, table of marks per Scoring Unit and question (with not-attempted / not-counted indicators), total obtained / target, summary, strengths, weaknesses, suggestions, status (Provisional / Finalized) and date. No grade. - **PDF-03** **Answer pages:** original page images in order with examiner-style overlays, **positioned by line**: - ✓ at the end of lines carrying correct key points/steps; ✗ at lines with wrong content. - Circle or bracket around a line (or line span) containing a spelling/grammar/factual error; the correction or remark written in the margin, joined by a short leader line. - "Missing: …" notes in the margin at the end of the answer. - Per-Scoring-Unit mark in the margin at the answer's first line (e.g., "3½ / 4"). - Brief explanation as a margin remark at the answer's end. - "Not counted" on answers excluded by choice rules. - Red-ink style (color configurable); remarks never cover the student's writing. - **PDF-04** Bengali (conjuncts, vowel signs), Arabic (shaping, RTL) and English MUST render correctly; broken shaping is a defect. - **PDF-05** Handwriting-style but clearly legible annotation font SHOULD be used. - **PDF-06 (v1.1)** Detailed explanations appendix. - **PDF-07** "Provisional" mark on the cover until finalized. - **PDF-08** ≤ 15 MB for a 24-page script; readable on mobile; printable on A4. - **PDF-09** No AI tokens consumed; built from stored results. - **PDF-10** Access via authenticated or short-lived signed links. --- ## 15. Credits and Usage Metering - **CR-01** Metered per tenant. Default: 1 credit per page extracted (blank pages free). Configurable rates for: extraction per page, scoring per script, re-scoring per script/question, rubric structuring per upload (a rate may be 0). - **CR-02** Estimate before batch start; block on insufficient balance; reserve at start; release unused on completion. - **CR-03** System-error failures not charged. - **CR-04** Actual AI tokens and cost recorded per page, script, batch and tenant; Platform Operator reports (cost vs credits); tenant usage view. - **CR-05** Credit top-up via administrative function/API (payment out of scope). --- ## 16. Data Requirements (logical) | Entity | Key attributes | |---|---| | Tenant | ID, Biddaloy school ID, name, settings, credit balance | | Catalog | Education system, class, subject (+ category), syllabus books/chapters (seeded) | | Exam | Tenant, name, academic year (v1.1: Biddaloy ID) | | Question Set | Tenant (or "ShikkhaNet bank"), exam, class, subject, syllabus, version, status, totals, pass mark, Step, rounding, gate, choice mode, duplicate mode, weights, source (own/copied-from) | | Section / Question / Scoring Unit | As §7 | | Exam Batch | Question Set version, sections, status, creator, counts, credit estimate/reservation | | Script | Batch, student ref, roll, anonymous ID, status, page count, legibility rating, totals, feedback, finalization, PDF versions | | Page | Script, order, classification, file ref, checksum, dimensions, DPI, legibility rating, status | | Element | Page, type, line bbox, reading order, languages, content (text / LaTeX / description / table), confidence, flags, candidates | | Correction | Element, corrected content, user, time | | Answer Segment | Script, Scoring Unit, element IDs, confidence, flags, counted | | Score Result | Segment, version, ratings, weights, counts, raw/final, brief, detailed, error items, positive items, flags, model info, tokens, time, source | | Flag, Audit Entry, Usage Record, Export Record | As §13, §15 | - **DATA-01** All text Unicode (UTF-8). - **DATA-02** Exports: batch results (CSV/Excel, Biddaloy layout and generic), analytics (CSV/Excel), script extraction (JSON). --- ## 17. Non-Functional Requirements ### 17.1 Priorities and token/cost rules Strict order: **1. extraction accuracy → 2. minimum AI tokens → 3. speed.** - **TOK-01** No AI processing for blank pages, not-attempted units, PDF generation, analytics or exports. - **TOK-02** One extraction pass per page (EX-00); one scoring call per script (SC-00); everything else reuses stored results. - **TOK-03** Identical context across scripts (rubric, instructions) MUST be reusable across calls where the provider supports caching. **[Architect decision]** - **TOK-04** Image resolution sent to AI = the minimum meeting §17.2, fixed during pilot validation as configuration. - **TOK-05** Different models per stage selectable by configuration. - **COST-01** **AI cost ceiling (planning figures, confirm in pilot):** average AI cost ≤ **US$0.03 per page** for extraction and ≤ **US$0.50 per script** end-to-end (extraction + scoring), measured over a full 500-script batch. The system MUST report actual figures per batch; a configuration or model change that exceeds the ceiling MUST NOT be released without product-owner approval. ### 17.2 Accuracy targets (pilot targets; confirm after pilot) | Metric | Target | |---|---| | Character error rate (CER), clear-to-average handwriting, per language | ≤ 3% | | Numbers read entirely correct | ≥ 99.5% | | Isolated label accuracy | ≥ 99% | | Answer-to-Scoring-Unit mapping | ≥ 98% | | Written page classified blank | 0 in validation set | | Fabricated content presented as certain | 0 in validation set (acceptance criterion) | | Show-through extracted as content | 0 in validation set (acceptance criterion) | | AI vs teacher marks | ≥ 90% of units within ±10% of max (min ±1 Step); MAD ≤ 7% of max | | Re-scoring stability | SC-22 | ### 17.3 Performance and capacity - **PERF-01** Planning volume: 500 scripts × avg 16 pages (max 40) ≈ 8,000 pages per batch. - **PERF-02** 8,000 pages: extraction, scoring and provisional PDFs within **24 hours** of upload completion (excluding teacher review waits). - **PERF-03** Results visible progressively per script. - **PERF-04** v1 capacity: 100 active tenants, 10 concurrent batches, fair queuing across tenants. - **PERF-05** UI pages load ≤ 3 s on typical Bangladeshi broadband. - **PERF-06** Single-script re-score ≤ 5 min; PDF regeneration ≤ 1 min. ### 17.4 Reliability - **REL-01** Page/script failures do not fail the batch; automatic retry (default 3 with backoff) then flag. - **REL-02** Every stage idempotent. - **REL-03** Provider outages/rate limits ⇒ queue and retry; failover to an alternative configured model SHOULD be supported and recorded. - **REL-04** Availability 99.5% monthly. **REL-05** Daily backups, tested restore, RPO ≤ 24 h. ### 17.5 Security and privacy - **SEC-01** Tenant isolation at every layer including file storage. - **SEC-02** TLS in transit; encryption at rest. - **SEC-03** Anonymized AI calls (SC-03); providers configured for no training use and minimum retention. - **SEC-04** Authenticated/signed file access. **SEC-05** Upload validation (type, size, content). - **SEC-06** Applicable Bangladeshi data-protection obligations met; data residency **[Architect decision]**, documented. ### 17.6 Retention and deletion - **RET-01** Defaults per tenant (configurable): page images 12 months after finalization; extraction, scores, PDFs 3 years; audit 5 years. - **RET-02** School Admin may request deletion of a batch or all tenant data; hard delete within 30 days; backups expire per backup retention; deletion itself audited. ### 17.7 AI provider independence - **AI-01** Provider-agnostic; models per stage pluggable by configuration. - **AI-02** Prompts, model choices and versions are versioned configuration; every AI output references them. - **AI-03** AI outputs validated against a structured schema before storage; invalid ⇒ retry or flag, never partial storage. ### 17.8 Usability - **UX-01** UI Bengali/English; rubric editors accept Bengali, English, Arabic. - **UX-02** A teacher with basic computer skills completes Question Set creation (via upload-and-structure), upload and review after ≤ 30 minutes of onboarding. - **UX-03** Review screen: keyboard navigation between flags; zoom/pan. ### 17.9 Operations and quality monitoring - **OPS-01** Processing dashboard for the Platform Operator: batches by state, pages/hour, failure counts, queue depth, AI latency and error rates, tokens and cost per tenant/batch. - **OPS-02** Alerts on: batch stalled beyond a threshold, failure rate above threshold, provider errors, credit reservation failures, PDF generation failures, disk/storage thresholds. - **OPS-03** **Quality sampling:** the Platform Operator MUST be able to pull a random sample of pages and Scoring Units across tenants (anonymized) for manual checking, and record outcomes; sampling results feed the §19 metrics. - **OPS-04** Structured logs with correlation IDs per batch/script/page; retention 90 days. - **OPS-05** Configuration (thresholds, weights, model choices, rates) changeable without deployment and audited. --- ## 18. Processing States and Error Handling **Batch:** Draft → Uploading → Processing → Needs Review → Scored → Finalized (partial/full) → Archived; Cancelled. **Script:** Uploaded → Identified → Extracting → Extracted → Mapped → (Awaiting Review) → Scoring → Scored → PDF Generated (Provisional) → Finalized; Failed (reason); Reopened. - **ERR-01** Every failure has a machine code and a human message in the UI language and appears in the review queue. - **ERR-02** Handled cases: corrupt/unsupported file, duplicate file, unmatched roll, duplicate roll, unreadable page, AI timeout/invalid output, insufficient credits, roster unavailable (retry), Question Set locked. - **ERR-03** Cancelling a batch stops processing and releases unused credits. --- ## 19. Validation and Acceptance - **VAL-01** ShikkhaNet provides a ground-truth validation set before pilot sign-off: ≥ 300 pages across Bengali, English, Arabic and mixed; every subject category; creative, essay, math, diagram and chemistry answers; faint writing and show-through examples; verified verbatim transcriptions; and teacher marks per Scoring Unit for ≥ 50 scripts. - **VAL-02** A re-runnable evaluation function reports §17.2 metrics and §17.1 cost per language and subject category. - **VAL-03** Acceptance: §17.2 targets met (or formally revised), COST-01 within ceiling, PERF-02 demonstrated on a full 500-script batch, every v1.0 MUST traced to a passing test. - **VAL-04** Any model/prompt/configuration change that lowers a metric below target or breaks the cost ceiling MUST NOT be released. --- ## 20. Assumptions 1. Schools scan each student's script into one PDF with a scanner document feeder and name it by roll number. 2. Students write question numbers in a recognizable form; scripts are free-form. 3. Biddaloy exposes SSO and a student roster endpoint for v1.0; results are imported into Biddaloy from the CSV export until the v1.1 API exists. 4. Teachers hold a written question paper and model answers/marking scheme for every exam (as is standard practice), which is the input to upload-and-structure. 5. Handwriting is rated once per script from page legibility ratings. 6. Pilot targets (§17.2), cost ceiling (COST-01) and planning figures (§17.3) are confirmed after the pilot. 7. Feedback language is Bengali or English; Arabic is supported for student content only. ## 21. Items to Confirm (non-blocking) 1. Final credit rates (CR-01) and the cost ceiling figures (COST-01). 2. Retention periods if different from defaults. 3. Biddaloy's result-import file layout (INT-07). 4. Accuracy targets after pilot. --- ## Appendix A — Example: BISE Creative Question ``` Section A: Answer any 7 of 11 questions. (7 × 10 = 70) Question 3 [max 10] Stimulus (উদ্দীপক): ক) Knowledge [1] Scoring Unit 3-ক খ) Comprehension [2] Scoring Unit 3-খ গ) Application [3] Scoring Unit 3-গ ঘ) Higher ability [4] Scoring Unit 3-ঘ Students may write "৩ (ক)", "৩.ক", "3(a)", "উত্তর ৩-গ", or only "ক)" under a "৩" heading. If 8 questions are answered, the best 7 count by default (SG-06). ``` ## Appendix B — Page Legibility Scale (0–10, produced by the extraction pass) | Rating | Descriptor | |---|---| | 9–10 | Very clear, consistent letterforms, even spacing/alignment, almost no cuttings | | 7–8 | Clear and easy to read, minor inconsistencies or a few cuttings | | 5–6 | Readable with effort; uneven size/spacing; noticeable cuttings/overwriting | | 3–4 | Often hard to read; many malformed letters or heavy overwriting | | 0–2 | Largely illegible | Handwriting is not penalized when the cause is image quality (blur, low resolution); such pages are flagged instead. ## Appendix C — Example Scoring Output (one Scoring Unit from a per-script call, illustrative) ```json { "script_anon_id": "S-8F3K2", "scoring_unit": "3-গ", "question_set_version": 2, "max_mark": 3, "weights": {"correctness": 0.5, "completeness": 0.25, "spelling": 0.1, "grammar": 0.1, "handwriting": 0.05}, "ratings": {"correctness": 0.7, "completeness": 0.6, "spelling": 0.8, "grammar": 0.9, "handwriting": 0.7}, "error_counts": {"spelling": 2, "grammar": 1}, "raw_score": 2.115, "step": 0.5, "final_score": 2.0, "brief_explanation": "উদ্দীপকের সাথে মিল সঠিক, তবে দ্বিতীয় দিকটি অনুপস্থিত; দুটি বানান ভুল।", "detailed_explanation": "…", "error_items": [ {"type": "spelling", "line_ref": "p4-l21", "as_written": "সমাজিক", "correction": "সামাজিক", "remark": "বানান ভুল"}, {"type": "missing_key_point", "line_ref": "p4-l22", "remark": "দ্বিতীয় সাদৃশ্য উল্লেখ নেই"} ], "positive_items": [{"key_point": "KP1", "line_ref": "p4-l19"}], "flags": [], "model": {"provider": "", "model": "", "version": ""} } ``` Raw 3 × (0.35 + 0.15 + 0.08 + 0.09 + 0.035) = 2.115 → nearest 0.5 = 2.0.