# AI-Examiner — Solution Design Document (v1.0) | Item | Value | |---|---| | Based on | Requirements Specification v2.1 (19 Oct 2026) | | Document version | 1.1 (PostgreSQL replaces MySQL for AI-Examiner; Biddaloy stays on MySQL) | | Date | 19 October 2026 | | Audience | Mid-level developers and AI coding agents implementing v1.0 | | Status | Ready for implementation | > **How to use this document.** §1–§5 are shared by everyone and must be read first (architecture, conventions, data model, job pipeline). §6 is split into **modules M0–M11**; each module can be built by a separate developer or agent and states exactly what it owns, its interfaces with other modules, step-by-step implementation, tests, and **what to do if the planned approach fails** ("Obstacles & alternatives"). §7–§9 hold prompts, schemas and algorithms that modules reference. Requirement IDs (e.g., `SC-12`) refer to the spec; every module lists the IDs it satisfies so coverage can be checked. > > Library versions: use the latest stable release at project start and pin it in lock files (composer.lock, requirements.txt/uv.lock, package-lock.json). Where this document names a version it is a minimum. --- ## 1. Architecture Overview ### 1.1 Shape A **modular monolith** (Laravel web application) plus a **Python AI worker** service, sharing one PostgreSQL database and one object store, connected by a database-backed job table. ``` Browser (teacher, Inertia+Vue) Biddaloy (CakePHP 2) │ HTTPS │ SSO JWT / roster API / CSV import ▼ ▼ ┌─────────────────────────────────────────────────────────┐ │ Laravel app ("app") │ │ tenancy · auth · question sets · uploads · review UI │ │ credits · audit · analytics · exports · notifications │ │ scheduler · light PHP queue (emails, exports) │ └───────────────┬───────────────────────────┬──────────────┘ │ PostgreSQL │ S3 (MinIO / cloud) ┌───────────────┴───────────────────────────┴──────────────┐ │ Python worker ("worker") × N │ │ rasterize · extract (LLM) · assemble · score (LLM) │ │ pdf (Chromium) · evaluation harness │ │ polls `jobs` table, writes pipeline tables │ └──────────────────────────────────────────────────────────┘ │ HTTPS AI providers (Anthropic / Google / OpenAI) via adapter ``` ### 1.2 Technology stack | Layer | Choice | Notes | |---|---|---| | Web app & API | PHP 8.3+, Laravel 12+ | Inertia.js + Vue 3, Tailwind; Laravel Sanctum for internal API tokens | | Database | PostgreSQL 16+ (UTF-8) | JSONB for elements/results with GIN indexes where queried; `FOR UPDATE SKIP LOCKED` for job polling; `LISTEN/NOTIFY` to wake idle workers. Biddaloy keeps its own MySQL database; AI-Examiner never connects to it (integration is via HTTP only, M9) | | Cache / locks / PHP queue | Redis 7 | Not used for the AI job queue (see §5) | | Object storage | S3-compatible (MinIO self-hosted, or cloud S3) | Buckets per environment; keys prefixed by tenant | | AI worker | Python 3.12+, FastAPI (health/metrics only), SQLAlchemy, Pydantic v2 | Runs as a long-lived poller; N replicas | | Image/PDF | PyMuPDF (rasterize, crop), Pillow | | | PDF rendering | Playwright + headless Chromium | HTML/CSS overlay → print to PDF | | Fonts | Noto Sans Bengali, Noto Naskh Arabic, Noto Sans; a handwriting-style Latin font (e.g., Caveat) | Bundled in worker image | | AI providers | Anthropic, Google, OpenAI SDKs behind `providers/` adapter | Chosen per stage by config after bake-off (M11) | | Browser upload | Uppy (resumable, tus protocol) + tusd server or Laravel chunked endpoint; pdf.js for optional client compression | | | Deployment | Docker Compose on 2–3 VMs (Singapore region); nginx; Let's Encrypt | Kubernetes not used in v1 | | CI | GitHub Actions: PHP tests, Python tests, lint, image build | | | Observability | JSON logs → Loki or CloudWatch; Prometheus metrics from worker `/metrics`; Grafana; alerting via Grafana | | ### 1.3 Repository layout (monorepo) ``` ai-examiner/ app/ Laravel application app/Modules/{Tenancy,Catalog,QuestionSets,Batches,Review,Credits,Audit,Analytics,Biddaloy,Notifications} database/migrations (single source of truth for ALL tables, incl. worker tables) resources/js/ Vue pages & components worker/ Python worker worker/pipeline/{rasterize,extract,assemble,score,pdf}.py worker/providers/{base,anthropic_,google_,openai_}.py worker/schemas/ Pydantic models + JSON schemas (generated to shared/) worker/eval/ evaluation harness (M11) worker/templates/ PDF HTML templates shared/ schemas/*.json JSON Schemas for extraction/scoring outputs (generated from Pydantic, consumed by PHP validators) fixtures/ sample pages, rubrics, expected outputs deploy/ docker-compose.*.yml, nginx, env templates docs/ this document, spec, ADRs ``` --- ## 2. Cross-Cutting Conventions (read before coding) - **IDs:** ULID strings (26 chars) for every primary key; generated by the creator (PHP or Python). Never expose auto-increment ints. - **Tenant scoping:** every tenant-owned table has `tenant_id`. Laravel: a global scope on all tenant models, set from the session; the worker: every query includes `tenant_id` taken from the job row. A query without `tenant_id` on a tenant table is a code-review reject. - **Time:** store `timestamptz` (UTC); display in Asia/Dhaka. - **Text:** database encoding UTF-8, collation `und-x-icu` (ICU) so Bengali and Arabic sort sensibly; never transliterate; LaTeX stored as plain strings. JSON data uses `jsonb`. - **PostgreSQL specifics:** IDs are `char(26)` ULIDs; enums are `text` + `CHECK` constraints (easier to migrate than PG enum types); booleans are real `boolean`; Laravel uses the `pgsql` driver, Python uses SQLAlchemy 2 with `psycopg` (v3). - **JSON contracts:** all AI outputs validated against `shared/schemas/*.json` (Pydantic generates them; PHP validates with `opis/json-schema`). A failed validation never reaches the database (AI-03). - **Money/credits:** integers in milli-credits; cost in micro-USD integers. - **Errors:** `error_code` strings `AIX--` (e.g., `AIX-UPL-003 duplicate_checksum`); human messages from Laravel lang files (bn, en). - **Config:** Laravel `.env` + `config/aix.php`; worker `settings.py` (pydantic-settings) reading the same env names. Runtime-tunable values (thresholds, model per stage, rates) live in the `settings` table (OPS-05), cached 60 s. - **Logging:** JSON lines with `tenant_id, batch_id, script_id, page_id, job_id, correlation_id`. - **Audit:** any mutation of audited tables goes through `AuditService::record()` (PHP) or `audit.record()` (Python). Audit table is insert-only (DB user for app has no UPDATE/DELETE on it). - **Anonymity:** the worker never reads `students`. Scripts carry `anon_id`; cover pages (`pages.kind = 'cover'`) are never sent to the scoring stage. - **Feature flags:** `settings` rows `feature.*` (e.g., `feature.client_compression`). --- ## 3. Data Model All tables created by Laravel migrations. Columns listed are the important ones; add `created_at/updated_at` everywhere and `tenant_id` where marked (T). ### 3.1 Platform and tenancy | Table | Key columns | |---|---| | `tenants` | id, biddaloy_school_id, name, settings_json (feedback_lang, review_mode, confidence_threshold, retention, pdf_options, annotation_color), credit_balance_milli, status | | `users` (T) | id, biddaloy_user_id, display_name, email, last_seen_at (no password column) | | `user_scopes` (T) | user_id, class_id, subject_id (null = all) | | `settings` | key, value_json, scope ('platform' or tenant_id), updated_by | | `audit_log` | id, tenant_id, actor_type ('user','system','worker'), actor_id, action, entity_type, entity_id, before_json, after_json, reason, qs_version, model_info_json, tokens_json, created_at | ### 3.2 Catalog (seeded, platform-owned) | Table | Key columns | |---|---| | `education_systems` | id, code, name_bn, name_en | | `classes` | id, education_system_id, code, name_bn, name_en, sort | | `subject_categories` | id, code ('language','theory','numerical'), default_weights_json | | `subjects` | id, class_id, code, name_bn, name_en, subject_category_id, expected_langs_json | | `syllabus_books` / `syllabus_chapters` | id, subject_id / book_id, title_bn, title_en, sort | ### 3.3 Question sets | Table | Key columns | |---|---| | `exams` (T) | id, name, academic_year, biddaloy_exam_id (nullable) | | `question_sets` (T) | id, exam_id, class_id, subject_id, syllabus_json (book/chapter ids), version, status ('draft','ready','locked','archived'), parent_id (copied-from), bank_source_id, total_questions, total_target_marks, pass_mark, step (1.0/0.5/0.25), rounding ('nearest','down','up'), content_gate bool, choice_mode ('best','first'), duplicate_mode ('higher','first'), weights_json, created_by | | `qs_sections` | id, question_set_id, label, sort, answer_count (N) nullable | | `questions` | id, question_set_id, section_id, label, alt_labels_json, type, text, stimulus_text, stimulus_image_key, expected_langs_json, cognitive_level, max_mark, sort | | `scoring_units` | id, question_id, label (null if the question itself), sort, max_mark, model_answer_text, model_answer_latex, model_answer_image_keys_json, key_points_json [{id,text,weight,mandatory}], instructions, alternatives_json, common_mistakes_json, weights_json (override), ai_drafted bool, approved_by | | `question_bank_sets` | id (= a question_sets row with tenant_id NULL), education_system_id, published_by, published_at | Question set versioning: on edit of a `ready`/`locked` set, copy the full tree to a new row with `version+1` and `parent_id`; scripts reference `question_set_id` of the exact version. ### 3.4 Batches and scripts | Table | Key columns | |---|---| | `batches` (T) | id, question_set_id, section_ids_json, class_id, status, created_by, script_count, page_count, credit_estimate_milli, credit_reserved_milli, started_at, completed_at | | `students` (T) | id, biddaloy_student_id, roll, name, class_id, section, group_name, synced_at | | `scripts` (T) | id, batch_id, student_id (nullable until matched), roll_from_filename, roll_from_cover, anon_id, original_file_key, status, page_count, legibility_avg, total_score, total_target, strengths_json, weaknesses_json, suggestions_json, summary, finalized_at, finalized_by, reopened_at, pdf_version, flags_count | | `pages` (T) | id, script_id, sort, kind ('cover','answer','supplementary','rough','blank','unreadable'), image_key, width, height, dpi, checksum, legibility (0–10), confidence, extracted_at, model_info_json, tokens_in, tokens_out, cost_micro | | `elements` (T) | id, page_id, sort, type ('line','equation','chem','figure','table','label','heading'), bbox_json [x0,y0,x1,y1] normalized, text, latex, description, table_json, lang_json, confidence, flags_json ('struck','inserted','uncertain','illegible','rough'), candidates_json, q_label (raw label string if this element is a question heading) | | `element_corrections` (T) | id, element_id, field, value, user_id | | `segments` (T) | id, script_id, scoring_unit_id, element_ids_json (ordered), content_text (assembled), confidence, flags_json, counted bool, attempt_no | | `score_results` (T) | id, segment_id, version, source ('ai','teacher'), ratings_json, weights_json, error_counts_json, raw_score, final_score, brief, detailed, error_items_json, positive_items_json, flags_json, model_info_json, qs_version, created_by, is_current bool | | `flags` (T) | id, batch_id, script_id, page_id, element_id, segment_id, type, severity ('block','warn','info'), status ('open','resolved','dismissed'), payload_json, resolved_by, resolution | | `script_pdfs` (T) | id, script_id, version, file_key, provisional bool, size_bytes | ### 3.5 Pipeline and metering | Table | Key columns | |---|---| | `jobs` | id, tenant_id, type ('rasterize','extract_page','assemble','score','pdf','structure_qs','rescore'), payload_json, priority, status ('queued','running','done','failed','cancelled'), attempts, max_attempts, run_after, locked_by, locked_at, last_error, result_json, correlation_id | | `usage_records` (T) | id, batch_id, script_id, page_id, op_type, credits_milli, tokens_in, tokens_out, cost_micro, provider, model | | `credit_ledger` (T) | id, type ('topup','reserve','release','charge','refund'), amount_milli, ref_type, ref_id, balance_after | | `exports` (T) | id, batch_id, type, file_key, created_by | | `notifications` (T) | id, user_id, type, payload_json, read_at, emailed_at | Indexes: `jobs(status, run_after, priority)` (partial index `WHERE status='queued'`), `pages(script_id, sort)`, `elements(page_id, sort)`, `segments(script_id, scoring_unit_id)`, `score_results(segment_id, is_current)`, `flags(batch_id, status)`. --- ## 4. Object Storage Layout ``` {bucket}/{tenant_id}/batches/{batch_id}/scripts/{script_id}/original.pdf .../pages/{page_sort}.jpg (optimized page image, 200 DPI, JPEG q≈75) .../pages/{page_sort}.ai.jpg (downscaled copy sent to the model, M3 decides size) .../pdf/v{n}.pdf {bucket}/{tenant_id}/question-sets/{qs_id}/uploads/* (teacher's question paper & model answers) {bucket}/{tenant_id}/question-sets/{qs_id}/images/* (stimulus / model answer images) {bucket}/{tenant_id}/exports/{export_id}.xlsx ``` Access from browser via pre-signed URLs (15 min), generated by Laravel (SEC-04, PDF-10). --- ## 5. Job Pipeline ### 5.1 Why a DB job table Both PHP and Python can read it with no serialization mismatch, it is visible in the admin UI, it survives restarts, and PostgreSQL `SELECT … FOR UPDATE SKIP LOCKED` makes polling safe with many workers. Enqueuers run `NOTIFY aix_jobs` after insert; idle workers `LISTEN aix_jobs` so they wake immediately instead of polling every second (polling every 5 s stays as the safety net). Redis is kept for PHP-only jobs (emails, exports). ### 5.2 Worker loop (Python) ``` loop: job = SELECT * FROM jobs WHERE status='queued' AND run_after<=NOW() ORDER BY priority DESC, created_at LIMIT 1 FOR UPDATE SKIP LOCKED if none: sleep 1s; continue UPDATE jobs SET status='running', locked_by=worker_id, locked_at=NOW(), attempts+=1 try: handler[job.type](job) → UPDATE status='done', result_json except RetryableError: UPDATE status='queued', run_after=NOW()+backoff(attempts) or 'failed' if attempts>=max_attempts (+ flag) except FatalError: UPDATE status='failed', last_error; create flag ``` Stale lock reaper (scheduler, every minute): `running` jobs with `locked_at` older than the job type's timeout → back to `queued`. ### 5.3 Fairness (PERF-04) `priority` is computed at enqueue as `base − (number of queued jobs of this tenant / 100)`; the scheduler recomputes every minute so one tenant's 8,000 pages don't starve others. ### 5.4 Job chain per script `rasterize(script)` → N × `extract_page(page)` → when all pages done: `assemble(script)` → `score(script)` → `pdf(script)`. The "all pages done" check is done by `extract_page` on completion (count query) and the last one enqueues `assemble`. Idempotency: each handler checks whether its output already exists and returns early (REL-02). ### 5.5 Timeouts and retries | Job | Timeout | Max attempts | |---|---|---| | rasterize | 10 min | 3 | | extract_page | 5 min | 3 | | assemble | 2 min | 3 | | score | 15 min | 3 | | pdf | 5 min | 3 | | structure_qs | 10 min | 2 | --- ## 6. Modules Dependency order: **M0 → (M1, M2, M9 in parallel) → M3 → M4 → M5 → M6 → M7; M8 and M10 alongside; M11 starts at M3.** Each module has: Scope · Satisfies · Owns · Interfaces · Steps · Tests · Obstacles & alternatives. --- ### M0 — Foundation (platform, tenancy, auth, infra skeleton) **Scope:** repo skeleton, Docker Compose dev environment, migrations for §3, tenancy, SSO login from Biddaloy, settings, audit service, base UI layout (bn/en), job table and worker skeleton that can run a `noop` job. **Satisfies:** INT-01, INT-02, INT-03, OPS-05, RV-07 (service), SEC-01, SEC-02. **Owns:** `tenants, users, user_scopes, settings, audit_log, jobs`; `deploy/docker-compose.dev.yml`; `worker/main.py` loop; `AuditService`; `TenantScope`. **Interfaces** - SSO: Biddaloy redirects to `GET /sso?token=`. JWT (HS256, shared secret per tenant, 2-minute expiry, `jti` single-use) claims: `school_id, user_id, name, email, scopes:[{class_code, subject_code}], iat, exp, jti`. Laravel validates, upserts tenant/user/scopes, creates session, redirects to `/`. - Settings read API for worker: worker reads `settings` table directly. **Steps** 1. Create Laravel project; add Inertia + Vue + Tailwind; add `spatie/laravel-permission` (only for internal "platform_admin" vs "tenant_user" distinction), `opis/json-schema`, `league/flysystem-aws-s3-v3`. 2. Write all migrations in §3 (one migration per table group). Add `TenantOwned` trait with global scope and `creating` hook that fills `tenant_id`. 3. Implement `SsoController` (JWT validation, replay protection using Redis `SETNX jti`). 4. Implement `SettingsService::get(key, tenant_id=null)` with 60 s cache and platform→tenant precedence. 5. Implement `AuditService::record(action, entity, before, after, reason=null, extra=[])`; grant DB user insert-only on `audit_log` (separate PostgreSQL role for the app with `GRANT INSERT, SELECT` only on `audit_log`; a migration-owner role holds DDL rights). 6. Worker: `main.py` poll loop (§5.2), `noop` handler, `/healthz`, `/metrics` (Prometheus: jobs processed, failures, durations). 7. Docker Compose: nginx, app (php-fpm), scheduler, php-queue, postgres, redis, minio, worker. `make dev` brings everything up with seed data. 8. GitHub Actions: `composer test`, `pytest`, `eslint`, build images. **Tests:** SSO happy path + expired/replayed token; tenant scope leak test (user of tenant A requests tenant B id → 404); worker processes `noop` job; stale-lock reaper. **Obstacles & alternatives** - *Biddaloy cannot issue JWTs easily (CakePHP 2):* implement a 60-line PHP helper in Biddaloy using `firebase/php-jwt`; fallback is a signed query string (HMAC of `school_id|user_id|exp`) with the same checks. - *Shared DB access from Python considered too coupling:* add a thin internal REST API in Laravel for the worker (`/internal/*` with Sanctum token). Keep the DB approach unless this becomes a real problem; it is faster to build. --- ### M1 — Catalog and Question Sets (authoring, structuring, question bank) **Scope:** seeded catalog, exam creation, question set CRUD with sections/questions/scoring units, validation, versioning/locking, copy, question bank, **upload-and-structure** (AI) with approval UI, factor weights UI. **Satisfies:** CTX-01/02, INT-05, INT-06, QS-01…QS-11, RB-01…RB-05, FW-01…FW-04, QS-07 (via worker job `structure_qs`). **Owns:** catalog tables, `exams, question_sets, qs_sections, questions, scoring_units, question_bank_sets`; Vue pages `QuestionSets/*`; worker handler `structure_qs`. **Interfaces** - `POST /question-sets` … standard REST; `POST /question-sets/{id}/validate` returns `QS-04` result; `POST /question-sets/{id}/ready`; `POST /question-sets/{id}/copy`; `POST /question-sets/{id}/structure` (uploads files, enqueues `structure_qs`); `GET /question-bank?education_system=`. - Worker job `structure_qs` payload: `{question_set_id, file_keys[]}` → writes a **draft tree** to `question_sets.draft_structure_json` (not the real tables) and sets status `structured_pending_review`. Laravel UI shows the draft; teacher approves → Laravel writes real rows. - Catalog seed: `database/seeders/CatalogSeeder.php` reading `shared/fixtures/catalog/*.csv` exported by ShikkhaNet (INT-05). **Steps** 1. Seeder + read-only catalog UI (platform admin can edit). 2. Question set editor (Vue): tree view section → question → scoring unit; inline editing; LaTeX preview with KaTeX; image upload for stimulus/model answers (pre-signed PUT). 3. Validation service implementing QS-04: `max_total = Σ compulsory max + Σ_sections topN(max_marks, N)`; must equal `total_target_marks`; every scoring unit needs model answer; weights sum 100 at every level. 4. Versioning: `ready → locked` when first batch is created; edit on locked = clone to new version (service `QuestionSetVersioner`). 5. Alt-label generator (PHP, deterministic): for label `3` produce `["৩","3","Q3","Q.3","3.","৩।","৩)","3)","প্রশ্ন ৩","উত্তর ৩","Ans 3","Ans. 3"]`; for sub-label `গ` produce `["গ","গ)","(গ)","গ.","৩(গ)","3(গ)","৩-গ","3 গ"]`. Teacher can edit. 6. Weights UI: category default → set → unit override; show effective weights. 7. Question bank: `tenant_id NULL` sets with `published_at`; copy action clones into tenant. 8. `structure_qs` worker handler: rasterize uploaded PDFs/images (PyMuPDF), send pages to the LLM with prompt §7.3 and schema `question_set_draft.schema.json`; Word files converted with `python-docx` to text first. Output goes to `draft_structure_json`. 9. Approval UI: side-by-side (uploaded page image | structured tree) with edit; "Approve" writes rows and sets `draft`. **Tests:** validation cases (choice sections, unequal max marks); versioning clone keeps all children; alt-label generator fixtures; structure_qs on 3 sample question papers (bn creative, en essay, math) produces a draft with correct counts. **Obstacles & alternatives** - *Structuring accuracy is poor on messy model-answer PDFs:* split into two calls (questions first, then model answers mapped to question labels); if still poor, ship manual editor + copy-from-bank first and keep structuring as "beta". - *KaTeX cannot render a teacher's LaTeX:* store as-is, show a warning, never block. --- ### M2 — Upload, Identification, Page Classification handoff **Scope:** batch creation, resumable upload of one PDF per student, filename→roll matching against roster, duplicate detection, credit estimate/reserve, page rasterization job, manual fixes (reassign, reorder, delete, add pages). **Satisfies:** UP-01…UP-13 (classification itself is produced by M3), CR-02, PERF-03 (progress). **Owns:** `batches, scripts, pages` (creation), upload endpoints, Vue `Batches/*`, worker `rasterize`. **Interfaces** - Resumable upload: tusd container (`/files/`) with hook `post-finish` → Laravel `POST /internal/uploads/finished` (metadata: batch_id, filename, checksum). Simpler alternative: Uppy with Laravel chunked endpoint (`POST /batches/{id}/upload-chunk`) assembling in temp storage; choose tusd (less code to maintain). - `POST /batches` (question_set_id, section_ids) → locks the question set; `GET /batches/{id}` (progress counts); `POST /batches/{id}/start` (estimate + reserve credits, enqueue rasterize per script); `POST /scripts/{id}/reassign`, `/pages/reorder`, `/pages/{id}/delete`, `/scripts/{id}/add-pages`. - Worker `rasterize` payload `{script_id}`: downloads original, for each page renders at 200 DPI (PyMuPDF `get_pixmap(dpi=200)`), grayscale if `settings.image.grayscale`, JPEG q=75 (target ≈300 KB; if > 450 KB lower q to 60; if > 40 pages → flag `too_many_pages`), writes `pages` rows and `{sort}.jpg`; also writes `{sort}.ai.jpg` at `settings.ai.image_long_edge_px` (default 1600, tuned by M11); computes quick non-AI blank check: if ink-pixel ratio < 0.15% → `kind='blank'` (conservative; M3 can override to non-blank if it ever sees it, but blanks are not sent). Then enqueues `extract_page` for every non-blank page. - Roll parsing: regex `^\s*([০-৯0-9]+)` on filename stem; Bengali digits normalized to ASCII for matching only. **Steps** 1. Batch create/list screens; start button disabled until ≥1 script and credits suffice. 2. Upload screen with Uppy (tus), drag-drop up to 500 files; per-file status: matched / unmatched / duplicate roll / duplicate file. 3. `post-finish` handler: checksum dedupe (UP-04), create `scripts` row with `roll_from_filename`, match roster (`students`), set `anon_id = 'S-' + random 6 chars`. 4. Credit estimate = page count unknown before rasterize → estimate from file size (avg 120 KB/page in scanned PDFs) with ×1.2 margin; reserve; true-up after rasterize (`CreditService`, M8). 5. Progress API polled every 5 s (or Laravel Echo later). 6. Manual fix endpoints + audit. **Tests:** 500-file upload simulation (script); checksum dedupe; roll parsing fixtures incl. Bengali digits; rasterize sample PDFs (portrait, landscape, mixed sizes) → page count and sizes; blank check on known blank/faint pages (0 false blanks on fixture set). **Obstacles & alternatives** - *tusd hooks awkward behind nginx:* use Uppy's XHR upload with Laravel chunked endpoint (the Uppy "xhr-upload" + `resumable` is not available; accept non-resumable for files ≤ 50 MB and show retry). - *Browser compression requirement (UP-02):* v1.0 ships server-side rasterization (same output, no client code). Enable `feature.client_compression` later: pdf.js renders pages → canvas → JPEG → upload as a ZIP; worker skips rasterize if images are present. - *Scanner outputs one giant PDF for the whole class:* add "split by blank separator page" in rasterize (detect blank page as separator) as a v1.1 feature; for v1.0 ask schools to scan per booklet. --- ### M3 — Extraction (single-pass page reading) **Scope:** the `extract_page` job: send a page image to the configured vision model, receive the §7.1 schema, validate, store elements, legibility, classification, cover fields, flags; token/cost accounting. **Satisfies:** EX-00…EX-16, UP-09, UP-11 (AI side), TOK-04, AI-01…AI-03, SEC-03. **Owns:** `worker/pipeline/extract.py`, `worker/providers/*`, `elements`, `pages` (update), schema `extraction_page.schema.json`. **Interfaces** - Input: `pages` row + `{sort}.ai.jpg` + question-set label list (so the model can tag headings) + previous page's last 3 lines (continuity). - Provider adapter (§7.5): `extract_page(image_bytes, prompt_vars) -> ExtractionPage` using the provider's structured-output/tool-call feature with the JSON schema. - Output: rows in `elements`; `pages.kind, legibility, confidence, model_info_json, tokens`, `usage_records`; flags: `low_confidence_extraction` (any element confidence < threshold), `unreadable_page`, `cover_roll_mismatch` (compares with `scripts.roll_from_filename`). **Steps** 1. Define Pydantic models (§7.1) and export JSON schema to `shared/schemas/`. 2. Implement `providers/base.py` interface + three implementations; each maps the schema to the provider's structured output mechanism; each returns `tokens_in/out`, `cached_tokens`, `cost_micro` computed from a `pricing` table in settings. 3. Prompt §7.2 with strict verbatim rules; system prompt kept identical across pages so provider caching applies. 4. Validation: schema; bbox sanity (0≤x0 0 for `answer` pages; if the model says `blank` but rasterize ink check said non-blank → keep page as `answer` and flag `blank_disagreement` (never silently blank, UP-11). 5. Store elements in reading order; `q_label` raw string for heading elements; `flags` from the model. 6. On completion count remaining pages for the script; last page enqueues `assemble`. 7. Retry policy: schema failure → retry once with "your previous output failed validation: " appended; provider 5xx/429 → RetryableError with backoff; model failover: if provider has failed 3 times in 5 minutes, switch to `settings.ai.extract.fallback_model` and record it (REL-03). **Tests:** fixture pages with expected JSON (bn creative, en essay, math with equations, geometry with point labels ক/খ/গ, Arabic page, faint page, show-through page) — assert CER, digit accuracy, labels present, no fabricated lines; schema-failure retry path; failover path with a mocked provider. **Obstacles & alternatives** - *Line bounding boxes unreliable from the chosen model:* (a) ask the model for boxes in a 0–1000 integer grid instead of floats (models do better); (b) draw a light 10×14 grid with coordinates on the AI copy of the image and ask for cell references; (c) fallback: no boxes — the PDF (M6) then places all remarks in the margin aligned to the answer's start, found by heading element order. Decide in M11. - *Verbatim violated (model "corrects" spelling):* strengthen prompt with explicit examples of misspelled words to keep; lower temperature to 0; if still failing, add a second cheap text-only pass that compares output to a stricter re-read of flagged lines — only if M11 shows it is needed. - *Cost above COST-01:* reduce `image_long_edge_px` stepwise (1600→1400→1200) while CER target holds; use a cheaper model for pages classified `rough`/`cover`. - *Mixed Bengali/Arabic ordering issues:* store `lang_json` per line and keep the model's logical order; do not post-process. --- ### M4 — Answer Assembly and Mapping (deterministic) **Scope:** `assemble` job: build Answer Segments per Scoring Unit from heading elements and labels; handle out-of-order, continuation, duplicates, choice rules, rough work; create flags. **Satisfies:** SG-01…SG-08. **Owns:** `worker/pipeline/assemble.py`, `segments`. **Algorithm (§9.1)** — pure Python, no AI except the ambiguous fallback. **Steps** 1. Load all elements of the script in `(page.sort, element.sort)` order, excluding `rough` pages/elements and `struck` lines (kept but flagged). 2. Build a label matcher from the question set: map every alt label → `(question_id, scoring_unit_id|null)`; normalize digits (bn→ascii), strip punctuation/spaces, lowercase. 3. Walk elements; a `heading` element (or a line that matches a label pattern with ≥ 0.9 confidence) opens a new segment; lines until the next heading belong to it. A sub-label heading (e.g., `গ`) under an open question heading maps to that question's unit. 4. Continuation: a segment for a unit that already exists and is not closed by a later heading is appended (SG-02). Duplicate attempts: `attempt_no` increments; apply `duplicate_mode` (SG-04) → the non-counted attempt gets `counted=false` and a flag. 5. Unmatched headings → `unmapped` segment + flag `unmapped_content`. Headings with confidence < 0.9 → segment created with `confidence` and flag `low_mapping_confidence`. 6. Choice rules are applied **after scoring** (M5 marks `counted`), but M4 records section membership. 7. `content_text` assembled: lines joined with `\n`; equations as `$…$`; figures as `[FIGURE: description]`; tables as Markdown; uncertain spans as `⟦uncertain: a|b⟧`; illegible as `⟦illegible⟧`. 8. If more than 20% of headings are low-confidence, enqueue an AI assist call (prompt §7.4) with the full page text and label list to assign segments; still flag for teacher. 9. Enqueue `score`. **Tests:** fixtures of element sequences → expected segments (out of order, continued on later page, duplicate with one struck, sub-labels without question heading, "any 7 of 11"). **Obstacles & alternatives** - *Students write headings inconsistently (e.g., only "ক" with no question number):* use "nearest preceding question heading" rule; if none, flag. Teacher drag-drop reassignment in M7 is the safety net. - *Heading detection misses because the model merged heading with first line:* in M3 prompt require heading lines be separate elements; in M4 also run the label regex on the first 12 characters of every line. --- ### M5 — Scoring (one call per script) **Scope:** `score` job: build the per-script scoring request (all segments + full rubric), call the model, validate §7.2 schema, compute marks deterministically (§9.2), apply content gate, rounding, guardrails, choice rules, store `score_results`, script feedback, flags; `rescore` job for unit/question/script/batch. **Satisfies:** SC-00…SC-22, SC-26…SC-28, TOK-02/03, COST-01 (measurement). **Owns:** `worker/pipeline/score.py`, `score_results`, `scripts` (totals, feedback), schema `scoring_script.schema.json`. **Key design decision:** the model returns **ratings and findings, not marks**. Marks are computed in code (SC-08…SC-12). This makes guardrails trivial and results reproducible. **Steps** 1. Build request: system prompt §7.3 (static per question set version → cacheable), user content: `context` (education system, class, subject, syllabus), `rubric` for every scoring unit, `answers` = segments with `content_text`, `legibility_avg`, `feedback_lang`. Attach page-region images only for units whose question `type` is figure-based (crop with PyMuPDF from bbox, max 800 px) (SC-02). 2. Token budget check: if estimated input > `settings.ai.score.max_input_tokens` (default 150k), split by questions into 2+ calls, keeping each question's units together; script feedback then produced by a small follow-up call that receives only the per-unit briefs (SC-20). 3. Validate schema; every `unit_id` in the response must exist; missing units → retry once with the list of missing ids. 4. Compute per unit (§9.2): ratings → raw → gate → round → clamp. Spelling/grammar ratings derive from `error_items` counts (not from the model's rating) for determinism. 5. Choice rules (SG-06): per section, rank unit totals by question, mark `counted=false` for extras; add `not_counted` error item. 6. Totals: script total = Σ counted; clamp to target (SC-12); store `scripts.total_score`. 7. Flags: `uncertain_affects_score` when a unit's content has `⟦uncertain⟧` and `max_mark × 0.1 ≥ step`; `rubric_minimal`. 8. Write `score_results` (`is_current=true`, previous `is_current=false`), `usage_records`; enqueue `pdf`. 9. `rescore` job payload `{scope:'unit'|'question'|'script'|'batch', ids, keep_overrides:true}`; for `unit`/`question` scope use a reduced request with only those units (the only allowed per-unit call, SC-00). **Tests:** golden fixtures: given a fixed model response JSON → expected marks (gate on/off, rounding modes, step 0.25/0.5/1, choice rule best/first, clamp); schema failure retry; split-call path on a 40-page fixture; determinism test by replaying the same response. **Obstacles & alternatives** - *Model inconsistent across scripts:* temperature 0; include 2–3 short worked examples per question type in the system prompt; if still noisy, enable `SC-23` sample re-scoring early (it is a config flag) and surface variance to the teacher. - *Per-script context too large for cheaper models:* per-question mode (config `ai.score.granularity=question`) is already supported by step 2. - *Spelling error counts unreliable in Bengali:* cap penalty effect by `settings.scoring.max_spelling_penalty` (default 0.5 of the factor) and let teachers see/edit error items; consider a dictionary-assisted check (Hunspell bn_BD) in v1.1. --- ### M6 — Annotated PDF (Chromium render) **Scope:** `pdf` job: build HTML (cover + pages with overlays) from stored results, render with Playwright/Chromium to PDF, upload, record `script_pdfs`; bulk ZIP export (Laravel side). **Satisfies:** PDF-01…PDF-05, PDF-07…PDF-10. **Owns:** `worker/pipeline/pdf.py`, `worker/templates/{cover.html, page.html, styles.css}`, `script_pdfs`; Laravel `GET /scripts/{id}/pdf`, `POST /batches/{id}/export-pdfs` (ZIP via PHP queue). **Rendering approach** - Each answer page: `
` sized to the image aspect at 210 mm width; the page image as ``; overlays positioned with `left/top/width/height` in % from `bbox`; margin column 28 mm on the right (or left for RTL subjects) for remarks; leader lines as inline SVG. - Marks: `✓`/`✗` glyphs at line end; `.circle` border-radius box over the error line span; `.remark` in margin at the same vertical position, stacked downward if collisions (simple greedy layout: keep `next_y = max(y, last_bottom + 2mm)`). - Score badge at the unit's first line: `"3½ / 4"` using Bengali numerals when `feedback_lang=bn` and the unit label is Bengali. - Cover page: table, totals, feedback, `Provisional` ribbon when `scripts.finalized_at IS NULL`. - Fonts via `@font-face` from worker-local files; CSS `font-family: "Caveat", "Noto Sans Bengali", "Noto Naskh Arabic", sans-serif;`. - Chromium: `page.pdf(format='A4', print_background=True)`; images embedded as data URIs or served from a local file path (`file://`) — use file paths to keep HTML small. - Target size: pages already ≈300 KB JPEG → 24 pages ≈ 7–8 MB (PDF-08). **Steps** 1. Templates with Jinja2; CSS print rules; test harness that renders fixtures to PNG for visual review. 2. Collision layout for margin remarks; truncate remark text to 150 chars (SC-14) with full text in the detailed appendix (v1.1). 3. Bengali numeral formatting helper; half-mark glyphs (½, ¼, ¾). 4. Upload `v{n}.pdf`, bump `scripts.pdf_version`. 5. Laravel: signed download; batch ZIP job (streams PDFs into a ZIP in S3; notification when ready). **Tests:** visual regression (render fixtures → PNG → compare hash within tolerance); Bengali conjunct rendering check (render "ক্ষ", "জ্ঞ", "ন্ত্র" and compare against reference PNG); Arabic shaping check; size assertion. **Obstacles & alternatives** - *Chromium memory on 40-page scripts:* render in chunks of 10 pages and merge with PyMuPDF `insert_pdf`. - *No usable bboxes (M3 fallback):* `page.html` switches to "margin-only" mode: all remarks for an answer stacked at the answer heading's vertical position (heading element still has an approximate y from reading order: `y ≈ sort / lines_on_page`). - *Playwright install trouble in the image:* use `mcr.microsoft.com/playwright/python` base image. --- ### M7 — Review UI, Overrides, Finalization **Scope:** review queue, side-by-side script viewer, extraction corrections, segment reassignment, score override, feedback edit, annotation item edit, finalize/reopen, history. **Satisfies:** RV-01…RV-08, EX-13, EX-15, SC-28, RV-06 blocking rules. **Owns:** Vue `Review/*` (queue, ScriptViewer, UnitPanel), endpoints under `/review/*`, services `OverrideService`, `FinalizeService`. **Interfaces** - `GET /batches/{id}/flags?status=open` grouped by type; `POST /flags/{id}/resolve|dismiss`. - `GET /scripts/{id}/view` → pages (signed URLs), elements, segments, current results. - `POST /elements/{id}/correct` (field, value) → `element_corrections` + flag resolve + optionally enqueue `rescore` for the affected unit. - `POST /segments/{id}/reassign` (scoring_unit_id) → enqueue `rescore` unit. - `POST /units/{segment_id}/override` `{final_score, brief?, detailed?, error_items?, positive_items?, reason?}` → validates step & guardrails (§9.2 in PHP), writes `score_results(source='teacher', is_current=true)`, recomputes totals, enqueues `pdf`. - `POST /scripts/{id}/finalize`, `/reopen`; `POST /batches/{id}/finalize`. **Steps** 1. Queue page with filters and keyboard shortcuts (j/k next/prev flag, enter open). 2. ScriptViewer: left page image with zoom/pan (use `panzoom`), element overlays toggle; right: unit panel with content, ratings, brief/detailed, error items (editable list), history drawer. 3. Correction flow: inline edit of a line's text → saves correction → shows "re-score this unit?" button. 4. Override flow with live guardrail validation; reason optional. 5. Finalize: check blocking flags; set `finalized_at`; enqueue `pdf` (non-provisional); create export notification; reopen sets `reopened_at` and clears `finalized_at` (audited). 6. All actions through `AuditService`. **Tests:** override above max rejected; override to 2.3 with step 0.5 rejected; finalize blocked by open `unmapped_content`; reopen → export marks revision; e2e Playwright test of correct→rescore→override→finalize. **Obstacles & alternatives** - *Overlay alignment confusing when bboxes are rough:* show overlays as thin side-bar markers instead of boxes (toggle). - *Large scripts slow to load:* paginate elements per page; lazy-load images. --- ### M8 — Credits, Audit views, Analytics, Notifications, Exports **Scope:** credit ledger and reservation, usage accounting, platform cost reports, batch analytics, in-app/email notifications, CSV/Excel exports (results in Biddaloy layout, analytics, extraction JSON). **Satisfies:** CR-01…CR-05, AN-01, NT-01, DATA-02, INT-07 (export), CR-04. **Owns:** `credit_ledger, usage_records (reads), exports, notifications`; `CreditService`, `AnalyticsService`, `ExportService`, `NotificationService`; Vue `Credits`, `Analytics`, `Exports`. **Steps** 1. `CreditService::estimate(batch)`, `reserve`, `trueUp(pages)`, `charge(usage_record)`, `release`, `refund`; all ledger writes in DB transactions with `SELECT … FOR UPDATE` on tenant row. 2. Rates from `settings`: `credits.extract_page=1000` (milli), `credits.score_script`, `credits.rescore_script`, `credits.structure_qs`. 3. Worker writes `usage_records` only; Laravel scheduler (every minute) converts new usage records to ledger charges (keeps Python out of ledger logic). 4. Analytics: SQL views over `score_results` + `segments` (per unit avg, attempt rate, most-missed key points from `error_items.type='missing_key_point'` grouped by `key_point_id`, top spelling errors grouped by `as_written`); Vue charts (Chart.js); Excel export via `maatwebsite/excel`. 5. Results export: `ResultsBiddaloyExport` (columns per INT-07; layout confirmed with Biddaloy team — item 21.3); generic export; extraction JSON per script. 6. Notifications: events (`BatchScored`, `FlagsPending`, `UploadFinishedWithIssues`, `ProcessingFailed`, `CreditsLow`) → DB notification + mail (Laravel Mail with SMTP provider); daily digest option. 7. Platform admin pages: cost vs credits per tenant, model usage, tokens per page trend (CR-04). **Tests:** ledger invariants (balance_after chain, no negative balance), estimate/true-up math, analytics SQL on fixtures, export column snapshot. **Obstacles & alternatives** - *Excel export memory on 500 scripts × 30 units:* stream with `FromQuery` + chunking. - *Email deliverability in BD:* use a transactional provider (e.g., Amazon SES or Postmark); fallback in-app only. --- ### M9 — Biddaloy Integration (Biddaloy side + AI-Examiner side) **Scope:** JWT SSO issuer in Biddaloy, roster API in Biddaloy, roster sync in AI-Examiner, results CSV import guidance, "Open AI-Examiner" link in Biddaloy. **Satisfies:** INT-01, INT-02, INT-04, INT-07 (file side). **Owns:** Biddaloy: `AiExaminerController` (CakePHP 2) with `sso()` and `api_students()`; AI-Examiner: `BiddaloyClient`, `RosterSyncService`, `students` table. **Interfaces** - Biddaloy `GET /ai-examiner/sso` (logged-in user) → builds JWT (§M0) and redirects to `https://aix.example/sso?token=…`. - Biddaloy `GET /api/ai-examiner/students?class=§ion=` header `X-Api-Key: ` → JSON `[{student_id, roll, name, class, section, group}]`. - AI-Examiner: `POST /roster/sync` (class, section) called when a batch is created and on demand; upserts `students`. - Results: teacher downloads `results_biddaloy.csv` from M8 and imports in Biddaloy's existing result import screen (v1.0). **Steps** 1. Biddaloy: add config table `ai_examiner_settings` (school → secret, api_key, base_url); controller with the two actions; menu link. 2. AI-Examiner: `BiddaloyClient` with timeout 10 s, retries 3, circuit breaker; `RosterSyncService`. 3. Contract tests with recorded JSON fixtures on both sides. **Obstacles & alternatives** - *Biddaloy roster endpoint delayed:* AI-Examiner accepts roster CSV upload (roll, name, student_id) as a stop-gap — implement this anyway as fallback (ERR-02 roster unavailable). - *Per-school API key management burdensome:* one platform-level key plus `school_id` parameter, signed with HMAC; acceptable for v1. --- ### M10 — Deployment, Operations, Security **Scope:** production Compose stacks, nginx/TLS, backups, monitoring, alerts, log shipping, retention jobs, deletion workflow, secrets. **Satisfies:** PERF-04 (capacity), REL-04/05, SEC-02/04/05/06, RET-01/02, OPS-01…OPS-04, ERR-03. **Steps** 1. VMs: `web` (nginx, app, scheduler, php-queue, redis), `db` (PostgreSQL, MinIO or cloud S3 instead), `worker-1..n` (worker containers; start with 2 VMs × 4 workers). Sizing: worker VM 4 vCPU/8 GB handles ~4 concurrent LLM calls + Chromium. 2. nginx with client_max_body_size 200 MB, TLS via certbot; tusd upstream. 3. Backups: `pg_dump` (custom format) nightly to S3 plus WAL archiving (e.g., `wal-g` or `pgBackRest`) for point-in-time recovery, + MinIO bucket replication (or cloud S3 versioning); restore drill documented. 4. Monitoring: worker `/metrics`, Laravel metrics via `prometheus` exporter, node exporter; Grafana dashboards: jobs by state, pages/hour, failure rate, LLM latency/error rate, tokens & cost per tenant/batch (OPS-01). 5. Alerts (OPS-02): job queued > 2 h; failure rate > 5%/15 min; provider error rate > 20%; credit reservation failures; disk > 80%. 6. Retention scheduler: daily job applies `tenants.settings.retention` → deletes page images after finalization + N months, PDFs/extraction after 3 years; deletion requests table with 30-day execution (RET-02). 7. Secrets in env files on VMs with restricted permissions (or cloud secrets manager); AI provider keys per stage. 8. Security checks: upload validation (magic bytes, size, page count), signed URLs, rate limiting on API, dependency audit in CI. **Obstacles & alternatives** - *VM-based Compose becomes unwieldy at >10 workers:* move worker only to a managed container service; app stays. - *Data residency decision:* default Singapore region; if BD-only hosting is demanded, MinIO + local VMs are already supported by the design. --- ### M11 — Evaluation Harness and Model Bake-off (starts immediately) **Scope:** scripts to run the extraction and scoring pipeline on the ground-truth set, compute §17.2 metrics and COST-01, compare providers/models/resolutions, and emit the chosen configuration. **Satisfies:** VAL-01…VAL-04, TOK-04, model selection (AI-01). **Owns:** `worker/eval/{run_extract.py, run_score.py, metrics.py, report.py}`, `shared/fixtures/ground-truth/` (pages, `transcripts/*.txt`, `teacher_marks.csv`). **Steps** 1. Ground-truth format: one folder per page with `page.jpg`, `truth.json` (lines in order, verbatim text, labels list, numbers list, blank flag) and for scored scripts `marks.csv`. 2. Metrics: CER via `jiwer`/`rapidfuzz` on normalized Unicode (NFC) per language; digit accuracy (sequence match of all numeric tokens); label accuracy; fabrication = output lines with no alignment to truth above 0.5 similarity; blank false positives; mapping accuracy (vs truth labels); score agreement (within ±10% max, MAD); determinism (run twice). 3. Bake-off matrix: providers × image long edge {1200, 1600, 2000} × prompt variants; output a Markdown report with cost per page and per script. 4. Writes recommended `settings` JSON; CI job runs a 30-page smoke subset on every prompt change and fails if metrics regress (VAL-04). **Obstacles & alternatives:** if no provider meets CER ≤ 3% on Bengali, report per-script-type results and agree revised targets (VAL-03) — do not block the build; the UI and pipeline are provider-independent. --- ## 7. Prompts and Schemas ### 7.1 Extraction output schema (Pydantic → `extraction_page.schema.json`) ```python class Element(BaseModel): type: Literal['line','heading','equation','chem','figure','table','label'] bbox: list[float] # [x0,y0,x1,y1] in 0..1; may be approximate text: str | None # verbatim; for heading: the raw label as written latex: str | None # equation/chem description: str | None # figure/table description of what is drawn table: list[list[str]] | None lang: list[Literal['bn','en','ar']] confidence: float # 0..1 flags: list[Literal['struck','inserted','uncertain','illegible','rough']] candidates: list[str] | None # alternative readings when uncertain class ExtractionPage(BaseModel): kind: Literal['cover','answer','supplementary','rough','blank','unreadable'] cover: dict | None # {"roll": "...", "registration": "...", "subject_code": "..."} as written legibility: int # 0..10 per Appendix B elements: list[Element] # reading order notes: str | None # anything the examiner should know (e.g., page torn) ``` ### 7.2 Extraction prompt (system, static) ``` You are a transcription engine for handwritten examination scripts in Bengali, English and Arabic. Rules (strict): 1. Transcribe EXACTLY as written. Keep every spelling, grammar and punctuation mistake. Never correct, normalize, complete or translate. Keep Bengali conjuncts, vowel signs, hasanta and Arabic diacritics as written. 2. Keep digits in the script used by the writer (Bengali ১২৩, ASCII 123, Arabic-Indic ١٢٣). Read digits with extreme care (১ vs ২, ৪ vs ৮, ০ vs 0 vs O). 3. Single letters used as labels (ক, খ, গ, ঘ, A, B, x, y …) in figures, geometry or as answer-part labels are separate "label" or "heading" elements; never merge them into words. 4. Question/part headings (e.g. "৩", "৩(গ)", "Ans to Q 3 c", "উত্তর-৫", "ক)") must be their own "heading" element with the raw label text. Known labels for this paper: {{labels}}. 5. Crossed-out text: transcribe with flag "struck". Inserted text (caret/arrow): place at the intended position with flag "inserted". 6. If you cannot read something with confidence, do NOT guess: use flag "uncertain" with up to 3 candidates, or flag "illegible" with text null. Never invent text. Ignore faint mirror-image show-through from the reverse side and printed booklet text. 7. Figures/graphs/tables: describe only what is actually drawn (parts, labels present, connections, axes, units, values). Do not describe what a correct figure would contain. 8. Equations: give LaTeX in "latex" and the linear text in "text", preserving the student's errors. 9. Give a reading-order list and an approximate bounding box (0..1 of page width/height) for every element. 10. Rate page legibility 0–10 (10 = very clear, 0 = illegible). Classify the page kind. If this is a cover page, read roll/registration/subject code exactly as written. Previous page ended with: {{prev_tail}} Output only JSON matching the provided schema. ``` ### 7.3 Scoring prompt (system, static per question set version → cacheable) ``` You are an experienced examiner for {{education_system}}, {{class}}, {{subject}} (syllabus: {{syllabus}}). Feedback language: {{feedback_lang}}. You will receive the full question paper with model answers and marking rules, and one student's answers (transcribed verbatim; "⟦uncertain: a|b⟧" marks unreadable parts, "⟦illegible⟧" unreadable text, "[FIGURE: …]" a drawing description, "$…$" an equation). Do NOT assign marks. For every scoring unit return: - correctness (0..1): accuracy of what is stated vs model answer, key points, alternatives, instructions. - completeness (0..1): coverage of key points/steps/parts/labels/units expected for this class. - key_points: for each rubric key point: present|partial|absent, with the line_ref where it is addressed. - error_items: every spelling, grammar, factual, method/step, calculation, missing-unit, label and irrelevant-content error, each with line_ref, as_written, correction (if any), remark (≤60 chars). Do not count proper nouns or rubric exceptions. Do not penalize uncertain/illegible text. - positive_items: correct key points/steps with line_ref. - brief (≤150 characters) and detailed explanation (correct → wrong → missing → spelling/grammar → how it combines → rules applied), in {{feedback_lang}}; quote the student's words where useful; never claim content that is not in the answer. Apply the same standard to every unit; be consistent; use the same criteria wording for the same situations. After all units, give script-level: strengths (3–5), weaknesses (3–5), suggestions (3–5, actionable, tied to chapters), summary (≤300 chars). Output only JSON matching the schema. ``` User content: `{context, rubric:[{unit_id, question_label, unit_label, type, question_text, stimulus, model_answer, key_points[], instructions, alternatives[], common_mistakes[], max_mark, expected_langs}], answers:[{unit_id, content_text, legibility}], legibility_avg}`. ### 7.4 Assembly-assist prompt (only for ambiguous scripts) Give the list of `{unit_id, labels}` and the page text with element ids; ask for `[{element_id_from, element_id_to, unit_id, confidence}]`. Temperature 0. ### 7.5 Provider adapter interface ```python class VisionProvider(Protocol): name: str def structured(self, *, system: str, user_parts: list[TextPart|ImagePart], schema: type[BaseModel], max_tokens: int, cache_system: bool=True) -> Result # Result: parsed BaseModel, raw_text, tokens_in, tokens_out, cached_tokens, latency_ms, model_version ``` Implementations map `cache_system=True` to each provider's prompt-caching mechanism where available. Pricing per model in `settings.ai.pricing` → `cost_micro`. ### 7.6 Scoring response schema (abridged) ```python class UnitFinding(BaseModel): unit_id: str correctness: float; completeness: float key_points: list[KeyPointStatus] error_items: list[ErrorItem] # type, line_ref, as_written, correction, remark positive_items: list[PositiveItem] brief: str; detailed: str class ScriptFinding(BaseModel): units: list[UnitFinding] strengths: list[str]; weaknesses: list[str]; suggestions: list[str]; summary: str ``` --- ## 8. Algorithms ### 8.1 Assembly (M4) — pseudocode ``` labels = build_label_index(question_set) # normalized label -> (question_id, unit_id|None) segments = {}; current = None; open_question = None for el in elements_in_reading_order(script, exclude_rough=True): lbl = match_label(el) if el.type=='heading' or looks_like_label(el.text[:12]) else None if lbl: q, u = resolve(lbl, open_question) # sub-label resolves under open_question if q: open_question = q key = u or default_unit(q) current = segments.setdefault(key, []).append(new_attempt(el)) if key else new_unmapped(el) continue if current: current.lines.append(el) else: unmapped_prefix.append(el) apply_duplicate_mode(segments); build_content_text(); create_flags() ``` ### 8.2 Score computation (M5, mirrored in PHP for overrides) ``` w = effective_weights(unit) # fractions summing to 1 r.spelling = max(0, 1 - n_spelling_distinct * penalty_sp) r.grammar = max(0, 1 - n_grammar * penalty_gr) r.handwriting = legibility_avg / 10 raw = max_mark * (w.c*r.c + w.comp*r.comp + w.sp*r.sp + w.gr*r.gr + w.hw*r.hw) if content_gate and r.c == 0: raw = 0 final = round_to_step(raw, step, mode) # nearest: floor(raw/step + 0.5)*step final = min(max(final, 0), max_mark) ``` Question mark = Σ unit finals ≤ question max; script total = Σ counted question marks ≤ total_target. ### 8.3 Choice rule For each section with `answer_count=N`: order answered questions by question mark desc (`best`) or by first appearance (`first`); the first N are `counted=true`, others `false`. ### 8.4 Blank detection (rasterize) Grayscale → Otsu threshold → ink ratio = dark pixels / total, ignoring a 4% border and horizontal ruled lines (remove rows with > 60% dark run). `blank` if ratio < 0.0015. --- ## 9. Delivery Plan ### 9.1 Parallel tracks (after M0, ~2 weeks) | Track | Modules | Suggested owner | |---|---|---| | A — Web core | M1, M2 (Laravel side), M7 | 2 developers/agents | | B — Pipeline | M3, M4, M5, M6 (worker) + M2 rasterize | 2 developers/agents | | C — Platform | M8, M9, M10 | 1 developer/agent | | D — Quality | M11 (from day one), fixtures, CI | 1 developer/agent | ### 9.2 Milestones 1. **M0 done:** SSO login, empty dashboard, worker runs noop. (Week 2) 2. **Vertical slice:** manual question set → upload 5 scripts → extraction → assembly → scoring → PDF viewed in UI, no review features. (Week 6) 3. **Review & credits:** M7, M8 complete; bake-off report chooses models/resolution. (Week 9) 4. **Pilot-ready:** M9 live with one school, M10 monitoring, 500-script load test, VAL-03 report. (Week 12) ### 9.3 Definition of done per module Code + migrations + tests passing in CI; endpoints documented in `docs/api.md` (OpenAPI generated); requirement IDs in the module header traced to tests in `docs/traceability.md`; feature flags for anything not ready. --- ## 10. Risk Register (summary) | Risk | Impact | Mitigation / alternative | |---|---|---| | Model CER on Bengali handwriting above target | Accuracy (priority 1) | Bake-off across providers/resolutions (M11); revised targets per VAL-03; "Review required" mode | | Unreliable bounding boxes | PDF quality | Grid/integer-coordinate prompting; margin-only PDF mode (M6 fallback) | | Per-page cost above COST-01 | Economics | Lower image resolution; cheaper model for cover/rough pages; caching of system prompt and rubric | | Teachers won't author rubrics | Adoption | Upload-and-structure (M1) + question bank; copy from previous year | | Biddaloy endpoints late | Integration | Roster CSV upload fallback; results CSV import (already v1.0 path) | | Chromium rendering of complex scripts | PDF defects | Noto fonts bundled; rendering regression tests | | Single DB bottleneck at 10 batches | Throughput | Indexes above; element writes batched per page (`COPY` or multi-row insert); PgBouncer if connection count grows; read replica only if measured | --- ## Appendix — Environment variables (shared names) `DB_CONNECTION=pgsql`, `DB_HOST/DB_PORT/DB_DATABASE/DB_USERNAME/DB_PASSWORD`, `DATABASE_URL` (worker, `postgresql+psycopg://…`), `REDIS_URL`, `S3_ENDPOINT/S3_BUCKET/S3_KEY/S3_SECRET`, `AIX_JWT_SECRET_DEFAULT`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, `OPENAI_API_KEY`, `AI_EXTRACT_MODEL`, `AI_SCORE_MODEL`, `AI_STRUCTURE_MODEL`, `AI_FALLBACK_MODEL`, `AI_IMAGE_LONG_EDGE`, `WORKER_CONCURRENCY`, `TUSD_URL`, `MAIL_*`, `APP_LOCALE=bn`. --- ## Addendum — Implementation notes (22 Oct 2026) Decisions taken during sprint 1 that change this design. Details and evidence: `docs/PROGRESS.md`. 1. Schema is plain SQL in `db/migrations/` (shared by Laravel and the worker), not Laravel PHP migrations. 2. Worker uses psycopg 3 directly (no SQLAlchemy). 3. Blank-page detection (§8.4) replaced: paper-relative ink threshold + connected strokes ≥ 30 px; blank if < 120 stroke px. 4. Jobs carry a `dedupe_key` (unique among live jobs) to make fan-in steps race-free. 5. M6 uses **PyMuPDF** (HarfBuzz shaping, Noto fonts, per-script font runs) instead of Playwright/Chromium; worker image is `python:3.12-slim` + `fonts-noto-core`. Fonts are subset before saving. 6. Spelling/grammar errors are shown as wavy underlines on the whole line with the correction in the margin (line-level boxes).