Data Transfer Features and Flows

1. Documentation Evidence

2. Feature Summary

3. Actor Matrix

4. Capability Matrix

5. User-Facing Flows

5.1 Upload a spreadsheet and get a validation report

Summary

An operator picks a file (students, guardians, or staff), uploads it, and immediately gets back — synchronously, in the same HTTP response — a full report of what the file contains, which rows have problems, and a preview of the first rows that would be created. Nothing is written to the database at this point, no matter how clean the file is.

Preconditions

• The operator is signed in and holds DataImport_CREATE.
• The file is .csv or .xlsx, at most 10 MB, at most 5,000 data rows.
• The operator ideally started from the matching template (§5.4), though any file with matching headers works.

Main Flow

Sequence Diagram

Branches and Edge Cases

5.2 Commit (apply) a validated import

Summary

Having reviewed the dry-run report, the operator explicitly asks for the file to be applied. This is the only action in the whole module that creates people. It runs asynchronously — the commit call returns immediately with a "queued" status, and the actual writes happen in a background worker.

Preconditions

• The upload's dry run reported at least one valid row and no missing headers.
• The request is still pending (not already committing, completed, or failed).
• The operator is the original uploader, or a superadmin.

Main Flow

Sequence Diagram

Branches and Edge Cases

5.3 Request, monitor, and download an export

Summary

An operator asks for a spreadsheet of students, guardians, or staff — optionally filtered the same way the screen they exported from was filtered. The file is built in the background; the operator polls (or is notified) until it is ready, then downloads it through the same authenticated API, never a raw storage link.

Preconditions

• The operator holds DataExport_CREATE to request, and DataExport_READ to view status/download.
• Salary and medical columns appear only if the requester's own active role already grants StaffSalary_READ/StudentMedical_READ — there is no way to request them without holding the permission.

Main Flow

Sequence Diagram

Branches and Edge Cases

5.4 Download a template

Summary

Before uploading anything, an operator downloads a blank starting spreadsheet for the scope they intend to import — every expected column, with a one-row example and, where relevant, a hint about the allowed values.

Preconditions

• The operator holds DataImport_READ.

Main Flow

Sequence Diagram

Branches and Edge Cases

5.5 Place imported pupils in classes

The student template carries three more columns — Grade, Section and Shift — so a school migrating its roll can put pupils in classes in the same upload that creates them, instead of placing several hundred of them one at a time afterwards.

Resolved by name, not by id. Nobody types a uuid into a spreadsheet, and the export already emits names for the same reason. None of the three carries an allowed list, Shift included: allowed is also the parser's gate, and a value it does not recognise drops the whole row before the writer sees it. Losing a pupil over the word "Morning Shift" is the outcome the other two columns avoid, so all three are validated in the writer instead — the way Guardian Relationship already is.

What each combination does

The enrolment date is max(today, session start date) — never the pupil's admission date. In a roll migration that date is an earlier year, and an enrolment outside the session is refused, so using it would fail every row of exactly the import this exists for. Bare "today" fails from the other side: nothing ties is_current to the calendar, so a school setting up next year and importing before term starts would be refused too.

Everything the dry run can predict, it predicts. Unknown names, half-stated classes, ambiguous shifts and admission dates later than the enrolment date are all reported before anything is written, and they count against validRows — so a file whose every class cell is wrong is refused at commit rather than importing nobody. Only capacity is left to write time, because only capacity is genuinely unknowable until then.

A pupil whose class cannot be resolved is not created. The enrolment is written inside that row's own transaction, so an unplaceable pupil rolls back with it rather than arriving classless and unnoticed. That is why the dry run matters: the operator sees every such row before committing.

Capacity is enforced and there is no override column. An import cannot ask a question, and a column would apply an override to as many rows as the operator dragged it down. A full class is a row issue.

Two whole-file refusals, because both are one fact about the upload rather than five thousand row failures: no academic year is marked current, and the current one has already ended.

Permissions

Placing pupils needs Students_CREATE — the permission on POST /students, which is the operation a class-bearing row performs: create a pupil and enrol them in one transaction. Uploading and committing still need only the DataImport_* codes.

The gate is evaluated at commit, against the committer (a superadmin may commit somebody else's upload, and the committer is who the enrolments are recorded against), and re-derived again in the worker from the actor's live permissions. Captured-and-still-true, both required: a payload flag alone would let a role revoked in between go unnoticed, and would let the dead-letter replay route — which needs only DataImport_UPDATE — carry an entitlement decided for somebody else.

A file with no class columns is unaffected by any of this and imports exactly as before.

What the report never contains

Import issues carry a registered error code and the operator's own input. They never carry a resolved class name or an occupancy count. The capacity refusal raised internally says "Class 5 A (morning) is full — 37 of 40 places are taken"; the uploader, who holds no Classes_READ, sees "That class is already full." on the Grade column. The refusal is sanitised where it is raised, not where it is rendered.

Export

The same three columns are emitted on a student export, gated on Classes_READ — a caller without it gets a file with no such columns, not empty ones, matching how the salary columns behave. A pupil with no class exports three empty cells.

6. Admin Flows

6.1 List and review processing dead letters

Route: GET /data/job-failures — DataImport_READ. Read-only; no mutation, no cache invalidation, no audit log beyond the row itself. Supports queueName (ignored/overridden to people if anything else is supplied) and unreplayedOnly filters, plus the shared pagination DTO.

6.2 Replay a dead-lettered job

Route: POST /data/job-failures/:publicId/replay — DataImport_UPDATE. Addressed by the row's public_id (a uuid), never its serial primary key — a sequential id on a route holding a read permission would let anyone walk the table and learn how much of the platform has failed and when.

The claim is written before the enqueue, not after — the alternative order can leave a job running against a row that still reads unreplayed if the follow-up update then fails, and the next operator replays it a second time. The cost of claiming first is that a claim can outlive an enqueue that never reached Redis, and since replayed_at is exactly what the already-replayed refusal reads, a row left in that state could never be replayed again through this route. The release step is what makes the ordering safe: it is conditional on this call's own replay_job_id, so it can only ever undo its own claim, never a claim a second operator made in the meantime.

There is no create, reorder, activate/deactivate, soft delete, or restore flow anywhere in this module — async_requests and job_failures rows are append-only records of events that happened, not entities an admin curates.

6.3 Send account invitations

Route: POST /data/invites — Users_UPDATE, not one of the DataImport_*/DataExport_* codes. Granting sign-in access to a person is an identity change, and this route is reachable without ever touching a spreadsheet — gating it on an import permission would let anyone who can import files also grant login access to arbitrary existing people, which is a different capability.

The operator supplies a list of userIds. Duplicates are removed, order preserved, so the same person listed twice yields one invitation rather than two competing single-use tokens (the second would invalidate the first, and the recipient's working link would not be the one they are most likely to click). The request row and its outbox event are written in one transaction, and the batch itself is queued and paced by the same worker an import's automatic invitations use — see §5.2 for that path.

7. Lifecycle and State Transitions

async_requests.status (import)

async_requests.status (export)

Note: an export never passes through committing — that status exists only for the import side of the lifecycle.

job_failures (per row — not a state machine, a one-way claim)

Every row addresses only by public_id from the API's side — the serial id never leaves the database. There is no "un-replay" once an enqueue actually succeeds — a row in that state stays replayed permanently.

9. Data and Side Effects by Flow

10. Error and Recovery Flows

11. Diagrams Required Per Module

• Actor capability diagram: §3 / §4 tables serve this role; every capability in this module maps to exactly one actor pairing (operator + worker), so a separate diagram would repeat the table.
• High-level module flow diagram: see §12.2.
• Sequence diagram for each major flow: §5.1-§5.4, §6.1-§6.2.
• State machine diagram for every lifecycle: §7.
• Data side-effect diagram for write flows: §12.6.
• Error branch diagram: §6.2 covers the richest branch set; the rest are tabulated in §10.

12. Mandatory Feature and Flow Deep-Dive Pack

12.1 Feature Inventory With Minor Behaviors

Rules followed: restore/retry/skip/duplicate-action/cache-miss/permission-failure behaviors above are called out individually rather than grouped, per the format's own instruction.

12.2 Business Process Diagram Pack

User journey map — an operator running an import end to end

Service blueprint — the commit flow across actor / API / service / DB / async

12.3 Business Rules and Policy Traceability

12.4 Tradeoffs and Product Rationale

12.5 Flow Edge-Case Matrix

12.6 Flow-to-Data Trace

12.7 Experience Quality Checklist

• The doc explains what the actor is trying to accomplish (turn a spreadsheet into records, or records into a spreadsheet, safely).
• The doc explains what the backend does that the actor does not see (per-row transactions, content-hash idempotency, role-gated column decisions made at request time, the outbox pairing).
• The doc covers every minor flow and branch (§12.1, §12.5).
• The doc includes user, admin, worker, and system flows where applicable.
• The doc explains business logic, tradeoffs, and rationale (§12.3, §12.4), including how an import's own invitation batch is chained.
• The doc maps every flow to API routes and backend side effects (§12.6).
• The doc includes diagrams appropriate to each flow type.
• The doc covers all edge cases and failure recovery (§10, §12.5).

13. Completion Checklist

• Every feature, minor action, and submodule capability is listed.
• Every actor has allowed and forbidden behavior.
• Every major and minor flow includes steps, branches, and diagrams.
• Every lifecycle has a transition table and state diagram.
• Every flow links to the API and backend docs.
• TDD dependencies are called out where they shape behavior — none exist for this module; none of the reviewed source references a separate TDD document.

See Also

• API doc: /docs/developer/data-transfer/api
• Backend doc: /docs/developer/data-transfer/backend
• TDD: not present for this module