Audience: Frontend engineers, mobile engineers, backend engineers, QA, and API consumers.
Scope: Admin-facing bulk import, export, and processing dead-letter APIs owned by the data-transfer module. There is no public, mobile, or webhook surface — every route requires an authenticated, permissioned admin identity.
Upload a spreadsheet and get a dry-run validation report.
Admin
GET
/data/imports/:requestId
Admin/operator (owner) or superadmin
JwtAuthGuard, RoleGuard
DataImport_READ
DataTransferController.getImport
Get an import's status and report/outcome.
Admin
POST
/data/imports/:requestId/commit
Admin/operator (owner)
JwtAuthGuard, RoleGuard
DataImport_UPDATE
DataTransferController.commitImport
Apply a validated import.
Admin
POST
/data/exports
Admin/operator
JwtAuthGuard, RoleGuard
DataExport_CREATE
DataTransferController.requestExport
Request an export.
Admin
GET
/data/exports/:requestId
Admin/operator (owner) or superadmin
JwtAuthGuard, RoleGuard
DataExport_READ
DataTransferController.getExport
Get an export's status and download link.
Admin
GET
/data/exports/:requestId/download
Admin/operator (owner) or superadmin
JwtAuthGuard, RoleGuard
DataExport_READ
DataTransferController.downloadExport
Download a built export file.
Admin
POST
/data/invites
Admin/operator
JwtAuthGuard, RoleGuard
Users_UPDATE
DataTransferController.sendInvites
Send account-activation invitations to a batch of people.
Admin
GET
/data/job-failures
Admin/operator
JwtAuthGuard, RoleGuard
DataImport_READ
DataTransferController.listJobFailures
List processing dead letters (pinned to the people queue).
Admin
POST
/data/job-failures/:publicId/replay
Admin/operator
JwtAuthGuard, RoleGuard
DataImport_UPDATE
DataTransferController.replayJobFailure
Re-run a failed bulk job under a fresh id.
There are no aliases, no nested sub-resources, and no parent-module route prefix beyond /data itself — DataTransferController is the only controller mounted under this prefix.
No — there is no @Public() or guest-accessible route anywhere in this controller
RoleGuard reads the @Permissions metadata and checks it against the actor's active role; a superadmin bypasses the permission check entirely (repo-wide behavior, not specific to this module)
Import/export status, commit, download
Same guard chain, plus an in-service ownership check
AuthUser.id compared to async_requests.actor_id
Same as above
No
DataTransferService.assertOwnRequest additionally refuses anyone but the original requester unlessactor.activeRole?.isSuperadmin — holding the permission is necessary but not sufficient
Export column gating
Not a guard — an in-service permission check at request time
AuthUser passed to PeoplePermissionsService.can
StaffSalary_READ, StudentMedical_READ (checked, not required — their absence just narrows the resulting file)
No
Decided once, when POST /data/exports runs; never re-evaluated by the worker that later builds the file
Send invitations
Same guard chain
@CurrentUser() supplies actor.id
Users_UPDATE — deliberately not DataImport_*/DataExport_*
No
Granting sign-in access is an identity change reachable with no spreadsheet involved; gating it on an import permission would let anyone who can import files also grant login access to arbitrary existing people
Every endpoint requires a valid JWT and requires login; none support a guest identity or optional auth.
spreadsheetMulterOptions: .csv/.xlsx only, max 10 MB
—
data-transfer.controller.ts:111
sendInvites is persisted on the request's stored result and read back at commit time: it decides whether every row this run creates gets can_login: true (PeopleImportWriter.apply's grantLogin parameter), and, once the commit succeeds, whether PeopleImportProcessor automatically queues an invitation batch for exactly the people it created — see §8.4 and the backend doc's runtime flow for the full sequence.
Set only when status: "failed", truncated to 2000 chars server-side
null
data-transfer.dto.ts:69
outcome
Record<string,unknown> | null
Yes, nullable
N/A
Present only after a successful or failed commit attempt; shape is { created, failed, issues, createdUserIds } — createdUserIds is what the worker hands to requestInvites when it chains an invitation batch
null before commit; {"created":115,"failed":3,"issues":[...],"createdUserIds":["018f...","018f..."]} after
downloadUrl is documented as a "short-lived signed URL" in one DTO comment (data-transfer.dto.ts:116-119), but the current implementation returns a plain path on this API (/api/data/exports/:requestId/download), never a signed object-storage URL — verified directly against DataTransferService.getExport (data-transfer.service.ts:346-349). The stored value is always a storage key, never a URL, for the reason the comment gives: a persisted signed URL would outlive its own signature.
Validated as an unversioned uuid (@IsUUID(undefined, ...)), not @IsUUID("4", ...) — every id this platform mints is a uuidv7, whose version nibble is 7, and pinning the validator to version 4 rejects every real id. userIds is de-duplicated (order preserved) and re-checked against the same 500-item cap server-side in DataTransferService.requestInvites, independently of @ArrayMaxSize, so a client-side bypass of the DTO validator still cannot exceed the batch limit.
Inherited from QueryDto, but false is refused here specifically — DeadLetterService.findAll throws 400 PAGINATION_LIMIT_INVALID before running any query, since job_failures only ever grows
Defined for typed use elsewhere; the actual :scope path param on GET /data/templates/:scope is read as a plain @Param("scope") string and validated by an inline DATA_SCOPES.includes(...) check with a silent fallback to "students" rather than by this DTO (data-transfer.controller.ts:76,80-82).
Returns a blank spreadsheet — one header row plus one example row — for the given scope, so an operator has the exact column set and an example of valid values before filling in real data. Call this before every first-time import of a scope, and whenever the operator wants a clean starting file rather than reusing an old one.
200, streamed binary, not the JSON envelope. Headers: Content-Type (text/csv; charset=utf-8 or the OOXML spreadsheet type), Content-Disposition: attachment; filename="skoolsewa-<scope>-template.<format>", Content-Length.
Unknown scope in the URL: silently serves the students template rather than erroring.
format omitted: defaults to xlsx.
format=csv on a school with Devanagari staff/student names in the example row: correctly rendered thanks to the UTF-8 BOM prepended to the CSV bytes.
The students, guardians, and staff templates each carry ten address columns — Permanent Province, Permanent District, Permanent Municipality, Permanent Ward, Permanent Tole, and the same five again for Current — matched by name against the geography lists at commit time, never accepted as ids. There is no house-number column on any template; it is settable afterward through the person forms. Leaving the whole Current group blank means "not recorded," never "same as the permanent address."
Uploads a spreadsheet and returns a full validation report synchronously, in the same response — total/valid row counts, every row-level issue, unknown/missing headers, and a preview. Call this every time an operator wants to check a file before committing it; nothing is written to the database by this call under any circumstances.
Rate limit: None specific to this route; bounded indirectly by the 10 MB / 5,000-row caps.
Idempotency: Yes — keyed on sha256(file bytes) under the scope, globally (not per actor). A duplicate upload returns the earlier report rather than creating a new request.
Empty input: a file with headers only and zero data rows returns totalRows: 0, validRows: 0 — not an error.
Blank search: N/A, no search on this route.
Invalid enum in a cell: reported as IMPORT_INVALID_ENUM_VALUE, does not block the rest of the file.
Expired session: standard 401 from JwtAuthGuard.
Duplicate request: byte-identical file returns the earlier report, flagged alreadyExisted in the service layer (surfaced only via the message text, not a dedicated response field).
Race condition: two identical uploads racing — one wins the insert, the other reads the winner's row; no error under normal conditions.
Cache miss: N/A, no cache.
DB row missing: N/A on this route.
Guest trying a logged-in-only action: N/A, no route accepts a guest at all.
Unsupported filter or sort option: N/A, no filtering on this route.
Rate-limit failure behavior: none configured specific to this route.
Polls an import request's current status and report/outcome. Call this after upload to confirm the dry-run report, and repeatedly after commit until status reaches completed or failed.
Applies a previously validated import. This is the only endpoint in the module that results in new students/guardians/staff (and users) rows. Call this only after the operator has reviewed the dry-run report and explicitly confirmed they want the file applied — there is no "undo."
Idempotency: Yes — the status transition pending -> committing is conditional and can succeed exactly once; a second commit call always fails with 409.
Database writes: async_requests status pending -> committing and, later (async, by the worker), -> completed/-> failed plus outcome; outbox_events insert (same transaction as the status flip); then, per valid row, users + one of students/guardians/staff (+ student_guardian for a student with guardian info).
Cache: none.
BullMQ jobs: PeopleJob.IMPORT_COMMIT enqueued via the outbox (not directly); the worker may in turn enqueue PeopleJob.SEND_INVITE_BATCH after it finishes, if sendInvites was requested and at least one row was created.
Realtime events: none.
Analytics events: none.
Email/push notifications: none from this HTTP call itself, but if the upload's sendInvites was true, the worker automatically queues separate invitation-batch requests once the commit succeeds (see §8.10) — invitation emails are sent from those requests' own worker runs, not from this one. An import may create up to 5,000 people while one invitation batch holds at most 500, so the people created are split into chunks of 500 and each chunk becomes its own request with its own id; a chunk that cannot be queued is counted and logged rather than silently dropped.
Audit logs: none dedicated.
External API calls: StorageManager.getFile (worker re-reads the stored file, asynchronously, not part of this HTTP call).
Empty input: N/A — a request always has an underlying file by the time it exists.
Duplicate request: two commit calls racing — exactly one succeeds; the transactional UPDATE ... WHERE status = 'pending' is the enforcement mechanism, not an application-level lock.
Race condition: covered above — this is the primary race this endpoint exists to prevent.
Cache miss: N/A.
DB row missing: 404.
Guest trying a logged-in-only action: N/A.
Unsupported filter or sort option: N/A.
Rate-limit failure behavior: none specific.
A row names an ethnicity/mother tongue not in the school's list: fails that row at write time with IMPORT_UNKNOWN_ETHNICITY/IMPORT_UNKNOWN_MOTHER_TONGUE inside outcome.issues, rather than importing the row with the column blank.
A row's Permanent Province/District/Municipality/Current Province/District/Municipality names a place not in the geography lists, or a district/municipality that does not belong to the parent given in the same row: fails that row with IMPORT_UNKNOWN_PLACE naming the offending column, rather than importing a half-resolved address. Each name is matched within the parent already resolved on that row — a district is looked up inside the given province, never by name alone, because two provinces can share a district name.
A row fills Permanent Ward/Current Ward without its municipality, or Permanent Tole/Current Tole without its district: fails with IMPORT_UNKNOWN_PLACE naming the column that is missing its parent, before the file ever reaches an insert.
A student row supplies neither Guardian Name nor Guardian Phone: fails that row with IMPORT_STUDENT_REQUIRES_GUARDIAN naming Guardian Name — a student import row with no guardian is refused outright, the same "at least one guardian" rule the people API enforces on POST /students with STUDENT_REQUIRES_ONE_GUARDIAN.
The upload's sendInvites was true: every row this commit creates gets can_login: true; if sendInvites was false (the default), every created row is login-disabled regardless of any cell in the file.
Requests a new export file for a scope, optionally scoped by the same filters the operator's screen was showing. Returns immediately with a pending request; the actual file is built asynchronously. Call this whenever an operator wants a downloadable spreadsheet of current data — a second request is never deduplicated against an earlier one.
BullMQ jobs: PeopleJob.EXPORT_BUILD enqueued via the outbox.
Realtime events: none.
Analytics events: none.
Email/push notifications: none.
Audit logs: none dedicated.
External API calls: two PeoplePermissionsService.can checks (StaffSalary_READ, StudentMedical_READ) against the caller's own active role — not an external service, but a permission-system call worth noting since it is what decides the file's column set.
No custom error codes are thrown directly by this endpoint. Standard 400 from DTO validation (scope invalid, format invalid) applies. See §8.7 for the errors surfaced once the request has progressed.
Polls an export request's status. Call this repeatedly until status: "completed" (or "failed"); downloadUrl is populated only once the file is actually ready.
Downloads the built export file's bytes. Call this only once GET /data/exports/:requestId reports status: "completed" and a non-null downloadUrl — this route is exactly that URL's path.
Storage file deleted out from under a completed row (operational/manual): would surface as whatever StorageManager.getFile throws — not specifically coded here.
Forwarded/guessed URL by a non-owner: refused regardless of whether the guesser separately holds DataExport_READ.
Lists processing dead letters — jobs that were enqueued successfully and then failed on their final attempt. Pinned to the people queue regardless of what queueName is passed; call this to build the "failed bulk jobs" operational screen.
pagination (default true; false is refused — see Error Cases), page (default 1), size (default 20, max 100), queueName (accepted, effectively ignored — always resolves to "people"), unreplayedOnly (optional boolean). sort/order/search are accepted by the shared QueryDto base but not used by this endpoint's query.
Empty result: data: [] with valid (zero-count) pagination metadata.
queueName set to anything other than "people": silently ignored; results are still scoped to people.
unreplayedOnly=true with no unreplayed rows: empty result.
pagination=false: refused outright, unlike every reference-table list endpoint elsewhere in the codebase that permits it up to PaginationUtil.UNPAGINATED_HARD_CAP.
Re-runs a failed job under a fresh BullMQ job id. Call this after diagnosing a transient cause (e.g. a storage timeout) for a dead-lettered job — not for a failure caused by bad data in the source file, which needs the file fixed and re-uploaded/re-requested instead.
Idempotency: The replay claim itself is idempotent — a conditional UPDATE ... WHERE replayed_at IS NULL admits exactly one caller. If the enqueue that follows the claim fails, the claim is released, and the row is claimable again — so a caller that retries after a 503 is not blocked by its own earlier attempt. A claim that does enqueue successfully can never be replayed again through this route.
Database writes: job_failures — replayed_at, replayed_by, replay_job_id set via a conditional UPDATE; reset back to null by a second UPDATE if the enqueue below then fails.
Cache: none.
BullMQ jobs: a fresh job added to the row's own queue_name (in practice always people), under buildJobId(["replay", id, timestamp]), carrying {...payloadRef, correlationId: replayJobId, replayOf: row.id}.
Realtime events: none.
Analytics events: none.
Notifications: none from this endpoint directly (the replayed job itself may send one, depending on which job it is).
Audit logs: replayed_by/replayed_at on the row serve as the audit trail while a claim holds; both are cleared again if the enqueue fails.
Two operators replaying the same row simultaneously: exactly one succeeds; the other gets JOB_FAILURE_ALREADY_REPLAYED.
The enqueue itself fails after the claim succeeds: the claim is released, matched on the failing call's own replay_job_id (never a different operator's concurrent claim), and the caller gets 503 instead of a false success.
Replaying a row whose queue no longer exists at the BullMQ level: ModuleRef.get(getQueueToken(...), { strict: false }) returning undefined and the subsequent queue.add throwing is handled by the same enqueue-failure path as a Redis outage — JOB_FAILURE_REPLAY_ENQUEUE_FAILED.
attempts above 10 or below 1: rejected by DTO validation before the service runs.
Sends account-activation invitations to a batch of already-existing people. Call this whenever an operator wants to (re-)invite specific people to sign in — whether or not they arrived via a bulk import. An import that itself asked for invitations queues this same underlying batch automatically after it commits; this endpoint is for every other case, including re-inviting people whose earlier invitation never arrived.
Permission: Users_UPDATE — deliberately not one of the DataImport_*/DataExport_* codes; granting sign-in access is an identity change reachable without a spreadsheet.
Guest support: None.
Rate limit: None specific to this route; the actual send rate is paced inside the worker, not here.
Idempotency: No — two identical calls queue two independent batches. userIds is de-duplicated within one call (order preserved), but DataTransferRequestService.create gives every invite request its own self-unique idempotencyKey, so nothing here recognises a repeated batch as the same request.
The response is shaped by ImportRequestDto (the same DTO the import endpoints use) because DataTransferService.requestInvites reuses toImportDto; dryRun: true and the empty report are artifacts of that shared mapping rather than anything meaningful for an invite request — poll §8.3-shaped behavior via GET /data/imports/:requestId with this requestId to see the eventual outcome ({ sent, skippedNoEmail, notInvitable, failed }).
Duplicate ids in the batch: de-duplicated before anything is written, order preserved — the same person listed twice is invited once.
The 500 cap applies to this route, not to an import. An import accepts up to 5,000 data rows, so PeopleImportProcessor splits the people it created into chunks of 500 and calls this same underlying operation once per chunk, each with its own request id. A caller of this route splits the batch itself.
What the invitation actually is: an account_invite verification record valid for 7 days, and an email carrying its link. It is redeemed at POST /api/auth/password/reset, which accepts either an account_invite or a password_reset token in its token field, and redeeming it retires any other account-entry link the same person holds. The one-time code is not printed in an invitation email — the OTP branch of that endpoint matches password resets alone, so an invitation is redeemable by link only.
Someone deleted, banned, or without sign-in access by the time the worker runs is excluded from the send and counted as notInvitable in the outcome, rather than failing the batch. The worker re-reads users instead of trusting the queued payload for exactly this reason.
Duplicate request: submitting the identical userIds twice creates two independent requests and, eventually, two invitation emails per person — never deduplicated.
A listed person is deleted, banned, or has sign-in switched off by the time the worker runs: excluded from the send and counted under notInvitable in the outcome, not an error on this call.
A listed person has no email address: excluded and counted under skippedNoEmail.
Empty input: refused with VALIDATION_FAILED before anything is written.
Guest trying a logged-in-only action: N/A, no route accepts a guest at all.
Unsupported filter or sort option: N/A, no filtering on this route.
Rate-limit failure behavior: none configured specific to this route; the worker's own pacing (~500ms per person) is what keeps the school's whole traffic inside the email provider's limit.
page/pageSize (QueryDto via PaginationUtil); pagination=false is refused with 400 PAGINATION_LIMIT_INVALID — unlike every reference-table list endpoint elsewhere in the codebase
20
100
None honored — always failed_at DESC, id DESC regardless of the inherited sort/order fields
queueName (effectively fixed to people), unreplayedOnly
None beyond size
No other endpoint in this module is a list endpoint. The preview array inside an import report is capped at 10 rows (DataTransferService.PREVIEW_ROWS) but is not a paginated list — it is a fixed sample.
Shared pagination utility used: PaginationUtil.normalize/getDrizzleParams/buildMetadata (apps/api/src/common/utils/pagination.util.ts), the same one used across the codebase. Broad-search detection and relevance scoring: not applicable — this endpoint has no search behavior despite inheriting the field. Cache behavior per query: none. Empty result behavior: a valid, empty data array with zero-count pagination metadata.
This module reads directly from PostgreSQL on every call; no cache layer exists for async_requests or job_failures.
—
BullMQ
Yes
Queue QueueName.PEOPLE; jobs people.import_commit, people.export_build, people.send_invite_batch — the last has two producers, POST /data/invites and a completed import that asked for invitations; retry/backoff via the queue's env-configured defaultJobOptions.
StorageManager (@skoolsewa/storage) for both import file staging and export file generation; driver-agnostic (local disk in development, a remote driver in other environments).
No third-party HTTP call is made directly by this module — invitation email delivery is handled by the auth module's own services (VerificationTokenService, AuthEmailService), called from PeopleInviteProcessor.
Covered per-endpoint in §8: minimal and full request bodies are shown for every body-accepting endpoint (uploadImport, requestExport, sendInvites, replayJobFailure); success responses are shown for every endpoint; validation and domain-error examples are tabulated per endpoint's "Error Cases." An explicit empty-list response is shown in §8.8. No endpoint in this module supports guest or optional-auth access, so no "public/guest request" example applies anywhere.
Route ownership (§9.1), request sequence (§9.2), error decision tree (§9.3), and the replay release-on-failure branch (§9.4) are provided above; the same commit-flow shape applies to requestExport/sendInvites with their own guard/status conditions as documented in each endpoint's own error table. Auth and permission flow: identical across every route — JwtAuthGuard then RoleGuard reading @Permissions(...), as shown in §5, except that sendInvites reads Users_UPDATE rather than a DataImport/DataExport code. Data contract map, cache flow, async/job flow, realtime/event flow: see the backend doc's §16.2 and §9 for the full versions; this module has no cache flow (no cache layer) and no realtime/event flow (no Socket.IO usage).
The two-step upload/commit split; polling for both import and export completion; the alreadyExisted-style duplicate-upload message; gated export columns depend on the caller's own active role; job failures are addressed by publicId (uuid), not a numeric id; a 503 from replay means "try again," not "this row is broken"
Display report.issues per-row for a dry run; show outcome.issues after commit; treat 409 responses as "try a different action," not "retry the same request"; retry a 503 JOB_FAILURE_REPLAY_ENQUEUE_FAILED against the same publicId
Stable — this is the primary consumer the API was shaped for
Mobile app
Not currently a consumer — no mobile-specific route exists in this controller
N/A
N/A
Admin panel destructive-mutation awareness
Commit is irreversible; replay re-runs a job, it does not undo a failure
Surface the commit confirmation clearly; do not offer an "undo" affordance, since none exists
Stable
QA
The idempotency behavior (identical file = same request, not a new one) is easy to mistake for a bug; row numbers in issues are 1-based including the header row
Reproduce a specific row failure by counting spreadsheet lines including row 1 as the header
Stable
Internal service / worker
Every job must tolerate at-least-once delivery; a job's payload should be treated as possibly stale relative to the current async_requests row
Re-read async_requests before acting; never trust payload fields as more current than the row
Stable — this is a hard invariant of the outbox pattern used repo-wide
Every parent route prefix and runtime URL is documented (all under /data, no parent-module composition).
Every DTO field, nested field, enum, default, transform, and validator is documented.
Every response field, nullable field, generated field is documented; there is no "omitted raw entity field" concern since every response is an explicit DTO, not a raw entity.
Every auth, guard, permission, role, public decorator, and guest identity branch is documented (there is no public/guest branch to document — every route requires the full chain).
Every success, validation, auth, permission, not-found, conflict, and rate-limit (none configured) branch is documented; there is no server-error-specific custom code, only the framework default.
Every database read/write, cache (none), queue job, realtime event (none), notification (none from this module directly), audit log, and external call is documented.
Every route has examples for minimal request, full request (where applicable), success response, and representative failures.
Every endpoint family has route, sequence, and error diagrams; activity-diagram detail is folded into the per-endpoint error tables rather than repeated as separate diagrams for near-identical shapes.
Every tradeoff and compatibility risk is documented.