Skoolsewa - Ecommerce Docs
Developer ResourcesData transfer

Data Transfer API Reference

Complete API contracts for Data Transfer, including routes, auth, DTOs, responses, errors, examples, and integration notes.

Data Transfer - API Reference

Audience: Frontend engineers, mobile engineers, backend engineers, QA, and API consumers. Scope: Admin-facing bulk import, export, and processing dead-letter APIs owned by the data-transfer module. There is no public, mobile, or webhook surface — every route requires an authenticated, permissioned admin identity.

1. Documentation Evidence

AreaFiles InspectedWhat Was Verified
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.tsRoutes, methods, guards, decorators, status codes.
DTOsapps/api/src/modules/data-transfer/dto/data-transfer.dto.tsRequest, query, response shapes, validation, defaults.
Servicesapps/api/src/modules/data-transfer/data-transfer.service.ts, shared/data-transfer-request.service.ts, shared/dead-letter.service.tsBehavior, side effects, response mapping, errors.
Schemapackages/db/src/schema/async-request/async-requests.ts, packages/db/src/schema/jobs/job-failures.tsIDs, persisted fields, constraints.
Migrationpackages/db/src/migrations/0004_job_failures_public_id.sqlHow job_failures.public_id was introduced.
Jobspackages/jobs/src/index.ts (QueueName.PEOPLE, PeopleJob, payload types)Async job names and payload shapes triggered by these routes.
Uploadsapps/api/src/common/utils/multer.util.ts, packages/storage/src/upload-allowlist.tsUpload size cap, extension/MIME allowlist.
Paginationapps/api/src/common/dto/query.dto.ts, apps/api/src/common/utils/pagination.util.ts, apps/api/src/common/dto/response-dto.tsQuery defaults, response envelope shape.
Permissionspackages/db/src/authorization/permission-catalog.tsDataImport/DataExport modules and the five actions every module carries.
Errorsapps/api/src/common/types/error-codes.tsThe bulk-transfer error code block.

2. Module Summary

FieldValue
Module nameData Transfer
Module slugdata-transfer
Primary actorsAdmin/operator (any role holding the relevant permission), superadmin, internal worker (BullMQ)
API surfacesAdmin only — no public, mobile, or webhook route exists in this controller
Base route prefixes/data (@Controller("data"), data-transfer.controller.ts:57)
Auth modelJWT (JwtAuthGuard) + permission (RoleGuard + @Permissions(...)), applied at the controller level to every route
PersistencePostgreSQL (async_requests, job_failures, and the shared outbox_events), file storage via StorageManager, BullMQ (QueueName.PEOPLE)
Runtime source of truthasync_requests row for import/export status; job_failures row for processing failures
Sibling docsBackend, features/flows

3. Concepts and Terminology

TermMeaningSource FileUsed By
DataScopeWhich people table a bulk operation targets: "students" | "guardians" | "staff"apps/api/src/utils/data-transfer/data-transfer.types.ts:11Templates, import, export
Dry runAn upload that validates and reports but never writes to the databasedata-transfer.controller.ts:99-105Upload endpoint
CommitThe explicit, separate call that applies a validated importdata-transfer.controller.ts:139-152Commit endpoint
requestIdThe public id of an async_requests row (a UUIDv7); the id every import/export route after the initial call is keyed onshared/data-transfer-request.service.ts:74,125Import/export status, commit, download routes
async_requests.statusLifecycle state: pending | committing (import only) | completed | faileddto/data-transfer.dto.ts:60-64ImportRequestDto, ExportStatusDto
Row issueA per-row (or whole-row) validation or write problem, numbered by the operator's own spreadsheet line, header row includedapps/api/src/utils/data-transfer/data-transfer.types.ts:38-50RowIssueDto, import report/outcome
Processing dead letterA job_failures row: a job that was successfully enqueued and then failed while running, on its final BullMQ attemptpackages/db/src/schema/jobs/job-failures.ts:15-33Dead-letter endpoints
publicId (job failure)The uuid a job_failures row is addressed by; the row's serial id never leaves the databasepackages/db/src/schema/jobs/job-failures.ts:56-69JobFailureDto, the replay route's path param
payloadRefThe reference to a failed job's work (e.g. requestId), never the job's actual payloaddto/data-transfer.dto.ts:157-162JobFailureDto
Invitation batchA bounded, paced set of account-activation emails to already-existing people, requested directly or chained automatically off a completed importapps/api/src/modules/data-transfer/data-transfer.service.ts:314-370SendInvitesDto, the invites endpoint
QueueName.PEOPLEThe single BullMQ queue this module's async work runs onpackages/jobs/src/index.ts:21Commit, export request, invites, replay

4. API Surface Map

SurfaceMethodPathActorAuth/GuardPermissionControllerPurpose
AdminGET/data/templates/:scopeAdmin/operatorJwtAuthGuard, RoleGuardDataImport_READDataTransferController.downloadTemplateDownload a blank import template.
AdminPOST/data/importsAdmin/operatorJwtAuthGuard, RoleGuardDataImport_CREATEDataTransferController.uploadImportUpload a spreadsheet and get a dry-run validation report.
AdminGET/data/imports/:requestIdAdmin/operator (owner) or superadminJwtAuthGuard, RoleGuardDataImport_READDataTransferController.getImportGet an import's status and report/outcome.
AdminPOST/data/imports/:requestId/commitAdmin/operator (owner)JwtAuthGuard, RoleGuardDataImport_UPDATEDataTransferController.commitImportApply a validated import.
AdminPOST/data/exportsAdmin/operatorJwtAuthGuard, RoleGuardDataExport_CREATEDataTransferController.requestExportRequest an export.
AdminGET/data/exports/:requestIdAdmin/operator (owner) or superadminJwtAuthGuard, RoleGuardDataExport_READDataTransferController.getExportGet an export's status and download link.
AdminGET/data/exports/:requestId/downloadAdmin/operator (owner) or superadminJwtAuthGuard, RoleGuardDataExport_READDataTransferController.downloadExportDownload a built export file.
AdminPOST/data/invitesAdmin/operatorJwtAuthGuard, RoleGuardUsers_UPDATEDataTransferController.sendInvitesSend account-activation invitations to a batch of people.
AdminGET/data/job-failuresAdmin/operatorJwtAuthGuard, RoleGuardDataImport_READDataTransferController.listJobFailuresList processing dead letters (pinned to the people queue).
AdminPOST/data/job-failures/:publicId/replayAdmin/operatorJwtAuthGuard, RoleGuardDataImport_UPDATEDataTransferController.replayJobFailureRe-run a failed bulk job under a fresh id.

There are no aliases, no nested sub-resources, and no parent-module route prefix beyond /data itself — DataTransferController is the only controller mounted under this prefix.

5. Auth, Identity, and Permissions

SurfaceGuard/DecoratorIdentity ShapePermissionGuest AllowedNotes
Every route@UseGuards(JwtAuthGuard, RoleGuard) at the controller level (data-transfer.controller.ts:57)req.user populated by JwtAuthGuard; @CurrentUser() supplies AuthUser to handlers that need the actorRoute-specific @Permissions(...) (see §4)No — there is no @Public() or guest-accessible route anywhere in this controllerRoleGuard reads the @Permissions metadata and checks it against the actor's active role; a superadmin bypasses the permission check entirely (repo-wide behavior, not specific to this module)
Import/export status, commit, downloadSame guard chain, plus an in-service ownership checkAuthUser.id compared to async_requests.actor_idSame as aboveNoDataTransferService.assertOwnRequest additionally refuses anyone but the original requester unless actor.activeRole?.isSuperadmin — holding the permission is necessary but not sufficient
Export column gatingNot a guard — an in-service permission check at request timeAuthUser passed to PeoplePermissionsService.canStaffSalary_READ, StudentMedical_READ (checked, not required — their absence just narrows the resulting file)NoDecided once, when POST /data/exports runs; never re-evaluated by the worker that later builds the file
Send invitationsSame guard chain@CurrentUser() supplies actor.idUsers_UPDATE — deliberately not DataImport_*/DataExport_*NoGranting sign-in access is an identity change reachable with no spreadsheet involved; gating it on an import permission would let anyone who can import files also grant login access to arbitrary existing people

Every endpoint requires a valid JWT and requires login; none support a guest identity or optional auth.

6. DTO and Model Reference

6.1 TemplateQueryDto (query — GET /data/templates/:scope)

FieldTypeRequiredDefaultValidationExampleSource
format"csv" | "xlsx"No"xlsx"@IsIn(FILE_FORMATS)"csv"data-transfer.dto.ts:22-27

6.2 UploadImportDto (body, multipart — POST /data/imports)

FieldTypeRequiredDefaultValidationExampleSource
scope"students" | "guardians" | "staff"YesN/A@IsIn(DATA_SCOPES)"students"data-transfer.dto.ts:78-81
sendInvitesbooleanNofalse@QueryBoolean(), @IsBoolean()falsedata-transfer.dto.ts:83-91
filebinary (multipart field file)YesN/AspreadsheetMulterOptions: .csv/.xlsx only, max 10 MBdata-transfer.controller.ts:111

sendInvites is persisted on the request's stored result and read back at commit time: it decides whether every row this run creates gets can_login: true (PeopleImportWriter.apply's grantLogin parameter), and, once the commit succeeds, whether PeopleImportProcessor automatically queues an invitation batch for exactly the people it created — see §8.4 and the backend doc's runtime flow for the full sequence.

6.3 RowIssueDto

FieldTypeRequiredDefaultValidationExampleSource
rownumberYes (response)N/AN/A4data-transfer.dto.ts:29-34 — the operator's own spreadsheet line, header row counted as row 1
columnstring | nullYes (response, nullable)N/AN/A"Admission Date" or null for a whole-row problemdata-transfer.dto.ts:35-36
messagestringYes (response)N/AN/A"Admission Date is required."data-transfer.dto.ts:37
errorCodestringYes (response)N/AN/A"IMPORT_REQUIRED_FIELD_MISSING"data-transfer.dto.ts:38

6.4 ImportReportDto

FieldTypeRequiredDefaultValidationExampleSource
scopestring (enum DATA_SCOPES)YesN/AN/A"students"data-transfer.dto.ts:42
totalRowsnumberYesN/AN/A120data-transfer.dto.ts:43
validRowsnumberYesN/AN/A118data-transfer.dto.ts:44
issuesRowIssueDto[]YesN/AN/Asee §6.3data-transfer.dto.ts:45
unknownHeadersstring[]YesN/AN/A["Notes"]data-transfer.dto.ts:46
missingHeadersstring[]YesN/AN/A["Admission Date"]data-transfer.dto.ts:47
previewRecord<string,string>[]YesN/AUp to 10 rows[{ "firstName": "Sita", ... }]data-transfer.dto.ts:48-49, data-transfer.service.ts:143-145 (PREVIEW_ROWS = 10)

6.5 ImportRequestDto (response — upload, get, commit)

FieldTypeRequiredDefaultValidationExampleSource
dryRuntrue (literal)YesN/AAlways true on every response this DTO appears intruedata-transfer.dto.ts:52-58
requestIdstringYesN/AN/A"018f...-uuid"data-transfer.dto.ts:59
status"pending" | "committing" | "completed" | "failed"YesN/AN/A"committing"data-transfer.dto.ts:60-65
reportImportReportDtoYesN/AN/Asee §6.4data-transfer.dto.ts:66
createdAtDate (ISO string over the wire)YesN/AN/A"2026-04-15T09:12:00.000Z"data-transfer.dto.ts:67
completedAtDate | nullYes, nullableN/ASet only on a terminal statusnull while pending/committingdata-transfer.dto.ts:68
errorstring | nullYes, nullableN/ASet only when status: "failed", truncated to 2000 chars server-sidenulldata-transfer.dto.ts:69
outcomeRecord<string,unknown> | nullYes, nullableN/APresent only after a successful or failed commit attempt; shape is { created, failed, issues, createdUserIds }createdUserIds is what the worker hands to requestInvites when it chains an invitation batchnull before commit; {"created":115,"failed":3,"issues":[...],"createdUserIds":["018f...","018f..."]} afterdata-transfer.dto.ts:70-76

6.6 ExportRequestDto (body — POST /data/exports)

FieldTypeRequiredDefaultValidationExampleSource
scope"students" | "guardians" | "staff"YesN/A@IsIn(DATA_SCOPES)"staff"data-transfer.dto.ts:95-97
format"csv" | "xlsx"No"xlsx"@IsIn(FILE_FORMATS)"xlsx"data-transfer.dto.ts:99-102
filtersRecord<string, string | number | boolean | null>No{}Not individually validated by the DTO; the reader service applies a fixed per-scope allowlist and ignores anything else{"employmentStatus":"active"}data-transfer.dto.ts:104-109

6.7 ExportStatusDto (response — request, get, and implicitly reflected by download)

FieldTypeRequiredDefaultValidationExampleSource
requestIdstringYesN/AN/A"018f...-uuid"data-transfer.dto.ts:113
status"pending" | "completed" | "failed"YesN/AN/A"completed"data-transfer.dto.ts:114-115
downloadUrlstring | nullYes, nullableN/APresent only when status === "completed" and a storage key exists"/api/data/exports/018f.../download" or nulldata-transfer.dto.ts:116-121, data-transfer.service.ts:332-349
rowCountnumber | nullYes, nullableN/ASet once the build finishes342data-transfer.dto.ts:122
createdAtDateYesN/AN/Adata-transfer.dto.ts:123
completedAtDate | nullYes, nullableN/AN/Adata-transfer.dto.ts:124
errorstring | nullYes, nullableN/AN/Adata-transfer.dto.ts:125

downloadUrl is documented as a "short-lived signed URL" in one DTO comment (data-transfer.dto.ts:116-119), but the current implementation returns a plain path on this API (/api/data/exports/:requestId/download), never a signed object-storage URL — verified directly against DataTransferService.getExport (data-transfer.service.ts:346-349). The stored value is always a storage key, never a URL, for the reason the comment gives: a persisted signed URL would outlive its own signature.

6.8 SendInvitesDto (body — POST /data/invites)

FieldTypeRequiredDefaultValidationExampleSource
userIdsstring[] (uuid)YesN/A@IsArray(), @IsUUID(undefined, { each: true }), @ArrayNotEmpty(), @ArrayMaxSize(500)["018f2e1a-...", "018f2e1b-..."]data-transfer.dto.ts:130-144

Validated as an unversioned uuid (@IsUUID(undefined, ...)), not @IsUUID("4", ...) — every id this platform mints is a uuidv7, whose version nibble is 7, and pinning the validator to version 4 rejects every real id. userIds is de-duplicated (order preserved) and re-checked against the same 500-item cap server-side in DataTransferService.requestInvites, independently of @ArrayMaxSize, so a client-side bypass of the DTO validator still cannot exceed the batch limit.

6.9 JobFailureDto (response)

FieldTypeRequiredDefaultValidationExampleSource
publicIdstring (uuid)YesN/AN/A — the id to replay this failure by; the row's serial primary key is never exposed"018f2e1a-...-uuid"data-transfer.dto.ts:149-153
queueNamestringYesN/AN/A — always "people" in practice, since only that queue is ever listed"people"data-transfer.dto.ts:154
jobNamestringYesN/AN/A"people.import_commit"data-transfer.dto.ts:155
jobIdstring | nullYes, nullableN/AN/A"outbox_1042"data-transfer.dto.ts:156
payloadRefRecord<string,string> | nullYes, nullableN/AReference fields only — never the actual job payload{"requestId":"018f...","scope":"students"}data-transfer.dto.ts:157-162
actorIdstring | nullYes, nullableN/AN/Adata-transfer.dto.ts:163
attemptsnumberYesN/AAlways >= 13data-transfer.dto.ts:164
lastErrorstring | nullYes, nullableN/ATruncated to 4000 chars server-sidedata-transfer.dto.ts:165
replayedAtDate | nullYes, nullableN/ASet together with replayedBy/replayJobId; reset to null again if a replay's own enqueue failsnull before replaydata-transfer.dto.ts:166
replayedBystring | nullYes, nullableN/AN/Adata-transfer.dto.ts:167
replayJobIdstring | nullYes, nullableN/AAlways a fresh id, never the originaldata-transfer.dto.ts:168
failedAtDateYesN/AN/Adata-transfer.dto.ts:169

6.10 ListJobFailuresQueryDto (query — extends QueryDto)

FieldTypeRequiredDefaultValidationExampleSource
paginationbooleanNotrueInherited from QueryDto, but false is refused here specifically — DeadLetterService.findAll throws 400 PAGINATION_LIMIT_INVALID before running any query, since job_failures only ever growstruecommon/dto/query.dto.ts:14-29, dead-letter.service.ts:71-76
pagenumberNo1@Min(1), inherited1common/dto/query.dto.ts:31-36
sizenumberNo20@Min(1), @Max(100), inherited20common/dto/query.dto.ts:38-44
sortstringNo"updatedAt"Inherited — accepted but not used; DeadLetterService.findAll always orders by failedAt DESC, id DESCcommon/dto/query.dto.ts:46-49, dead-letter.service.ts:99
order"asc" | "desc"No"desc"Inherited — accepted but not used, same reasoncommon/dto/query.dto.ts:51-54
searchstringNoN/AInherited, max 120 chars — accepted but not used; this endpoint has no free-text searchcommon/dto/query.dto.ts:56-63
queueNamestringNoN/AMax 50 chars — accepted but overridden: any value other than "people" (the only entry in REPLAYABLE_QUEUES) is silently ignored in favor of "people""people"data-transfer.dto.ts:173-180, dead-letter.service.ts:296-305
unreplayedOnlybooleanNoN/A@QueryBoolean(), @IsBoolean()truedata-transfer.dto.ts:182-188

6.11 ReplayJobFailureDto (body — POST /data/job-failures/:publicId/replay)

FieldTypeRequiredDefaultValidationExampleSource
attemptsnumberNo1@IsInt(), @Min(1), @Max(10)3data-transfer.dto.ts:191-204

6.12 DataScopeParamDto

FieldTypeRequiredDefaultValidationExampleSource
scope"students" | "guardians" | "staff"YesN/A@IsEnum(DATA_SCOPES)"students"data-transfer.dto.ts:206-210

Defined for typed use elsewhere; the actual :scope path param on GET /data/templates/:scope is read as a plain @Param("scope") string and validated by an inline DATA_SCOPES.includes(...) check with a silent fallback to "students" rather than by this DTO (data-transfer.controller.ts:76,80-82).

7. Enum Reference

EnumValueMeaningRuntime EffectSource
DATA_SCOPES"students"Target the students/students+users tables.Selects the student template, parser rules, and export reader.data-transfer.dto.ts:19
DATA_SCOPES"guardians"Target the guardians/guardians+users tables.Selects the guardian template, parser rules, and export reader.data-transfer.dto.ts:19
DATA_SCOPES"staff"Target the staff/staff+users tables.Selects the staff template, parser rules, and export reader.data-transfer.dto.ts:19
FILE_FORMATS"csv"Comma-separated values, UTF-8 with BOM.Produces a .csv with text/csv; charset=utf-8.data-transfer.dto.ts:20
FILE_FORMATS"xlsx"Excel workbook.Produces a .xlsx with the OOXML spreadsheet MIME type.data-transfer.dto.ts:20
async_requests.status"pending"Validated (import) or requested (export); awaiting the next step.Import: commit is allowed. Export: the worker will build it.data-transfer.dto.ts:61-64
async_requests.status"committing"Import only — the commit job is queued or running.Blocks a second commit; the worker will proceed only from this state.data-transfer.dto.ts:61-64
async_requests.status"completed"The commit or build finished successfully.Import: outcome is populated. Export: downloadUrl becomes available.data-transfer.dto.ts:61-64
async_requests.status"failed"The commit or build could not finish.error is populated; import/export must be re-attempted from scratch (there is no retry-in-place route).data-transfer.dto.ts:61-64
PeopleJob"people.import_commit"Applies a committed import.Handled by PeopleImportProcessor.packages/jobs/src/index.ts:927
PeopleJob"people.export_build"Builds a requested export.Handled by PeopleExportProcessor.packages/jobs/src/index.ts:929
PeopleJob"people.send_invite_batch"Sends account invitations to a batch of people.Handled by PeopleInviteProcessor; enqueued via requestInvites, from POST /data/invites and, automatically, from a completed import.packages/jobs/src/index.ts:940

8. Endpoint Reference

8.1 GET /data/templates/:scope

Purpose

Returns a blank spreadsheet — one header row plus one example row — for the given scope, so an operator has the exact column set and an example of valid values before filling in real data. Call this before every first-time import of a scope, and whenever the operator wants a clean starting file rather than reusing an old one.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:72-97
DTOapps/api/src/modules/data-transfer/dto/data-transfer.dto.ts:22-27 (query)
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:81-94
Templatesapps/api/src/utils/data-transfer/templates.ts
Testsapps/api/src/utils/data-transfer/templates.spec.ts

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_READ.
  • Guest support: None.
  • Rate limit: None specific to this route.
  • Idempotency: N/A — a pure read with no side effects.

Request

PartRequiredDetails
HeadersYesAuthorization: Bearer <token>.
ParamsYesscope — any string; only "students"/"guardians"/"staff" match a real template, anything else falls back to "students" with no error.
QueryNoformat ("csv" | "xlsx", default "xlsx").
BodyNo

Response

200, streamed binary, not the JSON envelope. Headers: Content-Type (text/csv; charset=utf-8 or the OOXML spreadsheet type), Content-Disposition: attachment; filename="skoolsewa-<scope>-template.<format>", Content-Length.

Side Effects

None — no database write, no cache, no job, no notification, no audit log.

Error Cases

None specific to this route beyond the guard chain's 401/403.

Edge Cases

  • Unknown scope in the URL: silently serves the students template rather than erroring.
  • format omitted: defaults to xlsx.
  • format=csv on a school with Devanagari staff/student names in the example row: correctly rendered thanks to the UTF-8 BOM prepended to the CSV bytes.
  • The students, guardians, and staff templates each carry ten address columns — Permanent Province, Permanent District, Permanent Municipality, Permanent Ward, Permanent Tole, and the same five again for Current — matched by name against the geography lists at commit time, never accepted as ids. There is no house-number column on any template; it is settable afterward through the person forms. Leaving the whole Current group blank means "not recorded," never "same as the permanent address."

Example Requests

GET /data/templates/students?format=csv HTTP/1.1
Authorization: Bearer TOKEN
curl -X GET "$API_URL/data/templates/students?format=csv" \
  -H "Authorization: Bearer TOKEN" \
  -o skoolsewa-students-template.csv

8.2 POST /data/imports

Purpose

Uploads a spreadsheet and returns a full validation report synchronously, in the same response — total/valid row counts, every row-level issue, unknown/missing headers, and a preview. Call this every time an operator wants to check a file before committing it; nothing is written to the database by this call under any circumstances.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:106-125
DTOapps/api/src/modules/data-transfer/dto/data-transfer.dto.ts:78-92
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:98-189
Schemapackages/db/src/schema/async-request/async-requests.ts
Testsapps/api/src/utils/data-transfer/parse-people-file.spec.ts

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_CREATE.
  • Guest support: None.
  • Rate limit: None specific to this route; bounded indirectly by the 10 MB / 5,000-row caps.
  • Idempotency: Yes — keyed on sha256(file bytes) under the scope, globally (not per actor). A duplicate upload returns the earlier report rather than creating a new request.

Request

PartRequiredDetails
HeadersYesAuthorization, Content-Type: multipart/form-data.
ParamsNo
QueryNo
BodyYesMultipart: file (binary, .csv/.xlsx, ≤10 MB), scope (required), sendInvites (optional boolean).
{
  "scope": "students",
  "sendInvites": false
}

(plus the binary file field, not representable as JSON)

Response

201:

{
  "message": "File validated.",
  "data": {
    "dryRun": true,
    "requestId": "018f2e1a-....",
    "status": "pending",
    "report": {
      "scope": "students",
      "totalRows": 3,
      "validRows": 2,
      "issues": [
        { "row": 3, "column": "Admission Date", "message": "Admission Date is required.", "errorCode": "IMPORT_REQUIRED_FIELD_MISSING" }
      ],
      "unknownHeaders": [],
      "missingHeaders": [],
      "preview": [
        { "firstName": "Sita", "lastName": "Rai", "admissionDate": "2026-04-15" }
      ]
    },
    "createdAt": "2026-04-15T09:12:00.000Z",
    "completedAt": null,
    "error": null,
    "outcome": null
  },
  "errorCode": null
}

On a duplicate upload, the same shape returns with "message": "This file has already been uploaded; showing the existing report.".

Side Effects

  • Database writes: async_requests insert (new file), or none (duplicate — the existing row is simply re-read).
  • Cache: none.
  • BullMQ jobs: none — the upload path never enqueues anything.
  • Realtime events: none.
  • Analytics events: none.
  • Email/push notifications: none.
  • Audit logs: none dedicated (the row itself is the record).
  • External API calls: StorageManager.handleUpload (new file only) — writes to the configured storage driver.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
400DATA_TRANSFER_FILE_REQUIREDNo file on the multipart request.Attach a file.data-transfer.service.ts:104-107
400IMPORT_FILE_TOO_LARGEFile exceeds 10 MB.Split into smaller batches.data-transfer.service.ts:109-115
400DATA_TRANSFER_FILE_UNREADABLEBytes could not be read from disk, or storage returned no usable key.Re-upload; possible infra issue if persistent.data-transfer.service.ts:122-127, data-transfer.service.ts:416-421
409DATA_TRANSFER_REQUEST_CONFLICTThe idempotency-key insert conflicted and the conflicting row vanished before it could be re-read (a concurrent rollback).Retry the upload.shared/data-transfer-request.service.ts:111-114
400 (Nest's own validation pipe, no custom code)scope missing/invalid, or sendInvites not boolean-coercible.Fix the request body.class-validator on UploadImportDto
400 (multer, no custom code)Wrong file extension (not .csv/.xlsx).Use the correct file type.apps/api/src/common/utils/multer.util.ts:180-197

Edge Cases

  • Empty input: a file with headers only and zero data rows returns totalRows: 0, validRows: 0 — not an error.
  • Blank search: N/A, no search on this route.
  • Invalid enum in a cell: reported as IMPORT_INVALID_ENUM_VALUE, does not block the rest of the file.
  • Expired session: standard 401 from JwtAuthGuard.
  • Duplicate request: byte-identical file returns the earlier report, flagged alreadyExisted in the service layer (surfaced only via the message text, not a dedicated response field).
  • Race condition: two identical uploads racing — one wins the insert, the other reads the winner's row; no error under normal conditions.
  • Cache miss: N/A, no cache.
  • DB row missing: N/A on this route.
  • Guest trying a logged-in-only action: N/A, no route accepts a guest at all.
  • Unsupported filter or sort option: N/A, no filtering on this route.
  • Rate-limit failure behavior: none configured specific to this route.

Example Requests

POST /data/imports HTTP/1.1
Authorization: Bearer TOKEN
Content-Type: multipart/form-data; boundary=----X

------X
Content-Disposition: form-data; name="scope"

students
------X
Content-Disposition: form-data; name="file"; filename="students.xlsx"
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet

<binary>
------X--
curl -X POST "$API_URL/data/imports" \
  -H "Authorization: Bearer TOKEN" \
  -F "scope=students" \
  -F "sendInvites=false" \
  -F "file=@students.xlsx"

8.3 GET /data/imports/:requestId

Purpose

Polls an import request's current status and report/outcome. Call this after upload to confirm the dry-run report, and repeatedly after commit until status reaches completed or failed.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:127-137
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:191-195

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_READ.
  • Guest support: None.
  • Rate limit: None specific.
  • Idempotency: N/A — read-only.

Request

PartRequiredDetails
HeadersYesAuthorization.
ParamsYesrequestId — the id returned by the upload call.
QueryNo
BodyNo

Response

200, same ImportRequestDto shape as §8.2, reflecting the current status/outcome.

Side Effects

None.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
404DATA_TRANSFER_REQUEST_NOT_FOUNDNo row for requestId.Check the id.shared/data-transfer-request.service.ts:153-159
403DATA_TRANSFER_NOT_YOURSCaller is neither the owner nor a superadmin.Ask the original uploader or a superadmin.data-transfer.service.ts:433-443

Edge Cases

  • Polling before the worker has picked up a committing job: returns status: "committing", outcome: null — normal, not an error.
  • Polling after a terminal failed status: error populated, outcome may still be null if the failure happened before any row was attempted.

Example Requests

GET /data/imports/018f2e1a-.... HTTP/1.1
Authorization: Bearer TOKEN
curl -X GET "$API_URL/data/imports/018f2e1a-...." -H "Authorization: Bearer TOKEN"

8.4 POST /data/imports/:requestId/commit

Purpose

Applies a previously validated import. This is the only endpoint in the module that results in new students/guardians/staff (and users) rows. Call this only after the operator has reviewed the dry-run report and explicitly confirmed they want the file applied — there is no "undo."

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:139-152
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:206-276
Outboxapps/api/src/modules/outbox/shared/outbox.service.ts
Workerapps/api/src/modules/data-transfer/workers/people-import.processor.ts, shared/people-import-writer.service.ts
TestsN/A — no dedicated spec

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_UPDATE.
  • Guest support: None.
  • Rate limit: None specific.
  • Idempotency: Yes — the status transition pending -> committing is conditional and can succeed exactly once; a second commit call always fails with 409.

Request

PartRequiredDetails
HeadersYesAuthorization.
ParamsYesrequestId.
QueryNo
BodyNo

Response

201, ImportRequestDto with status: "committing" and outcome: null (the outcome is not yet known — poll §8.3 for the final result).

Side Effects

  • Database writes: async_requests status pending -> committing and, later (async, by the worker), -> completed/-> failed plus outcome; outbox_events insert (same transaction as the status flip); then, per valid row, users + one of students/guardians/staff (+ student_guardian for a student with guardian info).
  • Cache: none.
  • BullMQ jobs: PeopleJob.IMPORT_COMMIT enqueued via the outbox (not directly); the worker may in turn enqueue PeopleJob.SEND_INVITE_BATCH after it finishes, if sendInvites was requested and at least one row was created.
  • Realtime events: none.
  • Analytics events: none.
  • Email/push notifications: none from this HTTP call itself, but if the upload's sendInvites was true, the worker automatically queues separate invitation-batch requests once the commit succeeds (see §8.10) — invitation emails are sent from those requests' own worker runs, not from this one. An import may create up to 5,000 people while one invitation batch holds at most 500, so the people created are split into chunks of 500 and each chunk becomes its own request with its own id; a chunk that cannot be queued is counted and logged rather than silently dropped.
  • Audit logs: none dedicated.
  • External API calls: StorageManager.getFile (worker re-reads the stored file, asynchronously, not part of this HTTP call).

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
404DATA_TRANSFER_REQUEST_NOT_FOUNDNo such request.Check the id.shared/data-transfer-request.service.ts:153-159
403DATA_TRANSFER_NOT_YOURSNot the owner, not a superadmin.Only the uploader (or a superadmin) can commit.data-transfer.service.ts:433-443
409DATA_TRANSFER_NOT_COMMITTABLEStatus is not pending — already committing, completed, or failed (message varies by which).Poll status instead of retrying the commit.data-transfer.service.ts:213-221, :248-253
409IMPORT_MISSING_HEADERSThe stored report still shows missing required headers.Re-upload a corrected file.data-transfer.service.ts:224-229
409DATA_TRANSFER_HAS_BLOCKING_ISSUESvalidRows === 0.Fix the reported issues and re-upload.data-transfer.service.ts:230-236

Edge Cases

  • Empty input: N/A — a request always has an underlying file by the time it exists.
  • Duplicate request: two commit calls racing — exactly one succeeds; the transactional UPDATE ... WHERE status = 'pending' is the enforcement mechanism, not an application-level lock.
  • Race condition: covered above — this is the primary race this endpoint exists to prevent.
  • Cache miss: N/A.
  • DB row missing: 404.
  • Guest trying a logged-in-only action: N/A.
  • Unsupported filter or sort option: N/A.
  • Rate-limit failure behavior: none specific.
  • A row names an ethnicity/mother tongue not in the school's list: fails that row at write time with IMPORT_UNKNOWN_ETHNICITY/IMPORT_UNKNOWN_MOTHER_TONGUE inside outcome.issues, rather than importing the row with the column blank.
  • A row's Permanent Province/District/Municipality/Current Province/District/Municipality names a place not in the geography lists, or a district/municipality that does not belong to the parent given in the same row: fails that row with IMPORT_UNKNOWN_PLACE naming the offending column, rather than importing a half-resolved address. Each name is matched within the parent already resolved on that row — a district is looked up inside the given province, never by name alone, because two provinces can share a district name.
  • A row fills Permanent Ward/Current Ward without its municipality, or Permanent Tole/Current Tole without its district: fails with IMPORT_UNKNOWN_PLACE naming the column that is missing its parent, before the file ever reaches an insert.
  • A student row supplies neither Guardian Name nor Guardian Phone: fails that row with IMPORT_STUDENT_REQUIRES_GUARDIAN naming Guardian Name — a student import row with no guardian is refused outright, the same "at least one guardian" rule the people API enforces on POST /students with STUDENT_REQUIRES_ONE_GUARDIAN.
  • The upload's sendInvites was true: every row this commit creates gets can_login: true; if sendInvites was false (the default), every created row is login-disabled regardless of any cell in the file.

Example Requests

POST /data/imports/018f2e1a-..../commit HTTP/1.1
Authorization: Bearer TOKEN
curl -X POST "$API_URL/data/imports/018f2e1a-..../commit" -H "Authorization: Bearer TOKEN"

8.5 POST /data/exports

Purpose

Requests a new export file for a scope, optionally scoped by the same filters the operator's screen was showing. Returns immediately with a pending request; the actual file is built asynchronously. Call this whenever an operator wants a downloadable spreadsheet of current data — a second request is never deduplicated against an earlier one.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:154-164
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:280-323
Workerapps/api/src/modules/data-transfer/workers/people-export.processor.ts, shared/people-export-reader.service.ts

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataExport_CREATE.
  • Guest support: None.
  • Rate limit: None specific.
  • Idempotency: NoidempotencyKey = requestId (self-unique), so every call creates an independent request even with identical parameters.

Request

PartRequiredDetails
HeadersYesAuthorization, Content-Type: application/json.
ParamsNo
QueryNo
BodyYesscope (required), format (optional, default xlsx), filters (optional object).

Minimal:

{ "scope": "students" }

Full:

{
  "scope": "staff",
  "format": "csv",
  "filters": { "employmentStatus": "active", "departmentId": 4 }
}

Response

201:

{
  "message": "Export queued.",
  "data": {
    "requestId": "018f2e1b-....",
    "status": "pending",
    "downloadUrl": null,
    "rowCount": null,
    "createdAt": "2026-04-15T09:20:00.000Z",
    "completedAt": null,
    "error": null
  },
  "errorCode": null
}

Side Effects

  • Database writes: async_requests insert, outbox_events insert (same transaction).
  • Cache: none.
  • BullMQ jobs: PeopleJob.EXPORT_BUILD enqueued via the outbox.
  • Realtime events: none.
  • Analytics events: none.
  • Email/push notifications: none.
  • Audit logs: none dedicated.
  • External API calls: two PeoplePermissionsService.can checks (StaffSalary_READ, StudentMedical_READ) against the caller's own active role — not an external service, but a permission-system call worth noting since it is what decides the file's column set.

Error Cases

No custom error codes are thrown directly by this endpoint. Standard 400 from DTO validation (scope invalid, format invalid) applies. See §8.7 for the errors surfaced once the request has progressed.

Edge Cases

  • Empty input: filters omitted defaults to {} — no filtering applied.
  • Unsupported filter key: silently ignored by PeopleExportReader's fixed per-scope allowlist; does not error.
  • Duplicate request: never deduplicated — two identical calls produce two independent requestIds and two independent files.
  • Requester lacking a gated permission: no error; includeGatedColumns is simply empty for that request, and the resulting file omits those columns.

Example Requests

POST /data/exports HTTP/1.1
Authorization: Bearer TOKEN
Content-Type: application/json

{"scope":"students","format":"xlsx"}
curl -X POST "$API_URL/data/exports" \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"scope":"students","format":"xlsx"}'

8.6 GET /data/exports/:requestId

Purpose

Polls an export request's status. Call this repeatedly until status: "completed" (or "failed"); downloadUrl is populated only once the file is actually ready.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:166-176
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:325-350

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataExport_READ.
  • Guest support: None.
  • Idempotency: N/A — read-only.

Request

PartRequiredDetails
HeadersYesAuthorization.
ParamsYesrequestId.

Response

200, ExportStatusDto (see §6.7).

Side Effects

None.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
404DATA_TRANSFER_REQUEST_NOT_FOUNDNo such request.Check the id.shared/data-transfer-request.service.ts:153-159
403DATA_TRANSFER_NOT_YOURSNot the owner, not a superadmin.Ask the original requester.data-transfer.service.ts:433-443

Edge Cases

  • Polling before the build completes: downloadUrl: null, status: "pending".
  • Build failed: status: "failed", error populated, downloadUrl remains null permanently for this request.

Example Requests

GET /data/exports/018f2e1b-.... HTTP/1.1
Authorization: Bearer TOKEN
curl -X GET "$API_URL/data/exports/018f2e1b-...." -H "Authorization: Bearer TOKEN"

8.7 GET /data/exports/:requestId/download

Purpose

Downloads the built export file's bytes. Call this only once GET /data/exports/:requestId reports status: "completed" and a non-null downloadUrl — this route is exactly that URL's path.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:186-204
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:360-389

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataExport_READ.
  • Guest support: None.
  • Idempotency: N/A — repeated calls simply re-serve the same file.

Request

PartRequiredDetails
HeadersYesAuthorization.
ParamsYesrequestId.

Response

200, streamed binary. Headers: Content-Type, Content-Disposition: attachment; filename="skoolsewa-<scope>-<requestId>.<format>", Content-Length.

Side Effects

None beyond reading the file from storage.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
404DATA_TRANSFER_REQUEST_NOT_FOUNDNo such request.Check the id.shared/data-transfer-request.service.ts:153-159
403DATA_TRANSFER_NOT_YOURSNot the owner, not a superadmin — checked independently of the status-poll call, since this URL is guessable/shareable.Ask the original requester.data-transfer.service.ts:360-365
409DATA_TRANSFER_EXPORT_NOT_READYStatus is not completed, or no storage key is present — message distinguishes "still being built" from "failed to build."Poll until ready, or re-request if it failed.data-transfer.service.ts:368-376

Edge Cases

  • Storage file deleted out from under a completed row (operational/manual): would surface as whatever StorageManager.getFile throws — not specifically coded here.
  • Forwarded/guessed URL by a non-owner: refused regardless of whether the guesser separately holds DataExport_READ.

Example Requests

GET /data/exports/018f2e1b-..../download HTTP/1.1
Authorization: Bearer TOKEN
curl -X GET "$API_URL/data/exports/018f2e1b-..../download" \
  -H "Authorization: Bearer TOKEN" -o export.xlsx

8.8 GET /data/job-failures

Purpose

Lists processing dead letters — jobs that were enqueued successfully and then failed on their final attempt. Pinned to the people queue regardless of what queueName is passed; call this to build the "failed bulk jobs" operational screen.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:248-258
DTOapps/api/src/modules/data-transfer/dto/data-transfer.dto.ts:172-189
Serviceapps/api/src/modules/data-transfer/shared/dead-letter.service.ts:63-113
Schemapackages/db/src/schema/jobs/job-failures.ts

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_READ.
  • Guest support: None.
  • Idempotency: N/A — read-only.

Request

PartRequiredDetails
HeadersYesAuthorization.
QueryNopagination (default true; false is refused — see Error Cases), page (default 1), size (default 20, max 100), queueName (accepted, effectively ignored — always resolves to "people"), unreplayedOnly (optional boolean). sort/order/search are accepted by the shared QueryDto base but not used by this endpoint's query.

Response

200, paginated envelope:

{
  "message": "Job failures fetched.",
  "data": [
    {
      "publicId": "018f2e1c-....",
      "queueName": "people",
      "jobName": "people.import_commit",
      "jobId": "outbox_1042",
      "payloadRef": { "requestId": "018f2e1a-....", "scope": "students" },
      "actorId": "a1b2....",
      "attempts": 3,
      "lastError": "Timed out reading the stored file",
      "replayedAt": null,
      "replayedBy": null,
      "replayJobId": null,
      "failedAt": "2026-04-15T10:00:00.000Z"
    }
  ],
  "count": 1,
  "currentPage": 1,
  "totalPage": 1,
  "errorCode": null
}

Side Effects

None.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
400PAGINATION_LIMIT_INVALIDpagination=false was requested.Page through the results instead — this table only ever grows.dead-letter.service.ts:71-76

Edge Cases

  • Empty result: data: [] with valid (zero-count) pagination metadata.
  • queueName set to anything other than "people": silently ignored; results are still scoped to people.
  • unreplayedOnly=true with no unreplayed rows: empty result.
  • pagination=false: refused outright, unlike every reference-table list endpoint elsewhere in the codebase that permits it up to PaginationUtil.UNPAGINATED_HARD_CAP.

Example Requests

GET /data/job-failures?unreplayedOnly=true&size=10 HTTP/1.1
Authorization: Bearer TOKEN
curl -X GET "$API_URL/data/job-failures?unreplayedOnly=true&size=10" -H "Authorization: Bearer TOKEN"

8.9 POST /data/job-failures/:publicId/replay

Purpose

Re-runs a failed job under a fresh BullMQ job id. Call this after diagnosing a transient cause (e.g. a storage timeout) for a dead-lettered job — not for a failure caused by bad data in the source file, which needs the file fixed and re-uploaded/re-requested instead.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:260-275
DTOapps/api/src/modules/data-transfer/dto/data-transfer.dto.ts:191-204
Serviceapps/api/src/modules/data-transfer/shared/dead-letter.service.ts:115-259
Job id rulepackages/jobs/src/build-job-id.ts
Testsapps/api/src/modules/data-transfer/shared/dead-letter.service.spec.ts

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: DataImport_UPDATE.
  • Guest support: None.
  • Idempotency: The replay claim itself is idempotent — a conditional UPDATE ... WHERE replayed_at IS NULL admits exactly one caller. If the enqueue that follows the claim fails, the claim is released, and the row is claimable again — so a caller that retries after a 503 is not blocked by its own earlier attempt. A claim that does enqueue successfully can never be replayed again through this route.

Request

PartRequiredDetails
HeadersYesAuthorization, Content-Type: application/json (if a body is sent).
ParamsYespublicId — the row's public_id (a uuid), parsed with ParseUUIDPipe. The serial job_failures.id is never accepted here.
BodyNoattempts (optional, 1-10, default 1) — the attempt count the replayed job gets.

Minimal:

{}

Full:

{ "attempts": 3 }

Response

201, JobFailureDto (see §6.9) reflecting the now-claimed row, with replayedAt/replayedBy/replayJobId populated.

Side Effects

  • Database writes: job_failuresreplayed_at, replayed_by, replay_job_id set via a conditional UPDATE; reset back to null by a second UPDATE if the enqueue below then fails.
  • Cache: none.
  • BullMQ jobs: a fresh job added to the row's own queue_name (in practice always people), under buildJobId(["replay", id, timestamp]), carrying {...payloadRef, correlationId: replayJobId, replayOf: row.id}.
  • Realtime events: none.
  • Analytics events: none.
  • Notifications: none from this endpoint directly (the replayed job itself may send one, depending on which job it is).
  • Audit logs: replayed_by/replayed_at on the row serve as the audit trail while a claim holds; both are cleared again if the enqueue fails.
  • External API calls: none.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
404JOB_FAILURE_NOT_FOUNDNo job_failures row for the given publicId.Check the id.dead-letter.service.ts:138-143
409JOB_FAILURE_QUEUE_NOT_PERMITTEDThe row's queue_name is not people.This screen cannot replay it; use the owning module's tooling.dead-letter.service.ts:144-152
409JOB_FAILURE_ALREADY_REPLAYEDreplayed_at already set — checked once eagerly, and again if the claim UPDATE affects zero rows (the concurrent-replay case).Refresh the list; someone already replayed it.dead-letter.service.ts:153-158, :186-192
503JOB_FAILURE_REPLAY_ENQUEUE_FAILEDThe claim succeeded but queue.add then threw (Redis unreachable, or the queue unresolvable). The claim is released before this is thrown.Try the same replay again.dead-letter.service.ts:208-219
400 (Nest's validation pipe)publicId param not a parseable uuid.Check the id.ParseUUIDPipe on the controller

Edge Cases

  • Two operators replaying the same row simultaneously: exactly one succeeds; the other gets JOB_FAILURE_ALREADY_REPLAYED.
  • The enqueue itself fails after the claim succeeds: the claim is released, matched on the failing call's own replay_job_id (never a different operator's concurrent claim), and the caller gets 503 instead of a false success.
  • Replaying a row whose queue no longer exists at the BullMQ level: ModuleRef.get(getQueueToken(...), { strict: false }) returning undefined and the subsequent queue.add throwing is handled by the same enqueue-failure path as a Redis outage — JOB_FAILURE_REPLAY_ENQUEUE_FAILED.
  • attempts above 10 or below 1: rejected by DTO validation before the service runs.

Example Requests

POST /data/job-failures/018f2e1c-.../replay HTTP/1.1
Authorization: Bearer TOKEN
Content-Type: application/json

{"attempts": 3}
curl -X POST "$API_URL/data/job-failures/018f2e1c-.../replay" \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"attempts":3}'

8.10 POST /data/invites

Purpose

Sends account-activation invitations to a batch of already-existing people. Call this whenever an operator wants to (re-)invite specific people to sign in — whether or not they arrived via a bulk import. An import that itself asked for invitations queues this same underlying batch automatically after it commits; this endpoint is for every other case, including re-inviting people whose earlier invitation never arrived.

Source Evidence

EvidencePath
Controllerapps/api/src/modules/data-transfer/data-transfer.controller.ts:222-235
DTOapps/api/src/modules/data-transfer/dto/data-transfer.dto.ts:130-144
Serviceapps/api/src/modules/data-transfer/data-transfer.service.ts:314-370
Workerapps/api/src/modules/data-transfer/workers/people-invite.processor.ts
Jobspackages/jobs/src/index.ts (PeopleJob.SEND_INVITE_BATCH, PeopleSendInviteBatchPayload)
TestsN/A — no dedicated spec for this route or DataTransferService.requestInvites

Auth and Permissions

  • Auth: JWT required.
  • Guard chain: JwtAuthGuard, RoleGuard.
  • Permission: Users_UPDATE — deliberately not one of the DataImport_*/DataExport_* codes; granting sign-in access is an identity change reachable without a spreadsheet.
  • Guest support: None.
  • Rate limit: None specific to this route; the actual send rate is paced inside the worker, not here.
  • Idempotency: No — two identical calls queue two independent batches. userIds is de-duplicated within one call (order preserved), but DataTransferRequestService.create gives every invite request its own self-unique idempotencyKey, so nothing here recognises a repeated batch as the same request.

Request

PartRequiredDetails
HeadersYesAuthorization, Content-Type: application/json.
ParamsNo
QueryNo
BodyYesuserIds — at least one uuid, at most 500.
{ "userIds": ["018f2e1a-....", "018f2e1b-...."] }

Response

201:

{
  "message": "Invitations queued.",
  "data": {
    "dryRun": true,
    "requestId": "018f2e1d-....",
    "status": "pending",
    "report": { "scope": "invite:batch", "totalRows": 0, "validRows": 0, "issues": [], "unknownHeaders": [], "missingHeaders": [], "preview": [] },
    "createdAt": "2026-04-15T09:30:00.000Z",
    "completedAt": null,
    "error": null,
    "outcome": null
  },
  "errorCode": null
}

The response is shaped by ImportRequestDto (the same DTO the import endpoints use) because DataTransferService.requestInvites reuses toImportDto; dryRun: true and the empty report are artifacts of that shared mapping rather than anything meaningful for an invite request — poll §8.3-shaped behavior via GET /data/imports/:requestId with this requestId to see the eventual outcome ({ sent, skippedNoEmail, notInvitable, failed }).

Side Effects

  • Database writes: async_requests insert (scope invite:batch), outbox_events insert — same transaction.
  • Cache: none.
  • BullMQ jobs: PeopleJob.SEND_INVITE_BATCH enqueued via the outbox.
  • Realtime events: none.
  • Analytics events: none.
  • Email/push notifications: none from this call itself — the worker sends them, per person, paced ~500ms apart, once it runs.
  • Audit logs: none dedicated (the row itself, and its actorId, are the record).
  • External API calls: none from this call; the worker later calls VerificationTokenService/AuthEmailService.

Error Cases

HTTP StatusError CodeConditionUser-Facing MeaningSource
400VALIDATION_FAILEDuserIds is empty after de-duplication.Choose at least one person to invite.data-transfer.service.ts:324-329
400INVITE_BATCH_TOO_LARGEMore than 500 unique userIds.Split the batch across multiple requests.data-transfer.service.ts:330-335
400 (Nest's validation pipe)userIds missing, empty at the DTO level, over 500 entries, or containing a non-uuid value.Fix the request body.class-validator on SendInvitesDto

Edge Cases

  • Duplicate ids in the batch: de-duplicated before anything is written, order preserved — the same person listed twice is invited once.
  • The 500 cap applies to this route, not to an import. An import accepts up to 5,000 data rows, so PeopleImportProcessor splits the people it created into chunks of 500 and calls this same underlying operation once per chunk, each with its own request id. A caller of this route splits the batch itself.
  • What the invitation actually is: an account_invite verification record valid for 7 days, and an email carrying its link. It is redeemed at POST /api/auth/password/reset, which accepts either an account_invite or a password_reset token in its token field, and redeeming it retires any other account-entry link the same person holds. The one-time code is not printed in an invitation email — the OTP branch of that endpoint matches password resets alone, so an invitation is redeemable by link only.
  • Someone deleted, banned, or without sign-in access by the time the worker runs is excluded from the send and counted as notInvitable in the outcome, rather than failing the batch. The worker re-reads users instead of trusting the queued payload for exactly this reason.
  • Duplicate request: submitting the identical userIds twice creates two independent requests and, eventually, two invitation emails per person — never deduplicated.
  • A listed person is deleted, banned, or has sign-in switched off by the time the worker runs: excluded from the send and counted under notInvitable in the outcome, not an error on this call.
  • A listed person has no email address: excluded and counted under skippedNoEmail.
  • Empty input: refused with VALIDATION_FAILED before anything is written.
  • Guest trying a logged-in-only action: N/A, no route accepts a guest at all.
  • Unsupported filter or sort option: N/A, no filtering on this route.
  • Rate-limit failure behavior: none configured specific to this route; the worker's own pacing (~500ms per person) is what keeps the school's whole traffic inside the email provider's limit.

Example Requests

POST /data/invites HTTP/1.1
Authorization: Bearer TOKEN
Content-Type: application/json

{"userIds":["018f2e1a-....","018f2e1b-...."]}
curl -X POST "$API_URL/data/invites" \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"userIds":["018f2e1a-....","018f2e1b-...."]}'

9. Flow Diagrams

9.1 Route Ownership

9.2 Request Sequence — commit (representative of the async-mutation family)

9.3 Error Branch — commit

9.4 Request Sequence — replay (the release-on-failure branch)

EndpointPagination TypeDefault SizeMax SizeSort FieldsFiltersResult Cap
GET /data/job-failurespage/pageSize (QueryDto via PaginationUtil); pagination=false is refused with 400 PAGINATION_LIMIT_INVALID — unlike every reference-table list endpoint elsewhere in the codebase20100None honored — always failed_at DESC, id DESC regardless of the inherited sort/order fieldsqueueName (effectively fixed to people), unreplayedOnlyNone beyond size

No other endpoint in this module is a list endpoint. The preview array inside an import report is capped at 10 rows (DataTransferService.PREVIEW_ROWS) but is not a paginated list — it is a fixed sample.

Shared pagination utility used: PaginationUtil.normalize/getDrizzleParams/buildMetadata (apps/api/src/common/utils/pagination.util.ts), the same one used across the codebase. Broad-search detection and relevance scoring: not applicable — this endpoint has no search behavior despite inheriting the field. Cache behavior per query: none. Empty result behavior: a valid, empty data array with zero-count pagination metadata.

11. Caching, Jobs, and External Integrations

IntegrationUsed?DetailsSource
Redis cacheNoThis module reads directly from PostgreSQL on every call; no cache layer exists for async_requests or job_failures.
BullMQYesQueue QueueName.PEOPLE; jobs people.import_commit, people.export_build, people.send_invite_batch — the last has two producers, POST /data/invites and a completed import that asked for invitations; retry/backoff via the queue's env-configured defaultJobOptions.packages/jobs/src/index.ts:920-940, apps/api/src/modules/data-transfer/workers/people-queue.processor.ts
Transactional outboxYesEvery enqueue from this module's request handlers goes through OutboxService.enqueue, never a direct queue.add().apps/api/src/modules/outbox/shared/outbox.service.ts
File storageYesStorageManager (@skoolsewa/storage) for both import file staging and export file generation; driver-agnostic (local disk in development, a remote driver in other environments).apps/api/src/modules/data-transfer/data-transfer.service.ts
External APINoNo third-party HTTP call is made directly by this module — invitation email delivery is handled by the auth module's own services (VerificationTokenService, AuthEmailService), called from PeopleInviteProcessor.

13. Mandatory Deep API Documentation Pack

13.1 Route-by-Route Completeness Matrix

RouteController MethodDTOsService MethodGuardsPermissionsCacheJobsDB TouchesErrorsTestsDocumented?
GET /data/templates/:scopedownloadTemplateTemplateQueryDto (query)buildTemplateJwtAuthGuard, RoleGuardDataImport_READN/AN/ANoneNone customtemplates.spec.tsYes
POST /data/importsuploadImportUploadImportDto, response ImportRequestDtouploadForDryRunJwtAuthGuard, RoleGuard, FileInterceptorDataImport_CREATEN/AN/Aasync_requestsDATA_TRANSFER_FILE_REQUIRED, IMPORT_FILE_TOO_LARGE, DATA_TRANSFER_FILE_UNREADABLE, DATA_TRANSFER_REQUEST_CONFLICTparse-people-file.spec.ts (parser only)Yes
GET /data/imports/:requestIdgetImportresponse ImportRequestDtogetImportJwtAuthGuard, RoleGuardDataImport_READN/AN/Aasync_requests (read)DATA_TRANSFER_REQUEST_NOT_FOUND, DATA_TRANSFER_NOT_YOURSNoneYes
POST /data/imports/:requestId/commitcommitImportresponse ImportRequestDtocommitImportJwtAuthGuard, RoleGuardDataImport_UPDATEN/AIMPORT_COMMIT (via outbox)async_requests, outbox_events, then (async) users/students/guardians/staff/student_guardianDATA_TRANSFER_REQUEST_NOT_FOUND, DATA_TRANSFER_NOT_YOURS, DATA_TRANSFER_NOT_COMMITTABLE, IMPORT_MISSING_HEADERS, DATA_TRANSFER_HAS_BLOCKING_ISSUESNoneYes
POST /data/exportsrequestExportExportRequestDto, response ExportStatusDtorequestExportJwtAuthGuard, RoleGuardDataExport_CREATEN/AEXPORT_BUILD (via outbox)async_requests, outbox_eventsNone customNoneYes
GET /data/exports/:requestIdgetExportresponse ExportStatusDtogetExportJwtAuthGuard, RoleGuardDataExport_READN/AN/Aasync_requests (read)DATA_TRANSFER_REQUEST_NOT_FOUND, DATA_TRANSFER_NOT_YOURSNoneYes
GET /data/exports/:requestId/downloaddownloadExportreadExportFileJwtAuthGuard, RoleGuardDataExport_READN/AN/Aasync_requests (read)DATA_TRANSFER_REQUEST_NOT_FOUND, DATA_TRANSFER_NOT_YOURS, DATA_TRANSFER_EXPORT_NOT_READYNoneYes
POST /data/invitessendInvitesSendInvitesDto, response ImportRequestDtorequestInvitesJwtAuthGuard, RoleGuardUsers_UPDATEN/ASEND_INVITE_BATCH (via outbox)async_requests, outbox_eventsVALIDATION_FAILED, INVITE_BATCH_TOO_LARGENoneYes
GET /data/job-failureslistJobFailuresListJobFailuresQueryDto, response JobFailureDto[]DeadLetterService.findAllJwtAuthGuard, RoleGuardDataImport_READN/AN/Ajob_failures (read)PAGINATION_LIMIT_INVALIDNoneYes
POST /data/job-failures/:publicId/replayreplayJobFailureReplayJobFailureDto, response JobFailureDtoDeadLetterService.replayJwtAuthGuard, RoleGuardDataImport_UPDATEN/AEnqueues a fresh job on the row's queuejob_failuresJOB_FAILURE_NOT_FOUND, JOB_FAILURE_QUEUE_NOT_PERMITTED, JOB_FAILURE_ALREADY_REPLAYED, JOB_FAILURE_REPLAY_ENQUEUE_FAILEDdead-letter.service.spec.tsYes

Every @Get/@Post in data-transfer.controller.ts appears above; there are no @Patch, @Put, or @Delete routes in this controller.

13.2 Request/Response Exhaustiveness

Covered per-endpoint in §8: minimal and full request bodies are shown for every body-accepting endpoint (uploadImport, requestExport, sendInvites, replayJobFailure); success responses are shown for every endpoint; validation and domain-error examples are tabulated per endpoint's "Error Cases." An explicit empty-list response is shown in §8.8. No endpoint in this module supports guest or optional-auth access, so no "public/guest request" example applies anywhere.

13.3 API Diagram Pack

Route ownership (§9.1), request sequence (§9.2), error decision tree (§9.3), and the replay release-on-failure branch (§9.4) are provided above; the same commit-flow shape applies to requestExport/sendInvites with their own guard/status conditions as documented in each endpoint's own error table. Auth and permission flow: identical across every route — JwtAuthGuard then RoleGuard reading @Permissions(...), as shown in §5, except that sendInvites reads Users_UPDATE rather than a DataImport/DataExport code. Data contract map, cache flow, async/job flow, realtime/event flow: see the backend doc's §16.2 and §9 for the full versions; this module has no cache flow (no cache layer) and no realtime/event flow (no Socket.IO usage).

13.4 Consumer Integration Notes

ConsumerRequired KnowledgeFailure HandlingContract Stability
Web admin frontendThe two-step upload/commit split; polling for both import and export completion; the alreadyExisted-style duplicate-upload message; gated export columns depend on the caller's own active role; job failures are addressed by publicId (uuid), not a numeric id; a 503 from replay means "try again," not "this row is broken"Display report.issues per-row for a dry run; show outcome.issues after commit; treat 409 responses as "try a different action," not "retry the same request"; retry a 503 JOB_FAILURE_REPLAY_ENQUEUE_FAILED against the same publicIdStable — this is the primary consumer the API was shaped for
Mobile appNot currently a consumer — no mobile-specific route exists in this controllerN/AN/A
Admin panel destructive-mutation awarenessCommit is irreversible; replay re-runs a job, it does not undo a failureSurface the commit confirmation clearly; do not offer an "undo" affordance, since none existsStable
QAThe idempotency behavior (identical file = same request, not a new one) is easy to mistake for a bug; row numbers in issues are 1-based including the header rowReproduce a specific row failure by counting spreadsheet lines including row 1 as the headerStable
Internal service / workerEvery job must tolerate at-least-once delivery; a job's payload should be treated as possibly stale relative to the current async_requests rowRe-read async_requests before acting; never trust payload fields as more current than the rowStable — this is a hard invariant of the outbox pattern used repo-wide

13.5 API Tradeoffs and Rationale

DecisionChosen BehaviorAlternatives ConsideredWhy This TradeoffRiskMitigation
Upload/commit split across two endpointsPOST /data/imports (dry run) then POST /data/imports/:id/commit (apply)A single endpoint that both validates and writesAn irreversible bulk write must have an explicit confirmation stepAn operator could commit without re-reading the reportUI-level concern; the API at least requires two distinct calls with two distinct permissions
Export download is a same-API path, not a signed storage URLGET /data/exports/:requestId/download, behind the full guard chainA time-limited signed URL from the storage driverA signed URL bypasses RoleGuard for anyone it is forwarded to, and is unsupported by the local storage driver entirelyNone materialThis is a permanent, deliberate choice
Export requests are never deduplicatedEvery POST /data/exports call creates a new requestIdempotency keyed on (scope, filters, format) like imports are keyed on file contentA later request for the same parameters is a request for current data; deduplicating would risk serving stale data silentlyA caller that retries a request naively (e.g. on a network blip) gets two independent buildsAcceptable — exports are cheap relative to imports, and the caller controls retry behavior
Dead-letter replay is scoped to one hardcoded queueREPLAYABLE_QUEUES = [QueueName.PEOPLE], ignoring the queueName query param for any other valueTrusting the caller-supplied queueNamejob_failures is shared, global infrastructure; trusting the query string would let one permission (DataImport_UPDATE) replay any queue's failuresNone material within this module's own scopeExtending to a second queue requires an explicit code change to the array, not a config toggle
Dead-letter rows are addressed by public_id, never the serial idA sequential id lets anyone with DataImport_READ enumerate every job the platform has ever failedPOST /data/job-failures/:publicId/replay, ParseUUIDPipeKeep the serial id as the route paramThe count and pace of failures is itself operational information this screen's own queue filter already exists to withholdNone — the serial id never appears in any response
A replay's claim is released if its enqueue failsClaim-then-enqueue is the safe ordering against concurrent operators, but leaves a claim that can outlive a failed enqueueRelease the claim (replayed_at/replayed_by/replay_job_id back to null) and return 503Leave the claim in place and require manual database interventionA claim left in place can never be replayed again through this route, since replayed_at is exactly what the already-replayed check readsThe release matches on replay_job_id as well as the row id, so a concurrent successful claim from a different operator is never clobbered
Sending invitations is a separate route gated on Users_UPDATEGranting sign-in access is an identity change, distinct from importing or exporting recordsPOST /data/invites, Users_UPDATEFold invitation-sending into the import commit permissionAn import permission would let anyone who can import files also grant login access to arbitrary existing peopleNone — the two capabilities are cleanly separated

13.6 API Change Impact

ChangeAffected ConsumersBackend ImpactData ImpactMigration Needed?Compatibility Plan
Renaming/removing an ImportReportDto/ExportStatusDto fieldWeb admin frontend (the only current consumer)Response mapper (toImportDto/toExportDto) updatedNone — these are computed response shapes, not stored columns directlyNoCoordinate with the frontend before removing a field it reads
Changing the idempotency scope of imports (e.g. to per-actor)Every operator relying on the current global-dedupe behaviorasync_requests_idempotency_idx schema changeRequires a migration and a decision on existing rowsYesWould need an explicit plan — this is called out as a deliberate, permanent choice in the backend doc, not expected to change
Adding a real "truncated" flag to ExportStatusDto when the 20,000-row cap is hitAny consumer currently inferring truncation from rowCount alonePeopleExportProcessor sets an additional result fieldNone to existing rowsNoAdditive; existing consumers unaffected
Extending REPLAYABLE_QUEUES to a second queueAny consumer relying on JOB_FAILURE_QUEUE_NOT_PERMITTED for every non-people failure todayDeadLetterService's hardcoded array grows by one entryNone to existing rowsNoAdditive — a previously-refused queue starts succeeding; no existing behavior changes

14. Zero-Omission API Checklist

  • Every controller route is documented.
  • Every parent route prefix and runtime URL is documented (all under /data, no parent-module composition).
  • Every DTO field, nested field, enum, default, transform, and validator is documented.
  • Every response field, nullable field, generated field is documented; there is no "omitted raw entity field" concern since every response is an explicit DTO, not a raw entity.
  • Every auth, guard, permission, role, public decorator, and guest identity branch is documented (there is no public/guest branch to document — every route requires the full chain).
  • Every success, validation, auth, permission, not-found, conflict, and rate-limit (none configured) branch is documented; there is no server-error-specific custom code, only the framework default.
  • Every database read/write, cache (none), queue job, realtime event (none), notification (none from this module directly), audit log, and external call is documented.
  • Every route has examples for minimal request, full request (where applicable), success response, and representative failures.
  • Every endpoint family has route, sequence, and error diagrams; activity-diagram detail is folded into the per-endpoint error tables rather than repeated as separate diagrams for near-identical shapes.
  • Every tradeoff and compatibility risk is documented.
  • The API doc links to backend and features/flows.

15. Integration Checklist

  • Every route from controllers is documented.
  • Every DTO field is documented.
  • Every enum value is documented.
  • Every response envelope is documented.
  • Every error code is documented.
  • Every auth guard and permission is documented.
  • Every cache key (none), queue job, realtime event (none), and external call is documented.
  • Every diagram matches the current code.
  • The API doc links to backend and features/flows.

See Also

On this page

Data Transfer - API Reference1. Documentation Evidence2. Module Summary3. Concepts and Terminology4. API Surface Map5. Auth, Identity, and Permissions6. DTO and Model Reference6.1 TemplateQueryDto (query — GET /data/templates/:scope)6.2 UploadImportDto (body, multipart — POST /data/imports)6.3 RowIssueDto6.4 ImportReportDto6.5 ImportRequestDto (response — upload, get, commit)6.6 ExportRequestDto (body — POST /data/exports)6.7 ExportStatusDto (response — request, get, and implicitly reflected by download)6.8 SendInvitesDto (body — POST /data/invites)6.9 JobFailureDto (response)6.10 ListJobFailuresQueryDto (query — extends QueryDto)6.11 ReplayJobFailureDto (body — POST /data/job-failures/:publicId/replay)6.12 DataScopeParamDto7. Enum Reference8. Endpoint Reference8.1 GET /data/templates/:scopePurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.2 POST /data/importsPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.3 GET /data/imports/:requestIdPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.4 POST /data/imports/:requestId/commitPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.5 POST /data/exportsPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.6 GET /data/exports/:requestIdPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.7 GET /data/exports/:requestId/downloadPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.8 GET /data/job-failuresPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.9 POST /data/job-failures/:publicId/replayPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests8.10 POST /data/invitesPurposeSource EvidenceAuth and PermissionsRequestResponseSide EffectsError CasesEdge CasesExample Requests9. Flow Diagrams9.1 Route Ownership9.2 Request Sequence — commit (representative of the async-mutation family)9.3 Error Branch — commit9.4 Request Sequence — replay (the release-on-failure branch)10. Pagination, Sorting, Filtering, and Search11. Caching, Jobs, and External Integrations13. Mandatory Deep API Documentation Pack13.1 Route-by-Route Completeness Matrix13.2 Request/Response Exhaustiveness13.3 API Diagram Pack13.4 Consumer Integration Notes13.5 API Tradeoffs and Rationale13.6 API Change Impact14. Zero-Omission API Checklist15. Integration ChecklistSee Also