Implement passwordless auth with NextAuth.js and Scalekit

TL:DR;

• Clear authority split: Scalekit verifies email ownership (OTP or magic link) entirely server-side, while NextAuth v5 is the sole session issuer, preventing client tampering and mixed states.
• Minimal, short-lived tokens: JWTs carry only ^sub and ^email with a short ^maxAge; the session callback mirrors email; OTPs, link tokens, and authRequestId never enter JWTs/sessions, simplifying audits and limiting blast radius.
• Safe edge gating: Middleware default-denies and allow-lists only ^{/passwordless/verify}, ^{/api/auth/passwordless/send}, NextAuth routes, and static assets; it calls ^auth() once and restricts next redirects to same-origin, keeping magic links functional.
• Extensible without churn: Prisma can later add profiles/roles without changing the JWT strategy; OAuth providers coexist by linking only on verified emails, with redirect allow-lists to avoid collisions.
• Types and tests as guardrails: TypeScript augmentation plus XOR-typed verify inputs remove casts and catch flow errors; unit/integration/Playwright tests lock send/resend/verify/guard and mirror Scalekit rate/attempt limits.

End-of-quarter close made passwordless the only sane choice

LedgerLoop, our fictional SaaS built on Next.js, still relied on a mix of SAML for a few big tenants, a legacy email-password flow for everyone else, and a GitHub button engineers preferred. A phishing drill forced mass resets mid-close, help-desk tickets spiked, and security flagged oversized tokens moving through laptops that IT locked down. End-of-quarter close turned access into a hazard. Finance needed uninterrupted dashboards; engineering was deploying fixes; audit logs were under a microscope.

Authentication fragmentation amplified risk. Password resets collided with deadlines, BYOD browsers broke OTP autofill, and shared credentials surfaced during audits. Route guards blocked legitimate magic-link landings, while logs spread across services made post-mortems slow. Compliance wanted shorter tokens and clearer trails; product wanted one-tap sign-in that worked on managed devices without compromising origin checks.

Scalekit plus NextAuth v5 stabilized the lane. Scalekit proved email ownership with OTPs or magic links over a server-only SDK. A custom NextAuth Credentials provider verified those artifacts and returned a minimal user. JWT callbacks issued short-lived tokens with just sub and email. Middleware default-denied everything yet allow-listed the verify and send endpoints so links landed cleanly. Optional Prisma and OAuth stayed behind env flags, preserving a DB-less baseline until roles and profiles actually arrived.

By the end of this guide, you’ll wire a custom NextAuth v5 provider that verifies with Scalekit, project a minimal identity through session callbacks, shape JWT lifetimes and rotation for audits, fit OAuth alongside passwordless without collisions, add Prisma only when profiles exist, harden middleware for safe redirects and magic-link entry, and lock the whole spine with concise unit, integration, and Playwright end-to-end tests.

Frame the problem before we touch code

From the intro, the real issue wasn’t UI friction, it was ambiguity along the auth path. Four pressure points kept failing in production: proving identity, issuing sessions, guarding routes, and observing what actually happened. Below is a compact map from symptom to the smallest effective countermeasure we’ll ship now. 

Two guardrails keep this lean design safe: mirror Scalekit’s rate/attempt limits in the UI and keep tokens tiny to limit blast radius; enforce magic-link “same origin” by forwarding ^{authRequestId} only on the server. With scope and ownership settled, we can build the smallest working spine, send → verify → issue → guard, and then layer adapters, OAuth, and tests without rework.  

From blank page to a minimal working flow

Start small, ship fast. The first cut was a narrow passwordless path: one API route to send the email, a custom NextAuth Credentials provider that verifies with Scalekit, a tiny JWT (^sub, ^email) projected into the session, and middleware that default-denies while allowing the verify and send endpoints. Roles, profiles, and social login stayed behind environment flags until the spine proved stable.

This diagram shows both OTP and magic-link sign-in from send → verify → JWT issue → guarded navigation.  

1.  Send step (shared): Client calls ^{POST /api/auth/passwordless/send { email }}. The API calls ^{sc.passwordless.sendPasswordlessEmail(...)} and returns ^{{ authRequestId, expiresIn, passwordlessType }}. UI either shows an OTP form or “check your email.”

2. OTP path: Client submits ^{signIn('scalekit', { email, code, authRequestId })}. The provider verifies with ^{verifyPasswordlessEmail({ code }, authRequestId) and returns { email }}. NextAuth v5 issues a signed JWT (^sub, ^email) and sets the session cookie.

3. Magic-link path: User lands on ^{/passwordless/verify?link_token=…&email=…&ari=….} That page immediately calls ^{signIn('scalekit', { email, linkToken, authRequestId })}. The provider verifies ^{{ linkToken }} and NextAuth issues the same minimal JWT/session.

4. Navigation after sign-in: Requests to protected routes hit Middleware. auth() reads the session; if present, allow; if absent, redirect to /login?next=…. Verify/send/auth/static paths are allow-listed so links don’t get blocked.

5. Security notes baked in: Verification runs server-side only; OTPs/link tokens/request IDs never enter JWT/session. authRequestId pairs the magic link to its origin, and Scalekit’s rate/attempt limits are mirrored in the UI using expiresIn.

At a glance: one API to send, one provider to verify, a tiny JWT to issue trust, and Middleware to guard everything else.

Establishing a clear contract for the “send email” step

We just mapped the end-to-end spine. The very first brick is a server-only endpoint that kicks off passwordless and returns exactly what the UI needs to proceed.

What you’re building (in this step): a minimal ^{POST /api/auth/passwordless/send} that calls Scalekit and returns ^{{ authRequestId, expiresIn, passwordlessType }}. This single contract drives OTP and magic-link UX and keeps verification artifacts off the client.  

The endpoint (server-only)

With the above code:

• The app can initiate a passwordless challenge without holding secrets in the browser.
• We have a stable handshake for the next steps (OTP form or magic-link landing) via authRequestId and expiresIn.
• Email content/transport stay in Scalekit; NextAuth is not coupled to sending.

Quick manual check (optional, paste in a terminal):

Request/response at a glance

Visual: send/resend with visible backoff

The diagram below shows how resend behaves: 429s when you’re too early, the UI disables the button and shows a countdown from expiresIn, then resend succeeds when the window opens.

UX hooks to implement now (tiny but important):

• Start a countdown from expiresIn after the first send.
• On HTTP 429, disable the button and show “Try again in N seconds.”
• When the timer hits zero, call resend with the same authRequestId and restart the countdown.

Exit criteria for this step:

1) The endpoint returns the fields above; 2) the UI renders OTP or “check your email” based on passwordlessType; 3) resend is rate-limited and visibly backed off; 4) no OTPs/link tokens/request IDs are stored in JWT/session; 5) logs capture email_sent with no secrets.

Verify identity in a custom NextAuth provider

After the send endpoint hands the UI an authRequestId and a countdown, the next hop is to prove email ownership once (OTP or magic link) so NextAuth can mint a session. Verification must happen server-side to prevent tampering and to keep artifacts (codes, link tokens) out of JWTs.

What you’re building: a tiny Credentials provider (id: "scalekit") whose sole job is to call Scalekit’s verify API and, on success, return { id, email }. It doesn’t send emails, redirect, or persist anything. 

Why verify? (and why server-side)

• Trust boundary: Only the server should contact Scalekit; otherwise a client could forge a “verified” state.
• Audit & minimal PII: We return just ^{{ id, email };} no OTPs or link tokens enter JWT/session.
• Same-origin enforcement: Passing ^{authRequestId} at verify blocks cross-browser/link-replay when enabled.

Why the XOR rule (OTP or linkToken, never both)?

• Correctness: Scalekit treats them as alternate proofs. Accepting both hides bugs (e.g., UI double-submits).
• Security: One clear proof prevents confused-deputy scenarios and simplifies audit trails.
• DX: Fail fast with ^{null → ?error=CredentialsSignin}, so the UI can show “Incorrect code” or “Link expired → resend.”

How the UI calls it

What this proves (and how to smoke-test)

• Proves: send → verify → issue (JWT) now works end-to-end with minimal claims.
• Quick run:
  
  1. Hit ^{/api/auth/passwordless/send} for your email.
  2. Complete OTP or visit the magic-link landing (it calls ^{signIn("scalekit", …)}).
  3. Expect redirect to ^/dashboard, a ^{next-auth.session-token cookie}, and ^{/api/auth/session} returning ^{{ user: { email } }}.

Failure mapping, you’ll see

• Wrong/expired OTP or consumed link → null → NextAuth redirects with ^{?error=CredentialsSignin}.
• Misconfig/network issues → thrown error (logged server-side), UI gets a generic failure message.

This provider stays intentionally “boring”: verify once, return ^{{ id, email }}, and hand control to the JWT/session callbacks you set up next.

Wiring the magic-link landing without user friction

You just built the verify capability inside the custom provider (it expects ^{{ email, linkToken, authRequestId })}. Now you need a public landing route where Scalekit’s magic link can drop the user so the client can hand those three values to ^{signIn("scalekit", …)} and let the server do the verification.

Why this page exists:

• Magic links open in a fresh browser context; the app must capture the token and forward it to your provider.
• Keeping this path public lets middleware stay strict everywhere else while still allowing verification to complete.
• Pairing linkToken with authRequestId at this step enables same-origin protection (replay across browsers gets blocked server-side).

What “done” looks like:

• Opening ^{/passwordless/verify?link_token=…&email=…&ari=…} redirects to ^/dashboard.
• A ^{next-auth.session-token cookie} appears; ^{/api/auth/session returns { user: { email } }}.
• Middleware continues to default-deny but allows ^{/passwordless/verify} so links never bounce.

If something’s wrong:

• Bad/expired token → provider returns null → NextAuth redirects with ^{?error=CredentialsSignin;} the verify page can show “Link expired, resend” with a button that calls your send endpoint.

‍

1. Click: User hits ^{/passwordless/verify?link_token=…&email=…&ari=…} from email.
2. Forward: The page calls ^{signIn('scalekit', { email, linkToken, authRequestId })}.
3. Verify: NextAuth’s Credentials provider calls ^{verifyPasswordlessEmail({ linkToken }, authRequestId)} on the server.
4. Success: Scalekit returns ^{{ email } → NextAuth} issues the JWT ^{{ sub, email }} → redirect to ^{callbackUrl (e.g., /dashboard)}.
5. Fail: Missing/invalid ^{authRequestId} or expired link → 400/403 → NextAuth redirects with ^{?error=CredentialsSignin} → UI shows “Link expired, resend.”

Magic-link verification pairs link_token with authRequestId to enforce same-origin.

Issuing a minimal JWT and projecting it into the session

Small tokens simplify audits. LedgerLoop stores only email (and aligns sub), so downstream services do not learn more than they must.

Guarding pages while keeping verification routes open

Middleware should whitelist the one path that magic links need. Everything else remains protected to keep quarter-close quiet.

Where we deepen next without rework

Deeper work builds on this spine. With the baseline running, we can layer type augmentation for friction-free UI, optional Prisma and OAuth for richer accounts, and pragmatic tests that exercise OTP errors, link expiry, and guarded redirects, exactly the failure modes that hurt LedgerLoop during close.

Middleware default-denies, allow-lists verify/send/auth, and builds a safe next redirect.

Designing session callbacks for predictable, minimal identity

Session callbacks decide what the client can trust. The goal is to project only a stable identity into the session while keeping secrets and transient data out. Map the verified email to both ^token.email and ^token.sub during the JWT callback, then mirror that into ^{session.user.email} in the session callback. Keeping the surface area this small makes audits simpler, reduces accidental data exposure in client bundles, and keeps downstream authorization logic deterministic even when additional providers or a database adapter are added later.

Callbacks should be short, explicit, and boring. The JWT callback runs on every sign-in and on subsequent requests; use it to set or preserve email and align sub. The session callback runs before the session is sent to the client; copy the same email onto the session.user so components can rely on a single field. Avoid embedding roles or profile details until a database adapter exists; when you add one, resolve them lazily on the server instead of bloating tokens.

Claims need a clear contract and reason. The table below captures exactly what travels where and why. Anything not in this table should stay server-side until a later phase introduces it deliberately, with tests and audit notes.

Common mistakes are easy to avoid with three rules. Never store Scalekit request IDs, link tokens, or OTPs in the JWT or session; those are verification artifacts, not identity. Never widen claims during unrelated features; keep a single place to add new fields with tests and reviewers from security. Never rely on client decoding of JWTs for authorization; use auth() on the server, and let middleware act on the already-verified session rather than parsing headers by hand.

Handling JWT tokens for short, auditable sessions without secrets

JWT purpose: JWTs represent the verified user, not the verification artifacts. NextAuth v5 signs the token; Scalekit never appears inside it. Keep claims minimal, sub, and email, so tokens are small, predictable, and easy to reason about in audits. Avoid names, avatars, roles, or request identifiers until you add a database-backed profile layer.

JWT lifetime: Token lifetime should be bound to the blast radius. Configure a reasonable max age in NextAuth and rely on server-side checks (auth()) for every sensitive request. Short lifetimes reduce exposure from leaked tokens and make secret rotation practical if your app needs “remember me,” encode that through separate cookie settings rather than inflating claims.

JWT revocation: Stateless tokens cannot be selectively revoked without help. Use one of three strategies: rotate the signing secret to invalidate all tokens, add a server-checked tokenVersion claim when a database exists, or maintain a short-lived denylist in cache for incident response. Start with short lifetimes; add versioning once profiles land.

JWT usage: Server usage should flow through auth() instead of manual header parsing. Middleware should gate access based on the session object, not by decoding JWTs on the edge. Client components should read session.user.email only; authorization decisions belong on the server where secrets and policies live.

Adding a database adapter only when profiles actually exist

Adapter goal: Add a user/profile store without changing the passwordless spine. Keep the session strategy as JWT, let the Prisma adapter manage User and Account rows for profiles and future roles, and postpone Session tables until you truly need database-backed sessions. This keeps the baseline light while giving you a clean place to hang roles, audit metadata, and OAuth links.

Schema first, with strict uniqueness. Use email as the canonical key and allow multiple Account rows per user for OAuth. Leave the Session out initially.

Enable the adapter behind an environment switch. Keep everything else unchanged; the session strategy remains jwt.

Linking strategy must prevent account takeover. Allow OAuth to attach to an existing user only when the provider returns a verified email that matches an existing ^User.email. Otherwise, block and ask the user to complete passwordless first.

Project roles without bloating tokens. When you eventually add User.role, resolve it server-side and avoid widening the JWT. Read it where you need it (server components, actions) instead of pushing it to every client.

Migration path stays low-risk. Add ^DATABASE_URL, run prisma migrate dev, set the env to enable the adapter, and deploy. Sessions continue to be JWT-based; existing sign-ins remain valid. When the product truly needs database sessions (revocation at row-level, device lists), introduce the Session model and flip session.strategy, until then, keep the adapter focused on profiles and account linking.

Integrating OAuth providers alongside passwordless without collisions

Coexistence keeps options open. NextAuth v5 can run Scalekit passwordless and GitHub (or any OAuth provider) side by side. The trick is separation of concerns: Let Scalekit handle email verification; let OAuth handle federation; let NextAuth own session issuance. Keep callbacks small and make linking rules explicit so accounts converge by verified email, not by guesswork.

Linking must prioritize verified email. Allow an OAuth account to attach only when the provider asserts a verified email that matches an existing user. Block unverified emails to prevent takeover, and prefer creating a user row only when no match exists. State from Scalekit’s send/verify flow stays out of NextAuth’s OAuth state to avoid accidental coupling.

OAuth linking hinges on a verified email; otherwise the sign-in is rejected.

Outcomes should be deterministic.

Redirects must be constrained. Use an allow-list in the redirect callback to block open redirects. Keep the magic-link landing path public and separate from OAuth callbacks so middleware remains simple. Do not pass Scalekit authRequestId or link tokens through OAuth query strings; those belong only to the passwordless verify path.

DX stays uniform across buttons. The login page can render “Continue with email,” “Continue with GitHub,” and others, yet downstream behavior stays identical: a minimal JWT with sub and email, the same middleware gates, and the same server-side ^auth() checks. This consistency lets UI and policy evolve without touching session semantics.

TypeScript definitions that remove casts and surface auth bugs early

Module augmentation clarifies session shape. NextAuth exposes ambient types that can be extended, so components rely on session.user.email without casts. Augmenting Session and JWT keeps identity consistent across server and client while preventing accidental widening of claims during refactors.

Input typing enforces the provider contract. A single, explicit type for verification inputs prevents “both code and link” bugs and keeps same-origin checks obvious. A small schema can enforce the XOR constraint at the boundary.

SDK wrapper types make I/O explicit. Thin wrappers around Scalekit methods document the API surface the app depends on and provide stable return types that tests can mock cleanly.

Route handlers benefit from typed payloads. Request and response bodies gain compile-time checks, and UI code receives exact fields the moment the contract changes.

Client helpers hide provider details behind types. Small client utilities keep signIn payloads correct and discoverable, which reduces form bugs and mismatched parameter names.

Environment typing prevents misconfig at deploy time. A tiny validator turns missing secrets into build-time failures rather than late-night 500s.

Type boundaries keep the story consistent. Augmented session types make UI code boring; XOR-typed verification inputs prevent ambiguous flows; wrapper return types simplify mocks; and validated envs stop drift. Together, these guardrails keep the minimal passwordless spine intact as deeper features, roles, adapters, or additional providers, arrive later.

Setting up middleware to protect routes without blocking verification

Middleware purpose: Middleware should gate everything by default, while explicitly allowing the small set of paths needed for sign-in, email sending, and magic-link landing. Keep the logic deterministic and cheap: read the session once with auth(), short-circuit public paths, and redirect unauthenticated users to /login with a safe next parameter.

Allow-list first, then deny by default. Public paths include static assets, NextAuth handlers, the passwordless send API, and the magic-link landing route. Everything else is protected. This shape prevents accidental blocks of verification while keeping the app tight during peak load.

Route policy at a glance

Redirect safety belongs here, too. Middleware should never forward arbitrary next destinations. Pair the login page with a tiny allow-list so post-login redirects can only land on known surfaces (^/dashboard, ^/settings, or the original in-app path from next), mirroring the redirect callback’s allow-list.

Operational notes that prevent surprises. Keep the magic-link route public even if you later add bot checks; verification still happens server-side in the provider. Avoid heavy logic in middleware, no DB calls, no remote fetches. If you add API routes that require auth, check ^auth() inside those handlers as well; middleware guards navigation, while handlers guard data paths explicitly.

Testing strategies that keep passwordless reliable under pressure

Testing goal: Prove that the four-stage pipeline, send, verify, issue, and guard, behaves deterministically for OTP and magic links. Focus on small, fast tests for logic and a couple of end-to-end runs to validate redirects, middleware, and session issuance during real navigation. Bake in Scalekit limits (max 2 sends/min; max 5 OTP attempts/10 min) to keep UX and copy honest.

Unit tests that pin the provider’s behavior to simple rules

Provider scope: Verify identity only; return ^{{ id, email };} never store artifacts.

JWT/session callbacks: Assert that email flows into ^token.email, ^token.sub, and ^{session.user.email}, and nothing else.

Integration tests that exercise send, resend, and verify paths

Route handlers: Hit ^{/api/auth/passwordless/send} and assert response shape and that magiclinkAuthUri is set. Simulate Scalekit rate-limit (HTTP 429) and ensure the route surfaces a friendly JSON error your UI can render as a countdown.

Attempt limits: For OTP verification, simulate four wrong codes and one correct code; on the sixth attempt, ensure your error mapping becomes “Too many attempts, start over.” This mirrors Scalekit’s five-attempts-in-ten-minutes cap.

End-to-end tests that validate navigation, guards, and redirects

Playwright scenarios: One OTP flow, one magic-link flow.

• OTP: Visit ^/login, request code, enter a test code, land on ^/dashboard, refresh, and confirm session persists.
  
• Magic link: Generate a fake link token in test, open ^{/passwordless/verify?link_token=…&email=…}, assert redirect to ^/dashboard, then hit a protected route to confirm middleware allows access.

Fixtures, fakes, and logging that make failures diagnosable

Before we add more tests, we make failures fast to reproduce and easy to explain. Two moves do the heavy lifting:

• Isolate Scalekit with a fake so unit/integration tests run in milliseconds and never flap on network/deliverability.
• Emit three audit events at stable points, ^email_sent, ^{verify_result}, ^{session_issued}, so when something breaks, logs tell you where and why without leaking secrets.

This combo cuts flakiness, pins the contract (send → verify → issue), and gives on-call a single breadcrumb trail across UI, API, and NextAuth.

What to implement (small, surgical pieces)

1) Swap the SDK with a deterministic fake in tests

Wire this via your thin wrapper so production uses the real client and tests import the fake.

2) Add structured, secret-free audit events

Call these in three places:

• After send route returns
• In the Credentials authorize (success/failure)
• After JWT callback issues ^{{ sub, email }}

This gives you a minimal “black box recorder” for auth without storing OTPs or link tokens.

Why the matrix exists

The following matrix is a review checklist and coverage map so every change touching send/verify/issue/guard has an explicit test; it keeps scope tight without sprawling E2E.

Use it during PRs: if your code touches a row, ensure there’s a test for that case.

Conclusion: Tying the minimal spine to real pain

We built a lean passwordless spine: a send endpoint that calls Scalekit, a custom NextAuth v5 Credentials provider that only verifies OTPs or magic links, JWT callbacks that project just ^sub and ^email, middleware that guards everything while keeping the verify route open, TypeScript augmentation that removes casts, optional Prisma and OAuth that plug in without touching the core, and tests that cover send/verify/issue/guard for both OTP and link paths.

The reset treadmill ends because identity is proven once by Scalekit and never embedded as secret state. Device quirks stop derailing sign-in because the verify page does one job and the provider runs server-side. Session sprawl disappears under a single JWT shape. Audits shorten because tokens carry minimal claims and responsibilities are explicit. Support volume drops as rate/attempt limits become predictable UX with clear copy and resend timers. Optional adapters and OAuth live behind flags, so complexity arrives only when the product needs it.

Where to go next (compact, high-leverage reading):

• Deepen authentication: WebAuthn/FIDO2 as step-up, device binding for risky actions, token versioning with a DB for selective revocation.
• Harden operations: Email deliverability (SPF/DKIM/DMARC), abuse throttling patterns, incident playbooks for secret rotation.
• Improve visibility: Auth audit events (“email_sent”, “verify_result”, “session_issued”), tracing around redirects, and e2e tests for expiry/429 paths.
• Docs to bookmark: Scalekit Passwordless guide (send/resend/verify, same-origin links), Scalekit Node SDK reference, NextAuth v5 callbacks and middleware, Prisma adapter notes on account linking.

Start by shipping the minimal spine in a feature flag: wire the send route, drop in the verify-only provider, keep the JWT tiny, and add two e2e tests, one OTP, one magic link. Once that’s stable in staging, gate Prisma and OAuth behind envs and turn them on per tenant. If you want a sanity check or example configs for your stack, say the word, and I’ll tailor the snippets to your repo layout.

FAQ

How do I enforce same-origin for Scalekit magic links in NextAuth.js v5?

Scalekit same-origin works by pairing ^link_token with the original ^{authRequestId}. Keep the verify route public and call the server-side SDK with ^{{ linkToken, authRequestId };} never persist these artifacts in JWT/session to maintain replay protection.

How should the UI handle Scalekit passwordless rate limits and OTP attempt caps?

Rate/attempt mirroring means: show a resend countdown from ^expiresIn, gray out send on HTTP 429, and reset the flow after 5 failed OTP attempts/10 minutes by requesting a new ^{authRequestId}.

What’s a practical NextAuth.js v5 JWT rotation and revocation strategy?

JWT containment uses short ^{session.maxAge} and periodic ^AUTH_SECRET rotation. For selective revocation, add a DB-checked ^tokenVersion (or a brief cache denylist) and keep claims minimal (^sub, ^email) to simplify audits.

How do I safely use Next.js Middleware (Edge Runtime) with NextAuth.js v5 sessions?

Edge hardening starts with an allow-list (static, NextAuth, send API, magic-link verify), then default-deny. Call ^auth() once, avoid DB/fetch in middleware, constrain next redirects to same-origin paths, and recheck auth inside protected API routes.

How do I prevent OAuth account takeover when linking providers in NextAuth.js v5?

Verified-email linking requires the provider to assert a verified email that matches your canonical User.email. Enforce unique keys ^{(User.email, Account(provider, providerAccountId))}, reject unverified emails, and never pass passwordless artifacts through OAuth callbacks.