Local evaluation (server-side SDKs, beta)

By default every flag evaluation is a request to the API. Local evaluation lets a server-side process download the flag ruleset once, refresh it in the background, and answer flags in-process with no request per evaluation. It is opt-in, and it is available in the JavaScript SDK (@getexperimently/js-sdk 0.1.0, evaluation: 'local') and the Python SDK (experimently 0.1.0, evaluation="local"). Every other SDK evaluates on the server.

What stays on the server in every mode: experiments (assignment is sticky and recorded by the server, so getAssignment / get_assignment and getVariant / get_variant are always requests) and every flag the ruleset marks as server-evaluated (below).

Server-side code only

The ruleset contains every flag's targeting rules, including the values written in them (user ids, e-mail domains, plan names). The key that downloads it needs the sdk:ruleset scope; keep that key on a server. The JavaScript SDK refuses evaluation: 'local' in a browser.

The promise

A local answer is the answer GET /api/v1/feature-flags/evaluate/{key} would give for the ruleset version the process holds: the same enabled and the same reason (targeting_rule, rollout or inactive). When the SDK cannot be sure of that, it does not guess: it makes the same request it makes in server mode, and the result carries source server.

That promise is checked against tests/sdk-contract/ruleset-vectors.json, a corpus of 2191 cases generated by running the server's own evaluator. Each SDK must answer the 1294 cases marked must_local locally with exactly the server's answer, and send every other case to the server.

Requirements

An Experimently server that serves GET /api/v1/sdk/ruleset and accepts POST /api/v1/tracking/evaluations (both are beta). A server without the ruleset answers 404, and the SDK then evaluates on the server (see Freshness).
An API key with the sdk:ruleset scope, created by a user who can change feature flags (ADMIN or DEVELOPER). The scope works only while its owner can change flags: it is checked on every request.
Node 18 or later (the JavaScript SDK), or Python 3.9 or later (the Python SDK).

JavaScript

import { ExperimentationClient } from '@getexperimently/js-sdk';

const client = new ExperimentationClient({
  apiUrl: process.env.EXPERIMENTLY_API_URL!,
  apiKey: process.env.EXPERIMENTLY_SERVER_KEY!, // a key with the sdk:ruleset scope
  evaluation: 'local',
  refreshIntervalMs: 30_000,                    // default 30 s, minimum 5 s, ±10% jitter
});

const ready = await client.ready({ timeoutMs: 5_000 }); // never rejects
if (!ready.ok) console.warn('flags are evaluated by the server for now:', ready.error.message);

const { enabled, reason, source } = await client.evaluateFlag('new_search', {
  userId: 'user-123',
  attributes: { country: 'US', plan: 'pro' },
});

// On shutdown: stop refreshing and send the evaluation counts not yet sent.
await client.close();

Option / method	Meaning
`evaluation: 'server' \\| 'local'`	`'server'` (default) is unchanged behaviour.
`refreshIntervalMs`	How often the ruleset is refreshed: default 30 000, values below 5 000 are raised to 5 000, jittered by ±10%. Waits after a failure (below) are exact, not jittered.
`maxStaleMs`	Optional. When the ruleset has not been refreshed successfully for this long, flags are evaluated by the server until a refresh succeeds. Default: no limit.
`ready({ timeoutMs? })`	Resolves `{ ok: true, rulesetVersion }` once flags can be answered locally, or `{ ok: false, error }` when the first fetch failed or `timeoutMs` passed. Never rejects. Evaluations never wait for it.
`status()`	`{ evaluation, ready, rulesetVersion, lastRefreshAt, lastError, serverEvaluatedFlags }`.
`close()`	Stops the refresh, sends the remaining evaluation counts, and from then on evaluates on the server.
`source` on `FlagEvaluation`	`'local'` or `'server'` (set only in local mode).
`onError(err, 'refresh' \\| 'flush' \\| 'evaluate')`	A failed ruleset refresh, a failed evaluation-count report, or an unexpected failure inside local evaluation (that call then goes to the server).

evaluateFlag, isFeatureEnabled and getAllFlags keep their signatures. A local answer uses the attributes passed to that call, so the "call clearCache() after changing attributes" caveat does not apply to it; it is still recorded for the key-less track() fan-out. getAllFlags is answered locally only when every active flag can be answered for those attributes; otherwise the whole call goes to the server. The refresh and report timers do not keep a Node process alive.

Python

import os

from experimentation import ExperimentationClient

client = ExperimentationClient(
    os.environ["EXPERIMENTLY_API_URL"],
    os.environ["EXPERIMENTLY_SERVER_KEY"],  # a key with the sdk:ruleset scope
    evaluation="local",
    refresh_interval_seconds=30,            # default 30 s, minimum 5 s, ±10% jitter
    on_error=lambda error, operation: print(operation, error),
)

ready = client.ready(timeout_seconds=5)     # never raises; truthy when ready
if not ready:
    print("flags are evaluated by the server for now:", ready.error)

flag = client.get_feature_flag("new_search", "user-123", {"country": "US", "plan": "pro"})
print(flag.enabled, flag.reason, flag.source)

# On shutdown: stop refreshing and send the evaluation counts not yet sent.
client.close()                              # or: with ExperimentationClient(...) as client:

Argument / method	Meaning
`evaluation="server" \\| "local"`	`"server"` (default) is unchanged behaviour.
`refresh_interval_seconds`	How often the ruleset is refreshed: default 30, values below 5 are raised to 5, jittered by ±10%. Waits after a failure are exact.
`max_stale_seconds`	Optional. When the ruleset has not been refreshed successfully for this long, flags are evaluated by the server until a refresh succeeds. Default: no limit.
`on_error(error, operation)`	`"refresh"`, `"flush"` or `"evaluate"`, as in JavaScript. A handler that raises is logged and ignored.
`ready(timeout_seconds=None)`	`ReadyResult(ok, ruleset_version, error)`, truthy when flags can be answered locally. Waits for the first fetch (at most `timeout_seconds`). Never raises.
`status()`	`LocalEvaluationStatus(evaluation, ready, ruleset_version, last_refresh_at, last_error, server_evaluated_flags)`.
`close()`, `with ... as client:`	Stops the refresh thread, sends the remaining evaluation counts, and from then on evaluates on the server.
`source` on `FlagEvaluation`	`"local"` or `"server"` (set only in local mode).

get_feature_flag, is_feature_enabled and get_all_flags keep their signatures, with the same attribute, fan-out and get_all_flags rules as JavaScript. The attributes are evaluated as the server would receive them: the same json.dumps the request would send, read back (tuples become lists, non-string keys become strings, datetime values become ISO strings).

Threads and processes. One daemon thread per client refreshes the ruleset and sends the counts; the client is safe to share between threads. A client created before os.fork() (gunicorn with --preload, multiprocessing with the fork start method) keeps working in the child: the child gets fresh locks, starts its own refresh thread, and reports only the evaluations it made itself; the parent's counts stay with the parent. Create one client per process and share it.

What is evaluated locally

The ruleset marks each active flag "evaluation": "local" or "remote". A flag is local when every condition in its rules uses one of these operators, with a value the SDK can compare exactly as the server does:

Operator (dashboard name)	Answered locally when
`eq`, `neq` (equals, not equals)	the attribute and the rule value are both strings, or both numbers
`in`, `not_in` (in list, not in list)	the attribute is a string and the list holds strings, or both are numbers
`contains`, `not_contains`, `starts_with`, `ends_with`	the attribute and the rule value are both strings
`gt`, `gte`, `lt`, `lte`	the attribute and the rule value are both numbers
`is_null`, `is_not_null`	always, except on a string containing a whitespace character (the server trims strings; see below)

Anything else is evaluated by the server, for the flag or for that one evaluation:

flags using any other operator (semantic version, contains_all / contains_any, date and time windows, geo distance, array length, JSON path, percentage bucket, and pattern matching, which is server-only because Python and JavaScript regular expressions are different dialects), and flags stored in the legacy list shape: status() lists them as server-evaluated flags;
flags whose rules use a segment (in_segment, not_in_segment). Membership lives on the server (an ID list can hold a million IDs), so the ruleset marks these flags "remote" and carries neither their rules nor the segment, status() lists them as server-evaluated flags, and each evaluation is a request, counted against the SDK rate limit;
an attribute whose type does not match the rule (a string against a number, a boolean, a numeric string such as "10"); the server's type coercions are not reproduced;
a number larger than 2^53 − 1 in magnitude, and a non-finite one;
a string, attribute name or user id containing an unpaired UTF-16 surrogate;
is_null / is_not_null on a string that contains any whitespace character, even an ordinary space;
a flag key the ruleset does not have yet (a flag created since the last refresh).

Rollout percentages bucket users exactly as the server does (md5-mod100-v1: the whole MD5 digest of "{userId}:{flagKey}" read as a big-endian integer, mod 100). This is not the consistentHash / consistent_hash utility the SDKs export.

Freshness

A change made in the dashboard (turning a flag off, a rollout step, a safety rollback) reaches a process evaluating locally at its next successful refresh: within the refresh interval (30 seconds by default) while the API is reachable.

While the API is unreachable (errors, timeouts), the SDK keeps serving the last ruleset it received, so a flag turned off during an outage stays on in that process until the API is reachable again. Set maxStaleMs / max_stale_seconds if you prefer evaluations to go to the server once the ruleset is that old. A process that starts while the API is unreachable has no ruleset: its evaluations go to the server, as in server mode.

Refresh answer	No ruleset yet	Ruleset loaded
200, or 304 (unchanged)	load it	replace it, or keep it
5xx, network error, timeout, or a 200 that is not a valid ruleset	evaluate on the server; retry with backoff up to 5 minutes	keep serving it; report `refresh`; retry with backoff
429	wait `Retry-After` (never less, and never less than the refresh interval)	keep serving it; the same wait
401 / 403 (key revoked, scope removed, owner no longer allowed)	evaluate on the server	discard it: every flag is evaluated by the server
404 (a server without the ruleset), or a ruleset format this SDK does not know	evaluate on the server	discard it

401, 403, 404 and an unknown format are reported once and retried every 10 minutes, so a scope granted later is picked up without a restart. The backoff doubles from the refresh interval up to exactly 5 minutes. Only the regular refresh interval is jittered: backoff steps, Retry-After and the 10-minute retry are not.

Safety monitoring and the dashboard

The server cannot see local evaluations, so the SDK reports how many it made: per flag, every 60 seconds and on close(), to POST /api/v1/tracking/evaluations. Safety monitoring divides the errors your server reports through POST /api/v1/tracking/errors with a server-side key (the same sdk:ruleset key), and the evaluations that fail on the server, by these counts, so automatic rollback keeps working for flags evaluated locally. Reports sent with a key without the scope are stored but not counted (see Which errors count). Evaluations sent to the server are not counted twice.

Two dashboard figures change as local evaluation is adopted, because a count carries no user and no rule:

the rule-match rate of a flag falls: locally answered evaluations are counted, but not attributed to a rule;
unique users are under-counted: only evaluations made on the server carry a user id.

The server side, step by step

What the SDKs do, with curl, against a running stack. Run these in one terminal, in order; each step uses the shell variables set by the ones before it.

Log in as the demo administrator:

TOKEN=$(curl -s -X POST localhost:8000/api/v1/auth/login \
  -H 'content-type: application/json' \
  -d '{"email":"admin@demo.com","password":"Demo1234!"}' | jq -r .access_token)

curl -s localhost:8000/api/v1/auth/me -H "Authorization: Bearer $TOKEN" | jq .role

It prints "ADMIN".

Create a flag, turned on ("is_active": true; a new flag is off otherwise) and serving no one except users in the US or Canada:

curl -s -X POST localhost:8000/api/v1/feature-flags/ \
  -H "Authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{
    "key": "local-eval-demo",
    "name": "Local evaluation demo",
    "is_active": true,
    "rollout_percentage": 0,
    "targeting_rules": {
      "logical_operator": "AND",
      "groups": [
        {
          "logical_operator": "AND",
          "conditions": [
            {"attribute": "user.country", "operator": "in", "value": ["US", "CA"]}
          ]
        }
      ]
    }
  }' | jq -c '{key, status}'

It prints {"key":"local-eval-demo","status":"active"}.

Create a server-side key with the sdk:ruleset scope, and a plain key without it. The plaintext key is in the create response only:

SERVER_KEY=$(curl -s -X POST localhost:8000/api/v1/api-keys \
  -H "Authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{"name": "local-eval-demo-server", "scopes": ["sdk:ruleset"]}' | jq -r .key)
PLAIN_KEY=$(curl -s -X POST localhost:8000/api/v1/api-keys \
  -H "Authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{"name": "local-eval-demo-plain"}' | jq -r .key)

curl -s -o /dev/null -w '%{http_code}\n' localhost:8000/api/v1/sdk/ruleset -H "X-API-Key: $PLAIN_KEY"
curl -s -o /dev/null -w '%{http_code}\n' localhost:8000/api/v1/sdk/ruleset -H "X-API-Key: $SERVER_KEY"

It prints 403 for the plain key, then 200 for the server-side key.

The flag's entry in the ruleset: evaluated locally, with its conditions in the engine's operator names (the dashboard's groups become nested groups):

curl -s localhost:8000/api/v1/sdk/ruleset -H "X-API-Key: $SERVER_KEY" \
  | jq -c '.flags[] | select(.key == "local-eval-demo") | {evaluation, conditions: [.. | objects | select(has("operator"))]}'

It prints {"evaluation":"local","conditions":[{"attribute":"user.country","operator":"in","value":["US","CA"]}]}.

A refresh sends the ETag back, and an unchanged ruleset answers 304 with no body:

ETAG=$(curl -s -D - -o /dev/null localhost:8000/api/v1/sdk/ruleset -H "X-API-Key: $SERVER_KEY" \
  | tr -d '\r' | awk 'tolower($1) == "etag:" {print $2}')
curl -s -o /dev/null -w '%{http_code}\n' localhost:8000/api/v1/sdk/ruleset \
  -H "X-API-Key: $SERVER_KEY" -H "If-None-Match: $ETAG"

It prints 304.

The server's own answer for a US user, which an SDK evaluating locally gives without a request:

curl -s "localhost:8000/api/v1/feature-flags/evaluate/local-eval-demo?user_id=user-1&context=%7B%22country%22%3A%22US%22%7D" \
  -H "X-API-Key: $SERVER_KEY" | jq -c '{enabled, reason}'

It prints {"enabled":true,"reason":"targeting_rule"}: country in the context answers the rule on user.country.

Troubleshooting

What you see	Cause	Fix
Constructor throws "evaluation: 'local' downloads every flag's targeting rules, …"	`evaluation: 'local'` in a browser (JavaScript)	Leave the browser client in server mode.
`refresh` error: "The API key was refused (403) fetching the flag ruleset: This API key does not have the 'sdk:ruleset' scope …"	The key lacks the scope	Create a key with the `sdk:ruleset` scope.
The same, with "This key's owner can no longer change feature flags or experiments …"	The key's owner is ANALYST or VIEWER	Use a key created by an ADMIN or DEVELOPER.
"The server has no /api/v1/sdk/ruleset (404). Local evaluation needs Experimently 0.11.0 or later …"	The server predates the ruleset	Upgrade the server, or leave evaluation at `server`.
"… a format this SDK does not understand …"	The server is newer than the SDK	Upgrade the SDK.
`status()` lists server-evaluated flags	Those flags use rules that are evaluated on the server	Nothing to fix; they keep working with a request each.
`evaluate` error: "Local evaluation failed; asking the server instead: …"	A defect in the SDK's evaluator	The call was answered by the server; please report it.