Sandbox & Testing

FirstHandAPI provides a full sandbox environment for testing your integration without spending real money or requiring real workers.

Sandbox vs Production

Feature	Sandbox (`fh_test_`)	Production (`fh_live_`)
API key prefix	`fh_test_sk_...`	`fh_live_sk_...`
Credits charged	No	Yes
AI scoring	Synthetic (always 4 stars)	Real (Claude Vision + Whisper)
Workers	Simulated (auto-completes)	Real humans
Job completion time	~15 seconds	Minutes to days
Webhooks fire	Yes	Yes
File storage	S3 (same as production)	S3
Rate limits	Same as production	Same as production

Getting a Sandbox Key

Every organization gets both live and test keys. Create test keys from the dashboard:

Go to dashboard.firsthandapi.com/api-keys
Click Create Key
Select Test environment

Or use the signup endpoint — the returned key defaults to live. Create test keys via:

curl -X POST https://api.firsthandapi.com/v1/api_keys \
  -H "Authorization: Bearer fh_live_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: create-test-key-1" \
  -d '{"name": "Integration Tests", "environment": "test", "scopes": ["jobs:read", "jobs:write"]}'

How Sandbox Simulation Works

When you create a job with a fh_test_ key:

Job created normally — returns job_ ID, status open
After ~15 seconds — a simulated worker “accepts” the job, uploads a precanned file, and receives a 4-star AI score
Job status → completed — the job.completed webhook fires
Files available — GET /v1/jobs/:id/files returns download URLs for the simulated file

This means your full integration pipeline — job creation → webhook handling → file retrieval — works identically in sandbox and production. The only difference is speed and cost.

Testing Webhooks

Sandbox fires the same webhook events as production:

submission.approved — when the simulated submission is scored
job.completed — when all requested files are “collected”

Use a service like webhook.site or ngrok to receive webhooks during development:

# Create webhook endpoint pointing to your dev tunnel
curl -X POST https://api.firsthandapi.com/v1/webhook_endpoints \
  -H "Authorization: Bearer fh_test_..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: webhook-setup-1" \
  -d '{
    "url": "https://your-tunnel.ngrok.io/webhooks/firsthand",
    "events": ["submission.approved", "job.completed"]
  }'

Switching to Production

When you’re ready to go live, change your API key from fh_test_ to fh_live_:

// Development
const client = new FirstHandClient({
  apiKey: process.env.FIRSTHAND_TEST_KEY, // fh_test_...
});
 
// Production
const client = new FirstHandClient({
  apiKey: process.env.FIRSTHAND_LIVE_KEY, // fh_live_...
});

No code changes needed — the API behaves identically. The only differences are real charges and real workers.

Error Simulation Cookbook

Sandbox is the right place to exercise your error-handling code. Concrete recipes for each class of error:

`402 insufficient_credits`

Create a test org with an empty balance and attempt to post a job. The error body includes the current balance so your retry logic can prompt a top-up.

# New test key on a fresh org has 0 credits
curl -X POST https://api.firsthandapi.com/v1/jobs \
  -H "Authorization: Bearer fh_test_..." \
  -H "Idempotency-Key: test-insufficient-1" \
  -d '{"type":"data_collection","description":"...","files_needed":10,"accepted_formats":["image/jpeg"],"price_per_file_cents":100}'
# → 402 insufficient_credits { balance_cents: 0, needed_cents: 1000 }

`400 validation_error`

Send any field outside its bounds to trigger the validator. Examples:

# description too short (< 10 chars)
-d '{"type":"data_collection","description":"too short","files_needed":10,...}'
 
# files_needed out of range
-d '{"files_needed":10001,...}'
 
# price below minimum
-d '{"price_per_file_cents":5,...}'

The details array on the response will name the exact field that failed.

`400 pricing_violation`

Set price_per_file_cents above $500 or at a floor-violating amount (e.g., $0.05 per 30-second video):

-d '{"accepted_formats":["video/mp4"],"min_duration_seconds":30,"price_per_file_cents":5,...}'
# → 400 pricing_violation { floor_cents: 40, provided_cents: 5 }

`409 conflict`

Re-cancel an already-cancelled job:

curl -X POST https://api.firsthandapi.com/v1/jobs/{job_id}/cancel \
  -H "Authorization: Bearer fh_test_..." \
  -H "Idempotency-Key: cancel-already-cancelled"
# → 409 conflict "Job already cancelled"

`409 idempotency_error`

Reuse an Idempotency-Key with a different request body:

# First call
curl -X POST /v1/jobs -H "Idempotency-Key: dup-1" -d '{... price 50 ...}'
 
# Second call — same key, different body
curl -X POST /v1/jobs -H "Idempotency-Key: dup-1" -d '{... price 100 ...}'
# → 409 idempotency_error "Idempotency-Key already used with a different request"

`429 rate_limit_exceeded`

Loop GET /v1/me 200 times in under a minute. Response includes Retry-After and X-RateLimit-Remaining.

`403 free_tier_job_limit`

On a free-tier sandbox org, post two jobs concurrently:

# First job — OK
curl -X POST /v1/jobs ...
# Second job while the first is open — 403 free_tier_job_limit
curl -X POST /v1/jobs ...

Example Webhook Payloads

Copy-paste ready fixtures for each event type. Useful for unit-testing your handler without standing up a full sandbox flow.

`submission.scored`

{
  "id": "evt_01TEST_SCORED",
  "object": "event",
  "type": "submission.scored",
  "api_version": "2026-03-18",
  "created_at": "2026-03-15T10:05:00Z",
  "data": {
    "submission_id": "sub_01TEST",
    "job_id": "job_01TEST",
    "status": "approved",
    "ai_star_rating": 4,
    "dimensions": { "relevance": 5, "quality": 4, "instruction_compliance": 4 },
    "annotations": { "type": "image", "objects": [{"label":"mailbox","confidence":0.95}] }
  }
}

`job.completed`

{
  "id": "evt_01TEST_COMPLETED",
  "object": "event",
  "type": "job.completed",
  "api_version": "2026-03-18",
  "created_at": "2026-03-15T12:00:00Z",
  "data": {
    "job_id": "job_01TEST",
    "files_approved": 50,
    "files_needed": 50,
    "total_worker_payout_cents": 2000
  }
}

`credits.low_balance`

{
  "id": "evt_01TEST_LOWBAL",
  "object": "event",
  "type": "credits.low_balance",
  "api_version": "2026-03-18",
  "created_at": "2026-03-15T11:30:00Z",
  "data": {
    "balance_cents": 1850,
    "threshold_cents": 2000,
    "auto_top_up_enabled": false
  }
}

Full payload schema for every event: Webhook Handling → Event Types.

Testing Best Practices

Always test with sandbox first before using production keys
Test webhook signature verification — sandbox uses the same HMAC signing
Test error handling — run through the recipes above and assert your handlers behave correctly
Test pagination — create multiple sandbox jobs to verify cursor-based pagination
Test idempotency — retry the same request with the same key to verify duplicate prevention
Test long-poll — use Prefer: wait=30 on GET /v1/jobs/:id against a recently-created sandbox job to see the connection held and released on status change

Authentication Pagination