Keep your data in sync

Mirror projects and listings in your own system with no gaps and no doubles, by reading the events feed from a cursor, letting webhooks say when, and retrying writes with Idempotency-Key.

What you'll build

A sync job that keeps your system's copy of each project and listing current. It reads every change in order from the events feed, so it misses nothing, and applies each event once. Webhooks tell it when to run, and writes it repeats after a timeout run once.

Before you start

  • Your own ids on the projects. Set external_id to your CRM's id when you create a project, as Onboard a client's product does.
  • A place to store 1 cursor per mode, and the ids of the events you applied.

Store our ids, prj_ and lst_, next to your records: they never change, and a listing keeps its id after a correction. To find a project from your side, ask by external_id.

curl "https://api.submitator.com/v1/projects?external_id=crm_8812" \
  -H "Authorization: Bearer $SUBMITATOR_API_KEY"

external_id matches the whole value, case-sensitive, and is unique in each mode. metadata holds up to 20 more keys of your own and comes back on every read.

2. Read the feed from your cursor

GET /events returns every change oldest first, and next_cursor is never null. Apply a page, store its next_cursor, and ask again with it until has_more is false; next time, start from the cursor you stored.

curl "https://api.submitator.com/v1/events?cursor=cur_Ev7Kq2Wx9Lt3Nb5R&limit=100" \
  -H "Authorization: Bearer $SUBMITATOR_API_KEY"
Response 200 (one event shown)
{
  "object": "list",
  "data": [
    {
      "object": "event",
      "id": "evt_5Rt8Wq2Lm7Xn3Kb9Vz1PdJ",
      "type": "listing.live",
      "created_at": "2026-10-08T14:17:01Z",
      "livemode": false,
      "project": "prj_4QzX1m9Lr2Vb7Nc8Tk3HwP",
      "data": {
        "object": { "object": "listing", "id": "lst_7Hk2Qn4Rt9Vx1Bm6Cz3LdK", "status": "live" },
        "previous_status": "submitted"
      }
    }
  ],
  "has_more": false,
  "next_cursor": "cur_Ev7Kq2Wx9Lt3Nb5R"
}

The feed has no gaps: a cursor returns every event recorded after it, in order. Without a cursor it starts at the oldest event it keeps, 90 days back.

3. Apply each event

data.object is the whole object right after the change, so write it over your copy by its id. Skip an event whose id you have applied: after a crash between applying a page and storing its cursor, that page comes again.

async function apply(event) {
  if (await alreadyApplied(event.id)) return;
  const object = event.data.object;
  if (object.object === "listing") await saveListing(object.id, object.status, object.live_url);
  if (object.object === "project") await saveProject(object.id, object.status, object.progress);
  await markApplied(event.id);
}

Statuses can move back. A live listing can become not_accepted or submitted again, and a not_accepted one can return: How it works lists the corrections, and each comes as its own event.

4. Let webhooks say when

A webhook tells you about a change within about a minute of it. The simplest way to use one: answer 2xx and run the feed reader from step 2. The cursor then keeps the order and fills any gap, and a delivery that comes twice changes nothing.

To act on a delivery's body directly, compare its data.previous_status with what you stored, since deliveries can arrive out of order. Receive webhooks builds the endpoint.

5. Retry a write safely

A POST that timed out may have run. Send each one with an Idempotency-Key, and send the same key when you retry: a repeat with the same key and body within 24 hours returns the first response, with Idempotent-Replayed: true, and changes nothing twice.

curl https://api.submitator.com/v1/projects \
  -H "Authorization: Bearer $SUBMITATOR_API_KEY" \
  -H "Idempotency-Key: crm_8812-create" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://ledgerly.test", "contact_email": "founder@ledgerly.test", "external_id": "crm_8812"}'

The examples retry a 409 too, for idempotency_key_in_use; check error.code in your own code, since project_exists is a 409 as well. On a 429, wait the seconds in Retry-After instead of the backoff.

6. Recover after a long gap

The feed keeps 90 days. A cursor older than that returns 410 cursor_expired: read the projects and listings you track again, then read the feed without a cursor and store the new one.

curl "https://api.submitator.com/v1/projects/prj_4QzX1m9Lr2Vb7Nc8Tk3HwP/listings?limit=100" \
  -H "Authorization: Bearer $SUBMITATOR_API_KEY"

Check it worked

  • For each project, your copy's counts by status equal progress on GET /projects/prj_4QzX1m9Lr2Vb7Nc8Tk3HwP.
  • Your stored cursor moves forward after each run, and a run with nothing new reads 1 page with has_more false.
  • Running the job twice in a row changes nothing the second time.

What can go wrong

CodeStatusFix
invalid_cursor400The cursor came from another list or was changed. Send next_cursor exactly as you received it.
cursor_expired410The cursor is older than 90 days. Read your objects again, then read the feed without a cursor.
idempotency_key_in_use409The first request with this key still runs. Retry in 1 second with the same key.
idempotency_key_reused422The key came with another body before. Use a new key for each new request.

Next steps