Integrating ClickHouse with Google Calendar

The Google Calendar registry item copies a canonical event reader with journaled page and sync-token progress into a chkit project.

Install

bunx chkit add google-calendar --with-tests
bunx chkit check
bunx chkit generate --name add_google_calendar
bunx chkit migrate --apply
bunx chkit ingest run --tag provider:google-calendar
bunx chkit ingest status --tag provider:google-calendar

Set GOOGLE_CALENDAR_ACCESS_TOKEN with calendar.readonly access before ingestion. Review sourceId, calendarId, and database in src/integrations/google-calendar/index.ts. primary is resolved through calendar metadata to detect a different account reusing the checkpoint. Schema imports do not request credentials or call Google.

Resource	Default ClickHouse table	Records synced	API reference
Events (`events`)	`google_calendar_events_raw`	Canonical events, recurring masters, exceptions, and cancellation payloads from the configured calendar	`GET/calendars/{calendarId}`, `GET/calendars/{calendarId}/events`

Sync behavior

The first run reads canonical events with recurring masters and exceptions, using singleEvents=false and showDeleted=true. Later runs request changes through Google’s syncToken. Cancellation payloads remain unmodified. Recurring occurrences require a separate bounded instances reader or application projection.

Each acknowledged page stores its next-page position and retains the same input sync token. Only a loaded terminal page promotes nextSyncToken, including an empty page. Rerun after interruption to resume; failed loads leave the previous checkpoint. Requests use fixed parameters, and validation rejects changed source scope or malformed pagination. Native JSON requires ClickHouse 25.3 or later.

Google forbids timeMin, timeMax, updatedMin, and orderBy on sync-token requests. This canonical reader rejects explicit --from/--to bounds. A named backfill without bounds runs an independent canonical sync. See Google synchronization and events.list parameters.

Expiry and stored data

410 Gone resets provider progress and starts a new baseline without deleting raw rows. A rejected page token replays its collection once. The destination keeps the latest observed payload per [sourceId, resolvedCalendarId, event.id]; absent records are not reconciled after a reset. A current-calendar view requires completed-baseline generations or explicit reconciliation before treating absence as removal.

The checkpoint retains the recovery count until the terminal sync page is acknowledged. Pauses and successful nonterminal replay pages keep that count; a second token rejection for the unfinished sync fails across executions. Adjust the editable stream budget.maxChunks, execution duration, or polling frequency so the sync finishes while tokens remain valid. After reviewing coverage and correcting those conditions, migrate saved state explicitly or use a new stream identity for a fresh sync.

Version 0.2.0 changes the former rolling occurrence-window semantics and row identities. Keep old tables as archives and start a new canonical dataset, or migrate identities explicitly; old expanded occurrences are not removed automatically. The installed README describes upgrade and recovery details.

Fixture verification

bun test src/integrations/google-calendar/tests/basic.test.ts

Fixtures cover page resumption, empty deltas, sink failures before token promotion, cancellation payloads, expired tokens, and calendar-account changes. Schedule repeat runs externally with one ingestion process per destination at a time.

Changelog

Version 0.2.0

Replace the rolling occurrence window with canonical events and incremental provider sync tokens, including cancellation payloads.
Resume acknowledged pages, promote the terminal sync token after destination acknowledgement, and bound recovery from rejected or expired tokens.
Scope row IDs to the installation and resolved calendar; migrate legacy occurrence identities or use a fresh table and stream when upgrading.

Version 0.1.0

Introduce raw events ingestion over a rolling occurrence window.