Integrating ClickHouse with GitHub

The GitHub registry item copies a small API reader and raw ClickHouse tables into a chkit project.

Install

bunx chkit add github
bunx chkit check
bunx chkit generate --name add_github
bunx chkit migrate --apply
bunx chkit ingest run --tag provider:github

Set GITHUB_TOKEN in the runtime environment before the ingestion run. Edit repositories in src/integrations/github/index.ts to select repositories, then remove either stream if it is not needed.

Resource	Default ClickHouse table	Records synced	API reference
Issues and pull requests (`issues`)	`github_issues_raw`	Issues and pull requests updated inside a timestamp window, including paginated comments and pull request context.	`GET/repos/{owner}/{repo}/issues`
Stargazers (`stargazers`)	`github_stargazers_raw`	Repository stargazers with starred_at when the token permits it	`GET/repos/{owner}/{repo}/stargazers`

Sync behavior

Issues use since with update-time ordering, a fixed local upper cutoff, and five minutes of overlap. GitHub has no server-side upper timestamp filter. The checkpoint advances only after the complete issue window and destination writes succeed; failed windows replay from their lower bound. The reader follows provider Link URLs and includes paginated issue comments plus pull request detail, reviews, review comments, commits and files. Provider caps are reflected in raw _chkit_context completeness fields.

Stargazers remain full reads. Their checkpoint records the last completed scan, including an empty scan; page numbers and star creation timestamps are not removal cursors. Removed stars and deleted issues remain stored. Live pagination and child-only updates require periodic reconciliation or a durable webhook companion. GitHub issue filters and pagination describe the API behavior.

bunx chkit ingest run --tag provider:github --tag resource:issues --backfill reconcile-2026-10-04 --from 2008-01-01
bunx chkit ingest status --tag provider:github --json

Historical replay reads current observations, not historical versions. Use a new backfill ID for each full reconciliation. The raw tables preserve provider fields and require ClickHouse 25.3 or later. See the installed README for token permissions and registry installation.

Test the reader

bunx chkit add github --with-tests
bun test src/integrations/github/tests/basic.test.ts

Changelog

Version 0.2.0

Sync issues through overlapping updated-time windows, advancing the watermark only after complete destination acknowledgement.
Collect all accessible issue comments and pull-request context, reporting provider caps on commits and files.
Record successful stargazer full scans and add failure-replay and empty-window fixtures without changing raw destinations.

Version 0.1.0

Introduce raw repository issues and stargazers ingestion.