Summary

Port should add first-class support for navigation-style cursor pagination, where APIs return the next page cursor via the HTTP Link header (rel="next") instead of embedding cursor metadata in the JSON body.

This is required for full compatibility with APIs like GitGuardian, which implement cursor pagination through Link headers in the same style as GitHub.

Background / Motivation

Many modern REST APIs paginate large datasets using the Web Linking standard:

Responses return a Link header containing one or more navigation URLs

Clients retrieve the next page by following the rel="next" URL directly

GitGuardian documents this explicitly in their pagination guide:

Cursor-based pagination is used across their API

Pagination is driven through links exposed in the response headers

GitGuardian Pagination Docs:

Real-World Example: GitGuardian API Pagination

GitGuardian provides a reference Python implementation:

while True:

response = requests.get(endpoint_url)

if "next" not in response.links:

break

endpoint_url = response.links["next"]["url"]

This shows that pagination works by repeatedly following:

response.links["next"]["url"]

rather than manually constructing cursor= params.

Related Industry Standard: GitHub REST API

GitHub uses the exact same navigation approach:

Paginated responses include:

Link: <...page=2>; rel="next",

<...page=last>; rel="last"

GitHub docs state:

"In all cases, you can use the URLs in the link header to fetch additional pages of results."

Problem Statement

Port’s current pagination support is primarily focused on:

explicit cursor parameters (cursor=...)

body-based cursor tokens (next_cursor, starting_after, etc.)

However, APIs like GitGuardian require clients to:

read the Link header

follow rel="next" URLs directly

Without native support, integrations must implement custom pagination logic, leading to:

duplication across connectors

incomplete ingestion support

difficulty integrating GitGuardian and GitHub-class APIs

Proposed Feature

Add “Nav Style Link Header Pagination” Strategy

Port should support a pagination mode where:

Initial request is made normally

Port checks the Link response header

If rel="next" exists:

Port automatically requests that URL

Continue until rel="next" disappears

This works even when the cursor is opaque or parameter names vary.

Configuration Proposal (Example)

pagination:

strategy: link_header

header: Link

rel: next

Or more simply:

pagination:

type: nav

Functional Requirements

Parse RFC-compliant Link headers (rel="next")

Support GitGuardian-style response.links["next"]

Treat next-page URLs as opaque (no assumptions about param naming)

Work across APIs using:

page=2

cursor=...

after=...

mixed/hidden pagination tokens

Coexist with existing cursor/body pagination support

Benefits

Immediate compatibility with GitGuardian APIs

Unlocks GitHub-style REST ingestion in Ocean Framework

Reduces connector-specific pagination code

Aligns Port with widely adopted HTTP navigation conventions

Improves robustness when cursor parameters differ across endpoints

Success Criteria

Port can paginate GitGuardian endpoints without custom integration code

Connectors can declaratively enable nav/link_header pagination

End-to-end ingestion works across all rel="next"-based APIs