Mapping - Drop key
T
Travis Gosselin
Current scenario: Integrations are calculating the payload based on the mapping YAML, and they send all properties that are being mapped.
Suggestion: Add the ability to drop the key in the mapping configuration. This way, the integration will not override existing keys in Port, while still having the ability to map data dynamically.
Example:
protected:
if (.branch.protected != null) then .branch.protected else {DROP_KEY} end
If the object contains the branch data, then map it to the entity, and if it does not a custom operation will drop the key from the payload, not overriding the current data.
M
Mark Tarry
Concrete use case, from a our
github-ocean
integration. Related support request: 11088.We ingest GitHub Actions workflow runs. Our blueprint's
conclusion
property is optional because a run still in flight genuinely has no conclusion — GitHub reports conclusion: null
until it finishes. So null
is the modelled-correct value, the mapping returns it, and every resync logs:Mapping error for kind workflow-run: conclusion, github_conclusion:
jq misconfiguration detected ... in fields: properties.conclusion
Two things make this frustrating:
- It isn't a failure by Port's own accounting. The kind finishes syncedwithfetched 3705,transformed 3705,transform.failed 0. The rows only degrade the error log, which is where we look for real faults.
- Port's own GitLab v2 mapping docs recommend // nullfallbacks for optional fields. Following the documented advice produces ERROR output.
We exhausted the alternatives. jq
empty
is coerced back to a present key holding null
(verified via the mapping-test endpoint). // ""
is rejected at load on an enum property. Coercing to a catch-all value would assert a terminal outcome that never happened.That leaves the write-once split: a second resource block with an inverted selector. It works, but Port runs a separate extraction pass per resource block, so it costs a second independent fetch of the same ~3,700 runs every four hours, and a duplicated sweep of GitHub rate-limit budget. Duplicating a whole extraction pass to control the shape of one property is the wrong trade, and we'd expect it to cost Port compute too.
So we shipped a synthetic
Pending
enum member instead. It silences the log, but puts a value in the data model that GitHub never reports. Flagged for removal if this lands.What would work for us:
- Opt-in per property, not a global mode.
- Triggered by the jq yielding nullorempty, dropping the key from the payload.
- A dropped key must leave any existing value untouched under enableMergeEntity— that is what makes it a true replacement for the write-once split.
- Must not count as a transform failure or emit ERROR output.
T
Travis Gosselin
Agreed, this would be helpful. Today we are creating different mappings with the same kinds that have different selectors and update only individual fields to workaround this. It would streamline mapping capability and make it much more convenient and readable. In certain exporters like GitHub Exporter, usage of inline expressions to revert to entity data has an impact on how the exporter deletes entities where this feature would be very useful (working with Omri on that).
Matan Grady
Travis Gosselin: Thank you. I agree this is indeed a possible workaround; however, it complicates the mapping and affects the ability to debug it properly.