Limit or filter Ingestion as part of a data source(Jenkins Exporter)
S
Subhasis Ray
Currently, while integrating an external data source with the Jenkins Exporter, the entire dataset—including all builds and their details—is being retrieved and updated.
This new feature will introduce the ability to pre-filter the incoming data before it is sent to Port.
It is especially useful for data sources with a large number of events where not all historical data is relevant.
For example, users can choose to retrieve only the last two builds or fetch only Jenkins builds from the past 10 days.
Additionally, it will allow filtering out specific Jobs or Builds that do not need to be ingested at all.
D
Dennis Bilson
Hi Subhasis Ray 👋🏾
Support for fetching a specific number of builds per job has now been added to the Jenkins integration. This allows you to limit the number of builds retrieved, making data syncs more efficient.
However, filtering builds by timestamp (for example, retrieving only builds from the past 10 days) should be handled using Port’s query selector via a JQ expression as this is not support by the Jenkins API. This lets you flexibly filter data based on your own conditions after it’s fetched.