Unified Data Connector Configuration
The unified data connector setup process is multi-step and involves close coordination with BigPanda support. Contact BigPanda support when you are ready to set up a data connector.
Streaming Modes
You can set up both historical and incremental streaming modes for each data connector. Most organizations will need to use both modes.
Historical mode
Historical modes sync data one time from a selected start date and after. This works well for one-time data loads or when you need to backfill data that was previously missed. Historical mode is the default, recommended method of initially syncing data.
Historical mode
Historical streams can take minutes or days, depending on the amount of data sent.
Re-pulling historical data
If you need to re-ingest a specific historical window, such as after a ServiceNow data correction, a major migration, or a gap caused by an upstream outage, your BigPanda account team can trigger a targeted date-range backfill against an existing pipeline without rebuilding the connector.
Backfills run in chunks (typically per day) and execute in parallel with your incremental sync without disrupting it. Large ranges can be scheduled to run only during off-peak windows in your time zone.
To request a date-range backfill, reach out to your account team with:
The connector and pipeline name
The exact start and end date of the window to re-pull
Any preferred execution window (for example, midnight–6 AM EST)
While the backfill is running, you can view per-table row counts and chunk completion status in the BigPanda support response or in the BigPanda monitoring views shared by your account team
Incremental mode
Incremental modes run periodically and sync all new or updated records. This mode works well for scheduled, recurring data loads. Depending on what functionality you are using within BigPanda, your incremental sync time period may vary.
No historical data
Incremental mode runs periodically and syncs only new or updated records since the previous successful run. Once an incremental pipeline is started, its cursor moves forward from that point and does not look back to the configured Start Date. To ingest historical records, run a one-time historical sync (or request a date-range backfill) on the same source. Most production deployments use both: one historical sync to seed the dataset, then incremental mode for ongoing updates.