Azure Data Factory - Rest API call to HubSpot- Copy Activity - Response provide URL but missing hapikey, How to append hapikey & paginate to get all records
Hello,
I know similar questions are asked but somehow I have tried many of that options and did not succeed. Need help with a clear step-by-step guide on how to resolve this.
I am using the ADF Copy activity to fetch deal data from HubSpot. Following are the screenshots of my current setup.
Linked Service:
Authentication Type: Anonymouse
Base URL: https://api.hubapi.com/crm/v3/objects/deals?limit=100&properties=dealname%2Cdealtype%2Cpipeline%2Cregion__c %2Ccreatedate&hapikey=<<key>>
Dataset:
Base URL: https://api.hubapi.com/crm/v3/objects/deals?limit=100&properties=dealname%2Cdealtype%2Cpipeline%2Cregion__c %2Ccreatedate&hapikey=<<key>>
Relative URL:
Copy Activity: Source Setting are as follows
HubSpot API is returning the next page URL but it does not have hapikey.
Using the above method how do I append the hapikey and iterate to read all the deals data ?
HubSpot API does not return a count of pages that need iteration so can not use For Loop.
Many have suggested in past to use OAuth2 client credential instead of hapikey but not sure how to setup it up as not sure what will be the token endpoint?
Any suggestions on how to append hapikey to the next page URL and make copy activity iterate will help.