We have a requirement to export Oracle cloud fusion data to azure datalake at a monthly cadence. We are able to use rest connector, but getting error with Oracle service cloud connector

Hi Dataarch_MM,

Thanks for reaching out to Microsoft Q&A.

This could be related to a DNS or configuration issue, as suggested by the "Service Unavailable DNS failure" message. This connector is still in preview which might explain some limitations or a bug, here are a few steps you could try:

1. DNS and Networking:

• Double check the hostname and network connectivity from the Azure IR. Ensure that the IR can resolve the Oracle Cloud Fusion endpoint correctly. You might want to test this with a simple curl or ping command if possible.
• Consider using a SHIR instead of Azure IR to see if networking is more reliable, especially for services that might have firewall restrictions or DNS resolution issues.

2. Service Cloud Connector Configuration:

• Since you're able to connect via the REST connector, confirm if the endpoint formats differ between the REST API and the Oracle Service Cloud connector. Check if any extra parameters, like authentication tokens or certificates, are required for the Oracle Service Cloud Connector.
• If you're working with Oracle Fusion, validate that the endpoint you're using is specifically for Oracle Fusion data and not a different service under Oracle Cloud.

3. Optimize REST Connector:

• Since the REST connector works but is slow, consider improving throughput by leveraging parallelism in Synapse pipelines. You can break down the export into smaller, paginated requests or use batch processing, if supported, to speed up the extraction. You may also want to adjust settings like degree of parallelism or concurrent connections in the REST connector to handle larger data volumes more efficiently.

4. Alternative Approach:

• If the built-in connectors are insufficient, consider building a custom solution using Azure Functions or Logic Apps to extract data from Oracle Fusion using its REST API. You can then store the data in Azure Data Lake. This method allows more control over error handling, data partitioning, and optimization of the extraction process.

There are third-party ETL tools such as Informatica or Talend that offer connectors for Oracle Cloud Fusion and can write to Azure Data Lake. These might provide better performance and reliability if Synapse connectors continue to pose challenges.

For a monthly full backup of Oracle Fusion, it's important to ensure the extraction process is both efficient and reliable. You might want to explore the bulk data export options within Oracle Fusion itself, as this could be better suited for large datasets than real-time APIs.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

@Dataarch_MM

Thanks for reaching out to Microsoft Q&A

Addressing Your Points:

Extra Parameters for Oracle Service Cloud Connector:

• Certificates: You might need to upload specific certificates to enable SSL communication. This can include trust certificates or identity certificates for two-way SSL communication.
• Authentication Tokens: Ensure you are using the correct tokens for authentication. This might involve OAuth tokens or JWTs, depending on your setup.
• Unfortunately, I couldn’t find a specific example of a successful connection to Oracle Fusion using the Oracle Service Cloud Connector. However, verifying the endpoint and ensuring all required parameters are correctly configured is crucial.

Self-hosted Integration Runtime (SHIR):

• Necessity: SHIR is often required when connecting to on-premises or private network data sources. For Oracle Cloud, it can help with network reliability and connectivity issues
• VM Location: The SHIR would typically reside on your side (Azure) rather than Oracle’s. It acts as a bridge between your on-premises network and Azure, ensuring secure and reliable data transfer

Cross-Cloud Connection Between OCI and Azure:

• Direct Interconnection: Setting up a direct interconnection between Azure and Oracle Cloud Infrastructure (OCI) using ExpressRoute and FastConnect can provide low latency and high throughput. This setup might seem complex but can be beneficial for large-scale, high-performance requirements.
• Overkill: If your primary need is monthly data export, this might indeed be more than necessary. However, it ensures robust and reliable connectivity, which could be advantageous in the long run.

OTHER APPROACH

Export to Oracle Storage: Exporting Oracle Fusion data to Oracle Storage and then using the Oracle Cloud Storage connector to transfer it to Azure Data Lake is a viable approach. This method can simplify the process and avoid potential issues with the Oracle Service Cloud Connector.

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.