Issue Summary

When testing a connection to a data source from Infoworks, users may observe that the Test Connection fails, even though the network settings on the Infoworks AKS cluster appear to be correct.

Root Cause

Infoworks jobs that use Databricks as the compute engine do not execute from the Infoworks AKS control plane. Instead, they run from Databricks-managed compute clusters. Therefore, any connectivity to the source system (e.g., databases, cloud storage, APIs) must be accessible from the Databricks network, not just from the Infoworks environment.

This is a common oversight—users often ensure connectivity from the AKS control plane but forget to allow network access from the Databricks side.

Applicable Scenarios

  • You're using Databricks as the compute engine in Infoworks

  • You're testing connections to:

    • RDBMS (e.g., Oracle, SQL Server, MySQL)

    • Cloud storage (e.g., AWS S3, Azure Blob)

    • Internal APIs or third-party endpoints

Symptoms

  • ❌ Test Connection fails in Infoworks UI

  • ✅ Connection works fine from within the AKS cluster (when tested manually)

  • ✅ JDBC/ODBC credentials are correct

  • Jobs fail during the source read step with connectivity errors (e.g., "Connection timed out", "Host unreachable")


Solution

Ensure the following network access is configured:

  1. ✅ Databricks compute clusters must have outbound access to the data source

    • Azure: Configure NSGs, Private Endpoints, or route tables to allow Databricks VNet to reach the source

    • AWS: Update VPC routing, security groups, or use VPC peering to ensure connectivity

  2. ✅ Whitelist Databricks IP ranges/subnets on the source system (if applicable)

  3. ✅ Validate connectivity from within a Databricks notebook, as shown below:

Testing Connectivity from Databricks

Open a notebook in Databricks and use %sh to run shell commands directly:

Check Port with nc (netcat)

%sh
nc -zv <hostname> <port>
# Example:
nc -zv my-db.internal.company.com 1433

Use telnet

%sh
telnet <hostname> <port>
# Example:
telnet my-db.internal.company.com 1521

Test HTTP/HTTPS with curl

bash
CopyEdit
%sh
curl -I https://api.example.com