Data Factory Managed Virtual Network to be Deployed by Bicep

Problem

I’m attempting to make a bicep module that deploys a data factory and a managed vnet. Here’s what I’ve got:

param dfName string
 param sqlId string

 resource df 'Microsoft.DataFactory/factories@2018-06-01' = {
   name: dfName
   location: resourceGroup().location
   identity: {
     type: 'SystemAssigned'
   }
 }

 resource integrationRuntime 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = {
   name: '${dfName}/managedVnetIr' 
   properties: {
     type: 'Managed'
     typeProperties: {
       computeProperties: {
         location: 'AutoResolve'
         dataFlowProperties: {
           computeType: 'General'
           coreCount: 8
           timeToLive: 0
         }
       }
     }
   }
   dependsOn: [
     df
   ]
 }

 resource managedVnet 'Microsoft.DataFactory/factories/managedVirtualNetworks@2018-06-01' = {
   name: '${dfName}/vnet'
   properties: { 
   }
   dependsOn: [
     integrationRuntime
   ]
 }

 resource managedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = {
   name: '${dfName}/vnet/pe'
   properties: {
     privateLinkResourceId:sqlId
     groupId: 'sql'
   }
   dependsOn: [
     managedVnet
   ]
 }

 output dfId string = df.identity.principalId

I receive the following error when I execute this module:

I’ve also experimented with the following: (based on answer from AnsumanBal-MT)

param dfName string
param sqlId string
param vnetName string

resource df 'Microsoft.DataFactory/factories@2018-06-01' = {
  name: dfName
  location: resourceGroup().location
  identity: {
    type: 'SystemAssigned'
  }
}

resource integrationRuntime 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = {
  parent: df
  name: '${dfName}-managedVnetIr' 
  properties: {
    type: 'Managed'
    typeProperties: {
      computeProperties: {
        location: 'AutoResolve'
        dataFlowProperties: {
          computeType: 'General'
          coreCount: 8
          timeToLive: 0
        }
      }
    }
  }
}

resource managedVnet 'Microsoft.DataFactory/factories/managedVirtualNetworks@2018-06-01' = {
  parent:df
  name: vnetName
  properties: { 
  }
  dependsOn: [
    integrationRuntime
  ]
}

resource managedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = {
  parent:managedVnet
  name: '${vnetName}-sql-pe'
  properties: {
    privateLinkResourceId:sqlId
    groupId: 'sql'
  }
  dependsOn: [
    managedVnet
  ]
}

output dfId string = df.identity.principalId

However, this results in the following error:

Could someone either point out where I went wrong or point me to a working example?

Asked by Rob Bowman

Solution #1

You must refer to an existing Vnet in your resource group when constructing a managed virtual network on the Data Factory.

Update:1

I ran into the same issue as you while testing for a managed private endpoint for a SQL database; using your code, it failed after 1hr 18mins with provisioning failed.

I discovered two errors during testing for SQL server: the groupId should be sqlServer, and the managed vnet for adf will not be able to communicate with SQL server because it is not added to the firewall and virtual networks.

To resolve the problem, take the following two steps:

The deployment will be successful once the first two phases are completed.

Update:2

Construct a SQL Server with a Vnet, then use the vnet and sql to create an adf controlled virtual network and a private endpoint.

Please use the code below, which I have tested to meet your needs:

param serverName string = uniqueString('sql', resourceGroup().id)
param sqlDBName string = 'SampleDB'
param administratorLogin string
@secure()
param administratorLoginPassword string
param virtualNetworkName string = 'azure_mysql_vnet'
param subnetName string = 'azure_mysql_subnet'
param virtualNetworkRuleName string = 'AllowSubnet'
param vnetAddressPrefix string = '10.0.0.0/16'
param subnetPrefix string = '10.0.0.0/16'
param dfName string

resource virtualNetworkName_resource 'Microsoft.Network/virtualNetworks@2020-06-01' = {
  name: virtualNetworkName
  location: resourceGroup().location
  properties: {
    addressSpace: {
      addressPrefixes: [
        vnetAddressPrefix
      ]
    }
  }
}

resource virtualNetworkName_subnetName 'Microsoft.Network/virtualNetworks/subnets@2020-06-01' = {
  parent: virtualNetworkName_resource
  name: subnetName
  location: resourceGroup().location
  properties: {
    addressPrefix: subnetPrefix
  }
}

resource serverName_resource 'Microsoft.Sql/servers@2020-02-02-preview' = {
  name: serverName
  location: resourceGroup().location
  properties: {
    administratorLogin: administratorLogin
    administratorLoginPassword: administratorLoginPassword
  }
}

resource serverName_sqlDBName 'Microsoft.Sql/servers/databases@2020-08-01-preview' = {
  parent: serverName_resource
  name: sqlDBName
  location: resourceGroup().location
  sku: {
    name: 'Standard'
    tier: 'Standard'
  }
}

resource serverName_virtualNetworkRuleName 'Microsoft.Sql/servers/virtualNetworkRules@2021-02-01-preview' = {
  parent: serverName_resource
  name: virtualNetworkRuleName
  properties: {
    virtualNetworkSubnetId: virtualNetworkName_subnetName.id
    ignoreMissingVnetServiceEndpoint: true
  }
}

 resource df 'Microsoft.DataFactory/factories@2018-06-01' = {
   name: dfName
   location: resourceGroup().location
   identity: {
     type: 'SystemAssigned'
   }
 } 

 resource integrationRuntime 'Microsoft.DataFactory/factories/integrationRuntimes@2018-06-01' = {
   parent: df
   name: '${dfName}-managedVnetIr' 
   properties: {
     type: 'Managed'
     typeProperties: {
       computeProperties: {
         location: 'AutoResolve'
         dataFlowProperties: {
           computeType: 'General'
           coreCount: 8
           timeToLive: 0
         }
       }
     }
   }
 } 
 resource managedVnet 'Microsoft.DataFactory/factories/managedVirtualNetworks@2018-06-01' = {
   parent:df
   name: virtualNetworkName
   properties: { 
   }
   dependsOn: [
     integrationRuntime
   ]
 }

 resource managedPrivateEndpoint 'Microsoft.DataFactory/factories/managedVirtualNetworks/managedPrivateEndpoints@2018-06-01' = {
   parent:managedVnet
   name: '${virtualNetworkName}-${serverName}-pe'
   properties: {
     privateLinkResourceId: serverName_resource.id
     groupId: 'sqlServer'
   }
   dependsOn: [
     managedVnet
   ]
 }

Output:

Note: After the deployment is complete, you must manually approve the pending private endpoint request from SQL server, as shown below:

Answered by AnsumanBal-MT

Post is based on https://stackoverflow.com/questions/69569062/bicep-to-deploy-data-factory-managed-virtual-network