Problem
I have a bicep script that will conduct the following steps:
The infrastructure is divided into two resource groups:
rg-shared => where the 2 key vault are (1 keyvault for key and GUID, and 1 keyvault for secrets (connection string)
rg-storage-account => where all the storage account get created
Biceps in azure The following are the scripts I have:
Storage.bicep
param ManagedIdentityid string
param uri string
param kvname string
param keyvaultrg string = 'XXXX' //<== SHARED Resource Group
var keyVaultKeyPrefix = 'Key-Data-'
var storagePrefix = 'sthritesteur'
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
// Create storage accounts
resource storageAccount 'Microsoft.Storage/storageAccounts@2021-04-01' = [for tenantCode in tenantCodes: {
name: '${storagePrefix}${tenantCode}'
location: resourceGroup().location
kind: 'StorageV2'
sku: {
name: 'Standard_RAGRS'
}
// Assign the identity
identity: {
type: 'UserAssigned'
userAssignedIdentities: {
'${ManagedIdentityid}':{}
}
}
properties: {
allowCrossTenantReplication: true
minimumTlsVersion: 'TLS1_2'
allowBlobPublicAccess: false
allowSharedKeyAccess: true
networkAcls: {
bypass: 'AzureServices'
virtualNetworkRules: []
ipRules: []
defaultAction: 'Allow'
}
supportsHttpsTrafficOnly: true
encryption: {
identity: {
// specify which identity to use
userAssignedIdentity: ManagedIdentityid
}
keySource: 'Microsoft.Keyvault'
keyvaultproperties: {
keyname: '${kvname}-${keyVaultKeyPrefix}${toUpper(tenantCode)}'
keyvaulturi:uri
}
services: {
file: {
keyType: 'Account'
enabled: true
}
blob: {
keyType: 'Account'
enabled: true
}
}
}
accessTier: 'Hot'
}
}]
resource storage_Accounts_name_default 'Microsoft.Storage/storageAccounts/blobServices@2021-04-01' = [ for (storageName, i) in tenantCodes :{
parent: storageAccount[i]
name: 'default'
properties: {
changeFeed: {
enabled: false
}
restorePolicy: {
enabled: false
}
containerDeleteRetentionPolicy: {
enabled: true
days: 7
}
cors: {
corsRules: []
}
deleteRetentionPolicy: {
enabled: true
days: 30
}
isVersioningEnabled: true
}
}]
module connectionString 'shared.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(keyvaultrg)
name: storageName
params: {
storageAccountString: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
}
}]
keyvaultclient.bicep
param deploymentIdOne string = newGuid()
param deploymentIdTwo string = newGuid()
output deploymentIdOne string = '${deploymentIdOne}-${deploymentIdTwo}'
output deploymentIdTwo string = deploymentIdTwo
param storagerg string = 'XXXX' //<=== Storage Accounts Resource Groups
param sharedManagedIdentity string = 'mgn-identity-shared'
param keyvaultmain string = 'XXXX' //<=== KeyVault Name where to create GUID AND Keys
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
var clientDataKeyPrefix = 'Key-Data-'
resource managedIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2018-11-30' = {
name: sharedManagedIdentity
location: resourceGroup().location
}
resource keyVaultClients 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
name: keyvaultmain
}
resource kvClientsKey 'Microsoft.KeyVault/vaults/keys@2021-06-01-preview' = [for code in tenantCodes: {
parent:keyVaultClients
name: '${keyVaultClients.name}-${clientDataKeyPrefix}${toUpper(code)}'
properties: {
keySize: 2048
kty: 'RSA'
// Assign the least permission
keyOps: [
'unwrapKey'
'wrapKey'
]
}
}]
resource accessPolicy 'Microsoft.KeyVault/vaults/accessPolicies@2021-06-01-preview' = {
parent:keyVaultClients
name: 'add'
properties: {
accessPolicies: [
{
tenantId: subscription().tenantId
objectId: managedIdentity.properties.principalId
permissions: {
// minimum required permission
keys: [
'get'
'unwrapKey'
'wrapKey'
]
}
}
]
}
}
resource clientLearnersGuid 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultClients
name: '${keyVaultClients.name}${tenant}'
properties: {
contentType: 'GUID Key'
value: '${deploymentIdOne}-${deploymentIdTwo}'
}
dependsOn:kvClientsKey
}]
module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(storagerg)
name: storageName
params: {
ManagedIdentityid:managedIdentity.id
kvname:keyVaultClients.name
uri:keyVaultClients.properties.vaultUri
}
dependsOn:clientLearnersGuid
}]
shared.bicep
param keyvaultshared string = 'XXXX' //<=== Key Vault Where to Store the Storage Connection String Secret
param storageAccountString string
param tenantCodes array = [
'aabb'
'bbcc'
'ccdd'
]
resource keyVaultShared 'Microsoft.KeyVault/vaults@2021-06-01-preview' existing = {
name: keyvaultshared
}
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultShared
name: '${keyVaultShared.name}-test${tenant}'
properties:{
contentType: '${tenant} Storage Account Connection String'
value: storageAccountString
}
}]
Based on the tenantCode, that script completes all of the tasks I required. That’s exactly right. Everything goes smoothly and perfectly if I only specify one tenantCode, but the problem I’m having arises when I try to declare more than one. And this is the specific difficulty.
When I declare several codes, the script still generates all of the resources I require, including storage accounts, keys, encryptions, GUIDs, and ConnectionStrings Secrets. Nonetheless, it fails on the ConnectionStrings Secret.
The reason it fails is because of the following code block in those files:
Shared.bicep
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for tenant in tenantCodes: {
parent:keyVaultShared
name: '${keyVaultShared.name}-test${tenant}'
properties:{
contentType: '${tenant} Storage Account Connection String'
value: storageAccountString
}
}]
and
keuvaultsclient.bicep*
module StorageAccount 'storage.bicep' = [for (storageName, i) in tenantCodes :{
scope: resourceGroup(storagerg)
name: storageName
params: {
ManagedIdentityid:managedIdentity.id
kvname:keyVaultClients.name
uri:keyVaultClients.properties.vaultUri
}
dependsOn:clientLearnersGuid
}]
I have a multiple loop in which I discovered that I had the correct number of secrets in my shared keyvault under secrets (with three tenant codes, I have three secrets) and three versions of each secret (for each tenant code it generate a new version for each secret). The bicep script fails due to a looping problem, with the following message:
{"status":"Failed","error":{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"Conflict","message":"{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n }\\r\\n}\"\r\n },\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"StorageAccountOperationInProgress\\\",\\r\\n \\\"message\\\": \\\"An operation is currently performing on this storage account that requires exclusive access.\\\"\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"}]}}
I’m completely stuck at this point, and as a biceps novice, I’m out of ideas and testing to try to fix the problem.
How to reproduce
I believe I’ve expressed the problem I’m having clearly enough, and please let me know if you require any additional information.
Thank you very much for your time and assistance.
Asked by Nayden Van
Solution #1
I tested the same code with additional modifications; please try the following changes in the.bicep files:
keyvaultclient.bicep:
The loop for module was removed because it was creating two modules for the same item.
module StorageAccount './storage.bicep' = {
scope: resourceGroup(storagerg)
name: 'NestedStorage'
params: {
ManagedIdentityid:managedIdentity.id
kvname:keyVaultClients.name
uri:keyVaultClients.properties.vaultUri
}
dependsOn:clientLearnersGuid
}
storage.bicep:
The outputs will be stored in an array and passed to the next module using this string.
module connectionString './shared.bicep'={
scope: resourceGroup(keyvaultrg)
name: 'KeyvaultNested'
params: {
storageAccountString: [for (tenant,i) in tenantCodes :{
id:'DefaultEndpointsProtocol=https;AccountName=${storageAccount[i].name};AccountKey=${listKeys(storageAccount[i].id, storageAccount[i].apiVersion).keys[0].value};EndpointSuffix=${environment().suffixes.storage}'
}]
}
}
shared.bicep:
I changed the StorageAccountString parameter type from string to array and inserted [for (tenant,i) in tenantCodes in the secret portion so that I can use storageAccountString[i].id as the value.
param storageAccountString array
resource storageAccountConnectionString 'Microsoft.KeyVault/vaults/secrets@2021-06-01-preview' = [for (tenant,i) in tenantCodes: {
parent:keyVaultShared
name: '${keyVaultShared.name}-test${tenant}'
properties:{
contentType: '${tenant} Storage Account Connection String'
value: storageAccountString[i].id
}
}]
Outputs:
Answered by AnsumanBal-MT
Post is based on https://stackoverflow.com/questions/69735711/azure-bicep-pass-storage-account-connection-string-to-secret-keyvault-loop-issue