Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V2.0.0 alpha #36

Merged
merged 164 commits into from
Jan 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
164 commits
Select commit Hold shift + click to select a range
a40553c
new version alpha development
Pwd9000-ML Jan 15, 2024
f85576d
fix workflows
Pwd9000-ML Jan 15, 2024
1d00a97
remove workflows
Pwd9000-ML Jan 15, 2024
8702115
Merge remote-tracking branch 'origin/master' into v2.0.0-alpha
Pwd9000-ML Jan 15, 2024
32451eb
rem
Pwd9000-ML Jan 15, 2024
09e80c7
up
Pwd9000-ML Jan 15, 2024
6b254a7
temp comment out
Pwd9000-ML Jan 15, 2024
f9f5592
lint
Pwd9000-ML Jan 15, 2024
5d889ef
fixes on data block
Pwd9000-ML Jan 15, 2024
8701942
cosmos
Pwd9000-ML Jan 15, 2024
580218f
test
Pwd9000-ML Jan 15, 2024
eb6372e
up
Pwd9000-ML Jan 15, 2024
a848a6c
test
Pwd9000-ML Jan 15, 2024
398adf0
tst
Pwd9000-ML Jan 15, 2024
e4562f8
tst
Pwd9000-ML Jan 15, 2024
4cb1f0c
test
Pwd9000-ML Jan 15, 2024
c31d65b
test
Pwd9000-ML Jan 15, 2024
bd97aa4
d
Pwd9000-ML Jan 15, 2024
ec52d9b
test
Pwd9000-ML Jan 15, 2024
65b1069
test
Pwd9000-ML Jan 15, 2024
b23bae9
tst
Pwd9000-ML Jan 15, 2024
e42b69e
update
Pwd9000-ML Jan 15, 2024
2fc0d6b
test
Pwd9000-ML Jan 15, 2024
66b6ec4
test
Pwd9000-ML Jan 15, 2024
5901345
test
Pwd9000-ML Jan 15, 2024
deaa05c
test
Pwd9000-ML Jan 16, 2024
f783862
test
Pwd9000-ML Jan 16, 2024
0bbbe97
test
Pwd9000-ML Jan 16, 2024
1f9aa2e
test
Pwd9000-ML Jan 16, 2024
da22e92
up
Pwd9000-ML Jan 16, 2024
3010897
test
Pwd9000-ML Jan 16, 2024
7b6e634
update modules
Pwd9000-ML Jan 17, 2024
f0d86a4
test
Pwd9000-ML Jan 17, 2024
24e74b8
up
Pwd9000-ML Jan 17, 2024
fdbbc2e
test
Pwd9000-ML Jan 17, 2024
ca445fc
refactor
Pwd9000-ML Jan 17, 2024
a657175
lint
Pwd9000-ML Jan 17, 2024
1690840
test
Pwd9000-ML Jan 17, 2024
382cbb2
refactor
Pwd9000-ML Jan 17, 2024
ddd9543
up
Pwd9000-ML Jan 17, 2024
d3be512
update module
Pwd9000-ML Jan 17, 2024
19a2a32
fix
Pwd9000-ML Jan 17, 2024
7091feb
update
Pwd9000-ML Jan 17, 2024
6ff2a17
lint
Pwd9000-ML Jan 17, 2024
5cae418
up
Pwd9000-ML Jan 17, 2024
ed01777
fix
Pwd9000-ML Jan 17, 2024
aed73ef
test
Pwd9000-ML Jan 17, 2024
54d7eff
test
Pwd9000-ML Jan 17, 2024
6327155
update
Pwd9000-ML Jan 17, 2024
88a65d6
up
Pwd9000-ML Jan 17, 2024
c9c52ce
fix
Pwd9000-ML Jan 17, 2024
87e4b09
up
Pwd9000-ML Jan 17, 2024
1b30054
up
Pwd9000-ML Jan 17, 2024
6e2373b
test
Pwd9000-ML Jan 17, 2024
30d4126
up
Pwd9000-ML Jan 17, 2024
5f8791a
deploy kv
Pwd9000-ML Jan 17, 2024
c718ba7
up
Pwd9000-ML Jan 17, 2024
728bf20
test
Pwd9000-ML Jan 17, 2024
8033b3a
dependencies
Pwd9000-ML Jan 17, 2024
8b4e401
cognitive service
Pwd9000-ML Jan 17, 2024
1d4b083
fixes
Pwd9000-ML Jan 17, 2024
577ef3e
cleanup
Pwd9000-ML Jan 17, 2024
28b509f
fix sub domain name
Pwd9000-ML Jan 17, 2024
30a02eb
test
Pwd9000-ML Jan 17, 2024
39f8483
deploy cosmosdb
Pwd9000-ML Jan 17, 2024
d39465c
add var for subnet ids in kv
Pwd9000-ML Jan 17, 2024
5fde2a8
test asp and meiliserach and kv
Pwd9000-ML Jan 17, 2024
b005b86
test
Pwd9000-ML Jan 17, 2024
f3537dd
up
Pwd9000-ML Jan 17, 2024
7557dc3
documentation
Pwd9000-ML Jan 17, 2024
9bde748
add app config
Pwd9000-ML Jan 19, 2024
30b9999
UPDATE VARS
Pwd9000-ML Jan 20, 2024
01f1688
update vars
Pwd9000-ML Jan 20, 2024
f469a8d
alpha test run
Pwd9000-ML Jan 20, 2024
a7fec65
fixes on vars
Pwd9000-ML Jan 20, 2024
7d47479
fixes - add identity to libre app
Pwd9000-ML Jan 20, 2024
0225c5c
fix random output being shown in screen
Pwd9000-ML Jan 20, 2024
c09d76b
sdfsdf
Pwd9000-ML Jan 20, 2024
5167de0
change region
Pwd9000-ML Jan 20, 2024
b6159b5
add vision
Pwd9000-ML Jan 21, 2024
c77e42d
test
Pwd9000-ML Jan 21, 2024
d789902
up
Pwd9000-ML Jan 21, 2024
4bd8791
test
Pwd9000-ML Jan 21, 2024
4e01366
test
Pwd9000-ML Jan 21, 2024
81c80cc
test
Pwd9000-ML Jan 21, 2024
7ddc07e
test
Pwd9000-ML Jan 21, 2024
3ecb822
test
Pwd9000-ML Jan 21, 2024
ad46429
test
Pwd9000-ML Jan 21, 2024
a859d71
test
Pwd9000-ML Jan 21, 2024
f908d4e
test new
Pwd9000-ML Jan 21, 2024
4d25b67
fix casing
Pwd9000-ML Jan 21, 2024
1b9aa64
test
Pwd9000-ML Jan 21, 2024
a6b93ee
tst
Pwd9000-ML Jan 21, 2024
330a175
change keys
Pwd9000-ML Jan 21, 2024
cec5792
test
Pwd9000-ML Jan 21, 2024
107f62c
test
Pwd9000-ML Jan 21, 2024
7ce10c6
test
Pwd9000-ML Jan 21, 2024
72ecdaf
test
Pwd9000-ML Jan 21, 2024
f387700
test
Pwd9000-ML Jan 21, 2024
0e97190
test with random string
Pwd9000-ML Jan 21, 2024
df59f09
update
Pwd9000-ML Jan 21, 2024
90cf9f9
up
Pwd9000-ML Jan 21, 2024
616e9bd
up
Pwd9000-ML Jan 21, 2024
1151489
up
Pwd9000-ML Jan 21, 2024
8ad13d9
up
Pwd9000-ML Jan 21, 2024
c92172c
always on
Pwd9000-ML Jan 21, 2024
c4b1e86
test
Pwd9000-ML Jan 21, 2024
76d347c
test
Pwd9000-ML Jan 21, 2024
fbd047e
test
Pwd9000-ML Jan 21, 2024
effbd8c
test
Pwd9000-ML Jan 21, 2024
0ed39d0
test
Pwd9000-ML Jan 21, 2024
ea1dac1
test
Pwd9000-ML Jan 21, 2024
ec375e0
test
Pwd9000-ML Jan 21, 2024
e041460
test uksouth
Pwd9000-ML Jan 22, 2024
e989660
tws
Pwd9000-ML Jan 22, 2024
b3cc079
add meilisearch key as
Pwd9000-ML Jan 22, 2024
03c22a9
update
Pwd9000-ML Jan 22, 2024
a6f49bb
add dependencies
Pwd9000-ML Jan 22, 2024
8c971cc
add subnet dependencies
Pwd9000-ML Jan 22, 2024
5077ff1
tet
Pwd9000-ML Jan 22, 2024
68cec22
t
Pwd9000-ML Jan 22, 2024
36014f5
verbos elog
Pwd9000-ML Jan 22, 2024
b662db0
test
Pwd9000-ML Jan 22, 2024
3647888
up
Pwd9000-ML Jan 22, 2024
9391c55
dd
Pwd9000-ML Jan 22, 2024
a6c1cf9
test
Pwd9000-ML Jan 22, 2024
4c32af5
test
Pwd9000-ML Jan 22, 2024
b4d3d02
test
Pwd9000-ML Jan 22, 2024
1fd0e6d
test
Pwd9000-ML Jan 22, 2024
fbd5c85
test
Pwd9000-ML Jan 22, 2024
4ae2e4d
test
Pwd9000-ML Jan 22, 2024
92ed4e6
test
Pwd9000-ML Jan 22, 2024
6351b69
test
Pwd9000-ML Jan 22, 2024
15d3038
test
Pwd9000-ML Jan 22, 2024
105fbbc
test
Pwd9000-ML Jan 22, 2024
6d856be
test
Pwd9000-ML Jan 22, 2024
d62a816
lint
Pwd9000-ML Jan 22, 2024
f5a2214
test
Pwd9000-ML Jan 22, 2024
292c0b7
disable meili
Pwd9000-ML Jan 22, 2024
fac6c96
fix
Pwd9000-ML Jan 22, 2024
851bfde
remove meili
Pwd9000-ML Jan 22, 2024
230d46d
add ip_restrictions
Pwd9000-ML Jan 22, 2024
c2d0fc9
upgrade Terraform version
Pwd9000-ML Jan 22, 2024
01ec99a
fix
Pwd9000-ML Jan 22, 2024
ae093b0
fix
Pwd9000-ML Jan 22, 2024
de4690c
add todo
Pwd9000-ML Jan 22, 2024
c715184
up
Pwd9000-ML Jan 22, 2024
4b7e650
up
Pwd9000-ML Jan 22, 2024
2a4f5c4
change region to sweden central for Dall E
Pwd9000-ML Jan 22, 2024
598d190
up
Pwd9000-ML Jan 22, 2024
c68c756
lint
Pwd9000-ML Jan 22, 2024
4d05177
update app
Pwd9000-ML Jan 22, 2024
633d589
update app
Pwd9000-ML Jan 22, 2024
f26bc20
u[date
Pwd9000-ML Jan 22, 2024
4cf6801
test
Pwd9000-ML Jan 22, 2024
599d57b
up
Pwd9000-ML Jan 22, 2024
1aec2c7
liny
Pwd9000-ML Jan 22, 2024
c658542
deploy
Pwd9000-ML Jan 22, 2024
3d2fef6
lint
Pwd9000-ML Jan 22, 2024
2e77d04
up
Pwd9000-ML Jan 22, 2024
b35abe6
fui
Pwd9000-ML Jan 22, 2024
0c1cd90
up
Pwd9000-ML Jan 22, 2024
0764931
up
Pwd9000-ML Jan 22, 2024
d12d7f6
lint
Pwd9000-ML Jan 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@ updates:
- package-ecosystem: "terraform"
directory: "/"
schedule:
interval: "daily"
interval: "weekly"
day: "monday"
time: "00:00"
timezone: "Europe/London"

- package-ecosystem: "github-actions" # See documentation for possible values
directory: "/" # Location of package manifests
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
day: "sunday"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/manual-test-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- name: Checkout
uses: actions/[email protected]

- name: Run Dependency Tests - Plan AND Apply AND Destroy
- name: Plan AND Apply AND Destroy
uses: Pwd9000-ML/[email protected]
with:
test_type: plan-apply-destroy ## (Required) Valid options are "plan", "plan-apply", "plan-apply-destroy". Default="plan"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/manual-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
workflow_dispatch:

jobs:
manual_plan_apply_destroy:
manual_plan_apply:
runs-on: ubuntu-latest
permissions:
pull-requests: write
Expand All @@ -21,7 +21,7 @@ jobs:
- name: Checkout
uses: actions/[email protected]

- name: Run Dependency Tests - Plan AND Apply AND Destroy
- name: Plan AND Apply Only
uses: Pwd9000-ML/[email protected]
with:
test_type: plan-apply ## (Required) Valid options are "plan", "plan-apply", "plan-apply-destroy". Default="plan"
Expand Down
6 changes: 6 additions & 0 deletions 01_resource_group.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Create Solution Resource Group
resource "azurerm_resource_group" "az_openai_rg" {
name = var.resource_group_name
location = var.location
tags = var.tags
}
33 changes: 33 additions & 0 deletions 02_networking.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Create Solution Virtual Network
resource "azurerm_virtual_network" "az_openai_vnet" {
name = var.virtual_network_name
location = var.location
resource_group_name = azurerm_resource_group.az_openai_rg.name
address_space = var.vnet_address_space
tags = var.tags
}

# Azure Virtual Network Subnets
resource "azurerm_subnet" "az_openai_subnet" {
resource_group_name = azurerm_resource_group.az_openai_rg.name
virtual_network_name = azurerm_virtual_network.az_openai_vnet.name
name = var.subnet_config.subnet_name
address_prefixes = var.subnet_config.subnet_address_space
service_endpoints = var.subnet_config.service_endpoints
private_link_service_network_policies_enabled = var.subnet_config.private_link_service_network_policies_enabled
private_endpoint_network_policies_enabled = var.subnet_config.private_endpoint_network_policies_enabled

dynamic "delegation" {
for_each = var.subnet_config.subnets_delegation_settings
content {
name = delegation.key
dynamic "service_delegation" {
for_each = toset(delegation.value)
content {
name = service_delegation.value.name
actions = service_delegation.value.actions
}
}
}
}
}
27 changes: 27 additions & 0 deletions 03_keyvault.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Key Vault - Create Key Vault to save cognitive account, cosmosDB, App details
resource "azurerm_key_vault" "az_openai_kv" {
resource_group_name = azurerm_resource_group.az_openai_rg.name
location = var.location
#values from variable kv_config object
name = lower(var.kv_name)
sku_name = var.kv_sku
enable_rbac_authorization = true
tenant_id = data.azurerm_client_config.current.tenant_id
network_acls {
default_action = var.kv_fw_default_action
bypass = var.kv_fw_bypass
ip_rules = var.kv_fw_allowed_ips
virtual_network_subnet_ids = var.kv_fw_network_subnet_ids != null ? var.kv_fw_network_subnet_ids : azurerm_subnet.az_openai_subnet.*.id
}
tags = var.tags
depends_on = [azurerm_subnet.az_openai_subnet]
}


# Add "self" permission to key vault RBAC (to manange key vault secrets)
resource "azurerm_role_assignment" "kv_role_assigment" {
for_each = toset(["Key Vault Administrator"])
role_definition_name = each.key
scope = azurerm_key_vault.az_openai_kv.id
principal_id = data.azurerm_client_config.current.object_id
}
92 changes: 92 additions & 0 deletions 04_az_openai.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Create OpenAI Cognitive Account
resource "azurerm_cognitive_account" "az_openai" {
kind = "OpenAI"
location = var.location
name = var.oai_account_name
resource_group_name = azurerm_resource_group.az_openai_rg.name
sku_name = var.oai_sku_name
custom_subdomain_name = var.oai_custom_subdomain_name
dynamic_throttling_enabled = var.oai_dynamic_throttling_enabled
fqdns = var.oai_fqdns
local_auth_enabled = var.oai_local_auth_enabled
outbound_network_access_restricted = var.oai_outbound_network_access_restricted
public_network_access_enabled = var.oai_public_network_access_enabled
tags = var.tags

dynamic "customer_managed_key" {
for_each = var.oai_customer_managed_key != null ? [var.oai_customer_managed_key] : []
content {
key_vault_key_id = customer_managed_key.value.key_vault_key_id
identity_client_id = customer_managed_key.value.identity_client_id
}
}

dynamic "identity" {
for_each = var.oai_identity != null ? [var.oai_identity] : []
content {
type = identity.value.type
identity_ids = identity.value.identity_ids
}
}

dynamic "network_acls" {
for_each = var.oai_network_acls != null ? [var.oai_network_acls] : []
content {
default_action = network_acls.value.default_action
ip_rules = network_acls.value.ip_rules

dynamic "virtual_network_rules" {
for_each = network_acls.value.virtual_network_rules != null ? network_acls.value.virtual_network_rules : []
content {
subnet_id = virtual_network_rules.value.subnet_id
ignore_missing_vnet_service_endpoint = virtual_network_rules.value.ignore_missing_vnet_service_endpoint
}
}
}
}

dynamic "storage" {
for_each = var.oai_storage
content {
storage_account_id = storage.value.storage_account_id
identity_client_id = storage.value.identity_client_id
}
}
}

# Create OpenAI Cognitive Account Model Deployments
resource "azurerm_cognitive_deployment" "az_openai_models" {
for_each = { for each in var.oai_model_deployment : each.deployment_id => each }

cognitive_account_id = azurerm_cognitive_account.az_openai.id
name = each.value.deployment_id
rai_policy_name = each.value.rai_policy_name

model {
format = each.value.model_format
name = each.value.model_name
version = each.value.model_version
}
scale {
type = each.value.scale_type
tier = each.value.scale_tier
size = each.value.scale_size
family = each.value.scale_family
capacity = each.value.scale_capacity
}
}

# Save OpenAI Cognitive Account details to Key Vault for consumption by other services
resource "azurerm_key_vault_secret" "openai_endpoint" {
name = "${var.oai_account_name}-openai-endpoint"
value = azurerm_cognitive_account.az_openai.endpoint
key_vault_id = azurerm_key_vault.az_openai_kv.id
depends_on = [azurerm_role_assignment.kv_role_assigment]
}

resource "azurerm_key_vault_secret" "openai_primary_key" {
name = "${var.oai_account_name}-openai-key"
value = azurerm_cognitive_account.az_openai.primary_access_key
key_vault_id = azurerm_key_vault.az_openai_kv.id
depends_on = [azurerm_role_assignment.kv_role_assigment]
}
51 changes: 51 additions & 0 deletions 05_cosmosdb.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Create CosmosDB Account running MongoDB
resource "azurerm_cosmosdb_account" "az_openai_mongodb" {
name = var.cosmosdb_name
resource_group_name = azurerm_resource_group.az_openai_rg.name
location = var.location
offer_type = var.cosmosdb_offer_type
kind = var.cosmosdb_kind
enable_automatic_failover = var.cosmosdb_automatic_failover
enable_free_tier = var.use_cosmosdb_free_tier
tags = var.tags

consistency_policy {
consistency_level = var.cosmosdb_consistency_level
max_interval_in_seconds = var.cosmosdb_max_interval_in_seconds
max_staleness_prefix = var.cosmosdb_max_staleness_prefix
}

dynamic "geo_location" {
for_each = var.cosmosdb_geo_locations
content {
location = geo_location.value.location
failover_priority = geo_location.value.failover_priority
}
}

dynamic "capabilities" {
for_each = var.cosmosdb_capabilities
content {
name = capabilities.value
}
}

dynamic "virtual_network_rule" {
for_each = var.cosmosdb_virtual_network_subnets != null ? var.cosmosdb_virtual_network_subnets : azurerm_subnet.az_openai_subnet.*.id
content {
id = virtual_network_rule.value
}
}

is_virtual_network_filter_enabled = var.cosmosdb_is_virtual_network_filter_enabled
public_network_access_enabled = var.cosmosdb_public_network_access_enabled
depends_on = [azurerm_subnet.az_openai_subnet]
}

### Save MongoDB URI details to Key Vault for consumption by other services (e.g. LibreChat App)
resource "azurerm_key_vault_secret" "openai_cosmos_uri" {
name = "${var.cosmosdb_name}-cosmos-uri"
value = azurerm_cosmosdb_account.az_openai_mongodb.primary_mongodb_connection_string
key_vault_id = azurerm_key_vault.az_openai_kv.id
depends_on = [azurerm_role_assignment.kv_role_assigment]
}
Loading