gcp.mdx
1 --- 2 title: Deploying MLflow to Google Cloud platform 3 --- 4 5 # Deploying MLflow to Google Cloud platform 6 7 MLflow core components include: 8 9 - **MLflow server**: This is the Python backend for experiment tracking, model registry management, tracing for LLM observability, job scheduling, and AI Gateway endpoints. 10 - **Backend store**: The database in which entity metadata (e.g. trace info, experiment tracking metadata) is stored. 11 - **Artifact store**: Blob storage for larger pieces of persisted data (e.g. model weights, trace span data). 12 13 This guide walks you through deploying the MLflow server to Google Cloud Run, the backend store to Cloud SQL (PostgreSQL), and the artifact store to Google Cloud Storage (GCS). The guide also covers IAM service accounts, Cloud SQL connectivity, and Cloud Run networking settings. Once deployment is complete, you can access the MLflow web UI through a Cloud Run service URL such as: `https://mlflow-<unique-id>.<region>.run.app`. Your MLflow client code can connect to the MLflow server by setting the tracking URI to this URL. 14 15 The overall deployment architecture is as follows: 16 17  18 19 The deployment architecture has a couple of advantages: 20 21 - **High Availability** 22 - Cloud Storage (GCS) provides highly durable and globally distributed object storage for MLflow artifacts. Data is replicated across multiple zones within the selected region or multi-region location. 23 - Cloud SQL (PostgreSQL) supports automated backups and high-availability configurations with automatic failover to a standby instance in another zone, reducing downtime for the MLflow backend store. 24 - Cloud Run is a fully managed serverless platform that automatically restarts failed containers and scales the MLflow service based on incoming requests, ensuring high availability without manual infrastructure management. 25 26 - **Security by design** 27 - Cloud Run endpoints are automatically served over HTTPS, ensuring secure communication between MLflow clients and the server. 28 - The Google Cloud storage bucket blocks public access, and the MLflow server running in Cloud Run accesses the artifact store securely through a service account with the "Storage Object User" role. 29 - Cloud Run can connect to Cloud SQL using the built-in Cloud SQL integration or via the Cloud SQL Auth Proxy mechanism, avoiding the need to expose the database publicly. 30 - The architecture can integrate with MLflow authentication configurations. 31 32 - **Operational Simplicity** 33 - Serverless Compute (Cloud Run), no virtual machines to manage, patch, or scale manually. 34 - Managed Database (Cloud SQL): Automated backups, patching, and failover. 35 - Managed Object Storage (Cloud Storage): Highly scalable and durable storage for MLflow artifacts, eliminating the need to manage scalable storage infrastructure. 36 37 ## Step 1: Create a docker image and upload it to GCP: 38 39 Build a docker image using the following `Dockerfile`: 40 41 ```docker 42 FROM ghcr.io/mlflow/mlflow:<mlflow-version>-full 43 44 RUN pip install google-cloud-storage 45 ``` 46 47 The `<mlflow-version>` in the Dockerfile is a value like `v3.10.0`. 48 49 This docker image is based on the official MLflow docker image, but preinstalls "google-cloud-storage" package which is required by MLflow server to access Google Cloud storage. 50 51 In Google Cloud "Artifact Registry" console, create a repository with name of "mlflow-repo", set the repository format to "Docker", the repository path is like `<region>-docker.pkg.dev/<gcp-project-name>/mlflow-repo`, then build the above docker image and push it to the Google docker repository as follows: 52 53 ```bash 54 docker build -t <google-docker-repository-path>/mlflow-gcp:v3.10.0 . 55 56 docker push <google-docker-repository-path>/mlflow-gcp:v3.10.0 57 ``` 58 59 ## Step 2: Create a bucket in Google Cloud Storage 60 61 In Google Cloud "Cloud Storage" console, create a bucket with name like "mlflow-artifact-12345" as follows, note that you should turn on "Public access prevention" option to block public access to the bucket. 62 63  64 65 ## Step 3: Grant user role to the default compute service account 66 67 In Google Cloud "IAM & Admin" console, click "grant access", then assign the "Storage Object User" role to the default compute service account as follows: 68 69  70 71 ## Step 4: Create cloud SQL instance 72 73 In Google Cloud "Cloud SQL" console, click "Create an instance" button to create a "PostgreSQL" instance as follows: 74 75  76 77 The database connection string used by MLflow is like: 78 79 `postgresql://<admin-name>:<admin-password>@/<database-name>?host=/cloudsql/<gcp-project-name>:us-central1:<cloud-sql-instance-id>` 80 81 The default values of "admin-name" and "database-name" are both "postgres". 82 83 ## Step 5: Create Cloud Run instance 84 85 In Google Cloud "Cloud Run" console, click "Deploy container" button to create a cloud run instance. You need to configure the following items correctly: 86 87 - Container image URL: Select the "mlflow-gcp" image that you pushed to the Google docker repository. 88 - Service scaling: set both minimum and maximum number of instances to 1. 89 - Ingress: All (Allow direct access to your service from the internet) 90 - Container port: 5000 91 - Container command: mlflow 92 - Container arguments: `server --backend-store-uri <database-connection-string> --artifacts-destination gs://<cloud-storage-bucket-name> --host 0.0.0.0 --port 5000 --disable-security-middleware` 93 - Resources: at least 2GiB memory and 1 CPU. 94 - Cloud SQL connections: Click "Add connection" button and add the connection to the cloud SQL instance that is created in "Step 4". 95 96 After created the cloud run instance, you can view the application URL like `https://<instance-name>-<unique-id>.<region>.run.app` on the instance console page, and you can also view the instance metrics and logs as follows: 97 98  99 100 ### Integration with MLflow authentication 101 102 MLflow supports [basic authentication](https://mlflow.org/docs/latest/self-hosting/security/basic-http-auth/) and [authentication with OIDC plugin](https://mlflow.org/docs/latest/self-hosting/security/sso/#using-oidc-plugin), the 2 kinds of authentication settings require: 103 104 - Additional pip packages: preinstall them in the `mlflow-gcp` image in "Step 1". 105 - Additional environment variable settings: Set them as the environment variables in the container's "Variables & Secrets" settings. 106 - Additional mlflow server CLI options: Append them in the "Container arguments" setting. 107 108 ## Step 6: Validate the deployment 109 110 Use [MLflow demo](https://mlflow.org/docs/latest/api_reference/cli.html#mlflow-demo) CLI to validate the deployment. Run the command from your own laptop as follows: 111 112 ``` 113 mlflow demo --tracking-uri <GCP-cloud-run-application-URL> 114 ``` 115 116 then open the application URL in your browser, view the experiment with name "MLflow Demo", and explore GenAI features like traces, evaluation runs, prompt management etc.