workspace_tutorial.ipynb
1 { 2 "cells": [ 3 { 4 "cell_type": "markdown", 5 "id": "ee4d750ab76dea3d", 6 "metadata": {}, 7 "source": "# Evidently UI Tutorial" 8 }, 9 { 10 "cell_type": "markdown", 11 "id": "5aca8ca20d65b7c8", 12 "metadata": {}, 13 "source": [ 14 "## Working with Local Workspace\n", 15 "\n", 16 "First, let’s create a local workspace where all our Evidently projects and reports will be stored. This is essentially a directory to keep things organized.\n" 17 ] 18 }, 19 { 20 "cell_type": "code", 21 "id": "3969c5244c3176df", 22 "metadata": {}, 23 "source": [ 24 "from evidently.ui.workspace import Workspace\n", 25 "\n", 26 "ws = Workspace.create(\"workspace\")" 27 ], 28 "outputs": [], 29 "execution_count": null 30 }, 31 { 32 "cell_type": "markdown", 33 "id": "6f0262924aa126b2", 34 "metadata": {}, 35 "source": [ 36 "### Create a New Project\n", 37 "\n", 38 "Now, inside our workspace, we’ll create a new project. Projects in Evidently are containers for organizing related reports and monitoring runs.\n" 39 ] 40 }, 41 { 42 "cell_type": "code", 43 "id": "9e3d7284e749796d", 44 "metadata": {}, 45 "source": [ 46 "project = ws.create_project(\"My Project\")\n", 47 "project" 48 ], 49 "outputs": [], 50 "execution_count": null 51 }, 52 { 53 "cell_type": "markdown", 54 "id": "e975a32f58cc4b6d", 55 "metadata": {}, 56 "source": [ 57 "this creates project dir in `workspace/{project.id}`\n", 58 "\n", 59 "### Working with projects\n", 60 "\n", 61 "`Workspace` has some methods to work with projects.\n", 62 "\n", 63 "For example, we can edit project to add a description to clarify projects purpose or context.\n" 64 ] 65 }, 66 { 67 "cell_type": "code", 68 "id": "b5bc3f79706e62f3", 69 "metadata": {}, 70 "source": [ 71 "project.description = \"Evidently Service example project\"\n", 72 "ws.update_project(project) # or project.save()" 73 ], 74 "outputs": [], 75 "execution_count": null 76 }, 77 { 78 "metadata": {}, 79 "cell_type": "markdown", 80 "source": [ 81 "### List All Projects\n", 82 "\n", 83 "To see what projects are currently stored in the workspace, we can list them like this.\n" 84 ], 85 "id": "e64fa272b4dab4f6" 86 }, 87 { 88 "cell_type": "code", 89 "id": "652e8ba4da59f4f2", 90 "metadata": {}, 91 "source": [ 92 "ws.list_projects()" 93 ], 94 "outputs": [], 95 "execution_count": null 96 }, 97 { 98 "metadata": {}, 99 "cell_type": "markdown", 100 "source": [ 101 "### Search for a Project by Name\n", 102 "\n", 103 "If we need to locate a specific project, we can search for it by name.\n" 104 ], 105 "id": "b759a33be9558a01" 106 }, 107 { 108 "cell_type": "code", 109 "id": "94e016b377c7c27e", 110 "metadata": {}, 111 "source": [ 112 "ws.search_project(\"My Project\")" 113 ], 114 "outputs": [], 115 "execution_count": null 116 }, 117 { 118 "metadata": {}, 119 "cell_type": "markdown", 120 "source": [ 121 "### Delete a Project\n", 122 "\n", 123 "If a project is no longer needed, we can delete it by its project ID.\n" 124 ], 125 "id": "4886aa110042e3c0" 126 }, 127 { 128 "cell_type": "code", 129 "id": "a6c21ae4feee79a3", 130 "metadata": {}, 131 "source": [ 132 "ws.delete_project(project.id)" 133 ], 134 "outputs": [], 135 "execution_count": null 136 }, 137 { 138 "metadata": {}, 139 "cell_type": "markdown", 140 "source": [ 141 "### Create a Project Using ProjectModel\n", 142 "\n", 143 "Alternatively, projects can be created by defining a `ProjectModel` with specific properties and adding it to the workspace. This approach can be useful for more programmatic setups.\n" 144 ], 145 "id": "75ecc0a682df83d0" 146 }, 147 { 148 "cell_type": "code", 149 "id": "f841e0573e99e660", 150 "metadata": {}, 151 "source": [ 152 "from evidently.ui.workspace import ProjectModel\n", 153 "\n", 154 "project = ws.add_project(ProjectModel(name=\"My Project\", description=\"Evidently Service example project\"))\n", 155 "project" 156 ], 157 "outputs": [], 158 "execution_count": null 159 }, 160 { 161 "cell_type": "markdown", 162 "id": "6c94bbf31faf3355", 163 "metadata": {}, 164 "source": [ 165 "## Working with Reports and Runs\n", 166 "\n", 167 "Next, let’s create a simple Evidently report. We’ll define a metric to check the minimum value of a specific column, and include a test to assert that this value should be greater than 5.\n" 168 ] 169 }, 170 { 171 "cell_type": "code", 172 "id": "770322c8ee0ab980", 173 "metadata": {}, 174 "source": [ 175 "from evidently.tests import gt\n", 176 "from evidently import Report\n", 177 "from evidently.metrics import MinValue\n", 178 "\n", 179 "report = Report(metrics=[MinValue(column=\"col\", tests=[gt(5)])])\n", 180 "report" 181 ], 182 "outputs": [], 183 "execution_count": null 184 }, 185 { 186 "metadata": {}, 187 "cell_type": "markdown", 188 "source": "Let’s generate some example data and run the report on it.", 189 "id": "fd417dcd0a98b37a" 190 }, 191 { 192 "cell_type": "code", 193 "id": "c72c56cf9fbe8c31", 194 "metadata": {}, 195 "source": [ 196 "import pandas as pd\n", 197 "\n", 198 "data = pd.DataFrame({\"col\": [6, 10, 15]})\n", 199 "run = report.run(data)\n", 200 "run" 201 ], 202 "outputs": [], 203 "execution_count": null 204 }, 205 { 206 "metadata": {}, 207 "cell_type": "markdown", 208 "source": [ 209 "### Add the Run to the Project\n", 210 "\n", 211 "Now we can save this run into our project within the workspace." 212 ], 213 "id": "3915878cbb4a556" 214 }, 215 { 216 "cell_type": "code", 217 "id": "a35570a7b6124121", 218 "metadata": {}, 219 "source": [ 220 "ws.add_run(project.id, run)" 221 ], 222 "outputs": [], 223 "execution_count": null 224 }, 225 { 226 "metadata": {}, 227 "cell_type": "markdown", 228 "source": "To view all runs associated with this project, we can list them.\n", 229 "id": "1eb9bb430201b00d" 230 }, 231 { 232 "cell_type": "code", 233 "id": "b6a4254f596ad5e6", 234 "metadata": {}, 235 "source": [ 236 "runs = ws.list_runs(project.id)\n", 237 "runs" 238 ], 239 "outputs": [], 240 "execution_count": null 241 }, 242 { 243 "metadata": {}, 244 "cell_type": "markdown", 245 "source": "If we need to retrieve a saved run and inspect its contents, we can load it like this.\n", 246 "id": "6166983c73acae08" 247 }, 248 { 249 "cell_type": "code", 250 "id": "f481c66d0b1a1df3", 251 "metadata": {}, 252 "source": [ 253 "from evidently.core.report import Snapshot\n", 254 "\n", 255 "Snapshot.load_model(ws.get_run(project.id, runs[0]))" 256 ], 257 "outputs": [], 258 "execution_count": null 259 }, 260 { 261 "metadata": {}, 262 "cell_type": "markdown", 263 "source": "Similarly, we can delete a specific run by its ID if it's no longer needed.\n", 264 "id": "355e4106d126794e" 265 }, 266 { 267 "cell_type": "code", 268 "id": "b2ac389002e6b844", 269 "metadata": {}, 270 "source": [ 271 "ws.delete_run(project.id, runs[0])" 272 ], 273 "outputs": [], 274 "execution_count": null 275 }, 276 { 277 "metadata": {}, 278 "cell_type": "markdown", 279 "source": "To simulate multiple monitoring runs over time, we’ll generate several runs with slightly different random values and assign each a different timestamp.\n", 280 "id": "f96fd6cb83b181a0" 281 }, 282 { 283 "cell_type": "code", 284 "id": "d32830ff5332f0b5", 285 "metadata": {}, 286 "source": [ 287 "import random\n", 288 "import datetime\n", 289 "\n", 290 "for i in range(1, 5):\n", 291 " data = pd.DataFrame({\"col\": [i + random.randint(-2, 2) for _ in range(3)]})\n", 292 " run = report.run(data, timestamp=datetime.datetime.now() + datetime.timedelta(days=-2 + i))\n", 293 " ws.add_run(project.id, run)" 294 ], 295 "outputs": [], 296 "execution_count": null 297 }, 298 { 299 "metadata": {}, 300 "cell_type": "markdown", 301 "source": "Let’s list them to confirm they’re stored correctly.", 302 "id": "942c80edeb273c8e" 303 }, 304 { 305 "cell_type": "code", 306 "id": "898944ef6b322267", 307 "metadata": {}, 308 "source": [ 309 "ws.list_runs(project.id)" 310 ], 311 "outputs": [], 312 "execution_count": null 313 }, 314 { 315 "cell_type": "markdown", 316 "id": "ab395b13e4da2325", 317 "metadata": {}, 318 "source": [ 319 "### Start the Evidently UI Service\n", 320 "\n", 321 "To explore our projects and reports in a web interface, you normally need to start the Evidently UI service from your command line like this:\n", 322 "\n", 323 "```bash\n", 324 "evidently ui\n", 325 "```\n", 326 "\n", 327 "This will run the service at [http://127.0.0.1:8000](http://127.0.0.1:8000).\n", 328 "\n", 329 "For convenience, in the next cell we'll demonstrate how to launch the service directly from the notebook using a Python subprocess.\n", 330 "**Note:** This approach is handy for quick experiments, but it's generally recommended to run the service from your terminal in production or more persistent setups.\n" 331 ] 332 }, 333 { 334 "cell_type": "code", 335 "id": "2b236ab332eab8cc", 336 "metadata": {}, 337 "source": [ 338 "import subprocess\n", 339 "\n", 340 "# Start Evidently UI as a background process\n", 341 "process = subprocess.Popen([\"evidently\", \"ui\"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n", 342 "\n", 343 "# Store the process if you want to stop it later:\n", 344 "# process.terminate()" 345 ], 346 "outputs": [], 347 "execution_count": null 348 }, 349 { 350 "cell_type": "markdown", 351 "id": "219fd058999b10ff", 352 "metadata": {}, 353 "source": "Now you can open [http://127.0.0.1:8000](http://127.0.0.1:8000) in your browser to explore your workspace and project.\n" 354 }, 355 { 356 "cell_type": "markdown", 357 "id": "28c9d57863c58041", 358 "metadata": {}, 359 "source": [ 360 "### Working with RemoteWorkspace\n", 361 "\n", 362 "The workflow is the same for a remote Evidently UI instance.\n", 363 "Here’s how you’d connect to a service running remotely (or locally in this case) using `RemoteWorkspace`.\n" 364 ] 365 }, 366 { 367 "cell_type": "code", 368 "id": "6636ad7e37a3dd03", 369 "metadata": {}, 370 "source": [ 371 "from evidently.ui.workspace import RemoteWorkspace\n", 372 "\n", 373 "remote_ws = RemoteWorkspace(\"http://127.0.0.1:8000\")\n", 374 "remote_ws" 375 ], 376 "outputs": [], 377 "execution_count": null 378 }, 379 { 380 "metadata": {}, 381 "cell_type": "markdown", 382 "source": "Once connected, we can search for and retrieve a project from the remote UI service.\n", 383 "id": "125340325db3c848" 384 }, 385 { 386 "cell_type": "code", 387 "id": "adad3aa655955d8f", 388 "metadata": {}, 389 "source": [ 390 "remote_project = remote_ws.search_project(\"My Project\")[0]\n", 391 "remote_project" 392 ], 393 "outputs": [], 394 "execution_count": null 395 }, 396 { 397 "cell_type": "markdown", 398 "id": "6c7413ef0dfb97af", 399 "metadata": {}, 400 "source": [ 401 "### Setting up dashboard\n", 402 "\n", 403 "Finally, we can set up a dashboard panel to visualize metrics from our runs.\n", 404 "In this example, we’ll create a line plot panel showing the minimum value of the `\"col\"` column across different runs over time.\n" 405 ] 406 }, 407 { 408 "cell_type": "code", 409 "id": "374a7591713c6334", 410 "metadata": {}, 411 "source": [ 412 "from evidently.sdk.models import PanelMetric\n", 413 "from evidently.sdk.panels import line_plot_panel\n", 414 "\n", 415 "remote_project.dashboard.add_panel(\n", 416 " line_plot_panel(\n", 417 " title=\"Minimum value\",\n", 418 " values=[\n", 419 " PanelMetric(\n", 420 " legend=\"minimum value\",\n", 421 " metric=\"MinValue\",\n", 422 " metric_labels={\"column\": \"col\"},\n", 423 " ),\n", 424 " ],\n", 425 " size=\"full\",\n", 426 " )\n", 427 ")" 428 ], 429 "outputs": [], 430 "execution_count": null 431 }, 432 { 433 "cell_type": "markdown", 434 "id": "32a1460b56773e6d", 435 "metadata": {}, 436 "source": [ 437 "Now you can view your dashboard and metrics visualizations through the Evidently UI.\n", 438 "For more information, check the Evidently documentation at [Evidently AI Docs](https://docs.evidentlyai.com).\n" 439 ] 440 } 441 ], 442 "metadata": { 443 "kernelspec": { 444 "display_name": "Python 3 (ipykernel)", 445 "language": "python", 446 "name": "python3" 447 }, 448 "language_info": { 449 "codemirror_mode": { 450 "name": "ipython", 451 "version": 3 452 }, 453 "file_extension": ".py", 454 "mimetype": "text/x-python", 455 "name": "python", 456 "nbconvert_exporter": "python", 457 "pygments_lexer": "ipython3", 458 "version": "3.11.11" 459 } 460 }, 461 "nbformat": 4, 462 "nbformat_minor": 5 463 }