.. _tutor_operations: Tutor plugin: what it deploys and operations ============================================ .. contents:: On this page :local: :depth: 2 This page describes what the plugin actually builds, injects, and schedules, so you can operate it and troubleshoot. Most behavior is gated by :ref:`PANORAMA_MODE `. Images it builds ---------------- The plugin registers two buildable Docker images: * ``panorama-elt`` — the :ref:`extraction agent ` (``PANORAMA_DOCKER_IMAGE``). * ``panorama-elt-logs`` — a Fluent Bit image for shipping tracking logs (``PANORAMA_LOGS_DOCKER_IMAGE``). They are built, pulled, and pushed with the usual Tutor commands, for example ``tutor images build panorama-elt``. It also installs ``panorama-openedx-backend`` into the ``openedx`` image (only when ``PANORAMA_MFE_ENABLED`` is true), which is why you rebuild ``openedx`` after enabling the plugin. Settings it injects into the LMS -------------------------------- The plugin patches the LMS common settings with the values the :ref:`backend ` reads: * ``PANORAMA_MODE``, ``PANORAMA_ENABLE_STUDENT_VIEW``, ``PANORAMA_DEFAULT_USER_ARN``; * ``PANORAMA_AWS_ACCESS_KEY``, ``PANORAMA_AWS_SECRET_ACCESS_KEY``, ``PANORAMA_REGION``, ``PANORAMA_AWS_ACCOUNT_ID``. It also sets ``MFE_CONFIG["PANORAMA_URL"]`` so the platform knows where the MFE lives, and registers the :ref:`Panorama MFE ` (repository, version, port ``2100``) served at ``/panorama/``. The learner-dashboard link ~~~~~~~~~~~~~~~~~~~~~~~~~~~ **On the Sumac (19.x) line**, when ``PANORAMA_ADD_DASHBOARD_LINK`` is true, the plugin swaps the ``learner-dashboard`` MFE for an Aulasneo fork that adds a direct link to Panorama. .. warning:: If you already run a **custom** learner-dashboard MFE, enabling this overrides it (the plugin prints an alert when it detects a non-default repository). Leave it ``False`` if you maintain your own learner dashboard. .. note:: On the **Teak (21.x+)** line this variable and the learner-dashboard fork were removed. The plugin registers the Panorama MFE directly and does not modify the learner dashboard, so there is nothing to configure here. Initialization tasks -------------------- On ``tutor {local|k8s} do init`` (run during ``launch`` / ``quickstart``), the plugin runs: * **ELT init** (``panorama-elt`` service), skipped in ``DEMO`` mode: * ``test-connections`` — verify it can reach the data sources and data lake. * In ``CUSTOM`` mode only: ``create-datalake-tables --all`` then ``create-table-views --all`` (so you do not run the :ref:`first run ` by hand). * **Backend migrations** (``lms`` service), only in ``SAAS`` and ``CUSTOM`` modes: it runs the Django migrations for ``panorama_openedx_backend``. Periodic extraction ------------------- Extraction is scheduled from the ``PANORAMA_CRONTAB`` value (default ``55 * * * *``, i.e. hourly). It is skipped entirely in ``DEMO`` mode. * **Local / Docker Compose.** A ``panorama`` service runs a cron container that periodically executes ``panorama.py ... extract-and-load --all``. * **Kubernetes.** A ``CronJob`` named ``panorama-elt`` runs the same extraction on the schedule. .. note:: The always-present ``panorama-job`` Job (Kubernetes) is what backs the manual ``extract-and-load`` command described below; the scheduled ``CronJob`` is what runs extraction automatically. Tracking-log shipping --------------------- Tracking logs are shipped to the data lake by Fluent Bit (the ``panorama-elt-logs`` image), which enables event-based analytics. This is **not** run in ``DEMO`` or ``FREE`` mode (FREE extracts only relational and courseware data): * **Local.** A ``panorama_logs`` Fluent Bit service tails the LMS logs and uploads them, using ``PANORAMA_FLB_LOG_LEVEL``, ``PANORAMA_LOGS_TOTAL_FILE_SIZE``, and ``PANORAMA_LOGS_UPLOAD_TIMEOUT``. * **Kubernetes.** A Fluent Bit DaemonSet ships node/pod logs, gated by ``PANORAMA_RUN_K8S_FLUENTBIT`` and only in ``SAAS`` / ``CUSTOM`` modes. .. important:: On Kubernetes, run the Fluent Bit DaemonSet in **only one** namespace. If you have multiple Open edX namespaces, set ``PANORAMA_RUN_K8S_FLUENTBIT: false`` in all but one to avoid duplicate log collection. Running extraction on demand ---------------------------- The plugin adds an ``extract-and-load`` do-command so you can trigger extraction manually (useful for a first full load, or after config changes): .. code-block:: shell tutor local do extract-and-load --all tutor local do extract-and-load --tables student_courseenrollment,auth_user tutor local do extract-and-load --all --force The options mirror the ELT agent: ``--all`` / ``-a``, ``--tables`` / ``-t``, ``--force``, and ``--debug``. See :ref:`elt_commands` for what they do. On Kubernetes use ``tutor k8s do extract-and-load ...``. .. tip:: For a first full load of partitioned tables in CUSTOM mode, run the do-command once with ``--force`` after ``init`` has created the tables and views. Troubleshooting pointers ------------------------ * **Nothing is extracted.** Confirm ``PANORAMA_MODE`` is not ``DEMO`` and that the crontab / CronJob exists. Check ``test-connections`` output from the init task. * **Dashboards do not load.** The backend requires HTTPS and rejects local/dev hosts in DEMO mode; see :ref:`backend_configuration`. Verify the MFE is served at ``/panorama/`` and ``MFE_CONFIG["PANORAMA_URL"]`` is set. * **Access denied on AWS.** Recheck the IAM policy **and** Lake Formation grants (:ref:`tutor_configuration`). * **Pods OOM-killed (Kubernetes).** Set ``PANORAMA_K8S_JOB_MEMORY_REQUEST`` / ``PANORAMA_K8S_JOB_MEMORY_LIMIT``. `Back to Home `_