From a48a7cb4f6cfd038489fa2b24bcb30a2b0f6cf21 Mon Sep 17 00:00:00 2001 From: wimby Date: Tue, 29 Sep 2026 00:45:27 +0200 Subject: [PATCH] docs: record TubeSync storage and migration state --- TODO.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/TODO.md b/TODO.md index 202acf84..f8566812 100644 --- a/TODO.md +++ b/TODO.md @@ -4,37 +4,37 @@ These tasks track the operational fixes identified while processing the large Tu ## Database and queue storage -- [ ] Migrate the TubeSync Django database from SQLite to the existing MariaDB service. +- [x] Migrate the TubeSync Django database from SQLite to the existing MariaDB service. - MariaDB endpoint: `192.168.1.3:3306` from `nc-node1`. - Current server: MariaDB 11.4.5, `utf8mb4`. - - Create a dedicated `tubesync` database and least-privilege user. + - Production database is `tubesync` with dedicated `tubesync@192.168.1.21` credentials. - Store the connection string outside Git and set `DATABASE_CONNECTION=mysql://...` in deployment configuration. - Take a verified SQLite backup before migration. - Test export/import and Django migrations before switching production. - Verify row counts and application health after cutover. -- [ ] Move Huey SQLite queue files off NFS onto local ext4 storage on `nc-node1`. - - Keep `/downloads` on NFS. - - Bind-mount a local directory onto `/config/tasks`. - - Stop TubeSync before copying the queue DB/WAL/SHM files. - - Verify all four Huey consumers restart and queue counts are preserved. +- [x] Keep all persistent TubeSync runtime state off `nc-node1` local ext4. + - `/config`, `/config/tasks`, `/config/cache`, backups and secrets stay on `/data` (NFS). + - `/downloads` stays on NFS. + - Deno's transient compiler/module cache uses RAM via `DENO_DIR=/dev/shm/deno`. + - No TubeSync queue/cache/migration data is bind-mounted from local ext4. ## Backlog cleanup -- [ ] Collapse duplicate `download_media_metadata` jobs conservatively. +- [x] Collapse duplicate `download_media_metadata` jobs conservatively. - Current snapshot: ~19.9k metadata jobs represent only ~2.2k unique media IDs. - Keep at most one metadata job per media ID, preferring the highest-priority/latest viable task. - Back up the Huey limited-queue DB before modifying it. - Validate queue counts and sample task payloads before and after cleanup. -- [ ] Prevent the metadata queue from accumulating duplicate work again. +- [x] Prevent the metadata queue from accumulating duplicate work again. - Review `remove_duplicates` behavior for metadata scheduling. - Add a regression test covering repeated scheduling for the same media ID. - Prefer deduplication at enqueue time rather than waiting until execution. ## YouTube challenge handling -- [ ] Start `bgutil-ytdlp-pot-provider` and `yt-cipher` automatically with TubeSync. +- [x] Start `bgutil-ytdlp-pot-provider` and `yt-cipher` automatically with TubeSync. - Both services currently exist in s6 but are not part of the active `user` bundle. - Add the services to the appropriate s6 bundle/dependency graph in the fork. - Add a container-level health/smoke check for their local endpoints.