wimby/tubesync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: record TubeSync storage and migration state
All checks were successful
Build TubeSync image / container (push) Successful in 53s
Details

Browse Source
This commit is contained in:
wimby
2026-09-29 00:45:27 +02:00
parent 888b7e4426
commit a48a7cb4f6
1 changed files with 10 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
TODO.md
View File

@@ -4,37 +4,37 @@ These tasks track the operational fixes identified while processing the large Tu
## Database and queue storage
- [ ] Migrate the TubeSync Django database from SQLite to the existing MariaDB service.
- [x] Migrate the TubeSync Django database from SQLite to the existing MariaDB service.
- MariaDB endpoint: `192.168.1.3:3306` from `nc-node1`.
- Current server: MariaDB 11.4.5, `utf8mb4`.
- Create a dedicated `tubesync` database and least-privilege user.
- Production database is `tubesync` with dedicated `tubesync@192.168.1.21` credentials.
- Store the connection string outside Git and set `DATABASE_CONNECTION=mysql://...` in deployment configuration.
- Take a verified SQLite backup before migration.
- Test export/import and Django migrations before switching production.
- Verify row counts and application health after cutover.
- [ ] Move Huey SQLite queue files off NFS onto local ext4 storage on `nc-node1`.
- Keep `/downloads` on NFS.
- Bind-mount a local directory onto `/config/tasks`.
- Stop TubeSync before copying the queue DB/WAL/SHM files.
- Verify all four Huey consumers restart and queue counts are preserved.
- [x] Keep all persistent TubeSync runtime state off `nc-node1` local ext4.
- `/config`, `/config/tasks`, `/config/cache`, backups and secrets stay on `/data` (NFS).
- `/downloads` stays on NFS.
- Deno's transient compiler/module cache uses RAM via `DENO_DIR=/dev/shm/deno`.
- No TubeSync queue/cache/migration data is bind-mounted from local ext4.
## Backlog cleanup
- [ ] Collapse duplicate `download_media_metadata` jobs conservatively.
- [x] Collapse duplicate `download_media_metadata` jobs conservatively.
- Current snapshot: ~19.9k metadata jobs represent only ~2.2k unique media IDs.
- Keep at most one metadata job per media ID, preferring the highest-priority/latest viable task.
- Back up the Huey limited-queue DB before modifying it.
- Validate queue counts and sample task payloads before and after cleanup.
- [ ] Prevent the metadata queue from accumulating duplicate work again.
- [x] Prevent the metadata queue from accumulating duplicate work again.
- Review `remove_duplicates` behavior for metadata scheduling.
- Add a regression test covering repeated scheduling for the same media ID.
- Prefer deduplication at enqueue time rather than waiting until execution.
## YouTube challenge handling
- [ ] Start `bgutil-ytdlp-pot-provider` and `yt-cipher` automatically with TubeSync.
- [x] Start `bgutil-ytdlp-pot-provider` and `yt-cipher` automatically with TubeSync.
- Both services currently exist in s6 but are not part of the active `user` bundle.
- Add the services to the appropriate s6 bundle/dependency graph in the fork.
- Add a container-level health/smoke check for their local endpoints.