docs: add sprint planning — dependency diagram + specs for all 8 stories

Analyzes all Taiga stories, maps hard/soft dependencies, and writes full specs (Goal, Data Model, Components, Verification, Open Questions) for every story before any implementation begins. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 11:53:24 +02:00
commit 1070ba3ac1
10 changed files with 1239 additions and 0 deletions
@@ -0,0 +1,114 @@
+# Spec: 05 · Import Existing Publications
+
+**Taiga Story:** #5 · `05-import-existing-publications`
+**Status:** Ready
+**Branch:** `feature/05-import-publications`
+**Depends on:** `03-import-fruit-db` (fruits must exist), `04-fruit-publications` (publications tables)
+
+---
+
+## 1. Goal
+
+A standalone Python script imports all publications from `03-data/osws.xml`, their cover images, linked fruits (determined by filesystem scan), PDFs, and fruit images into the database.
+
+---
+
+## 2. Background / Scope / Context / Constraints
+
+- Separate file from the fruit import script — do not merge.
+- Publication identified by XML `<obj>` element where `<osw>` = `1`.
+- Fruit linking is NOT derived from `obstsorten.xml` tags (unreliable subset) but from filesystem scan of `03-data/osdb/{publicationId}/`.
+- Image existence is not guaranteed — skip silently if file not found.
+- `author` field: skip if value is `"."`.
+- Cover image path given by `<img>` tag content (relative to `03-data/`).
+
+### File patterns in `03-data/osdb/{publicationId}/`
+
+| Pattern | Import target |
+|---------|--------------|
+| `{fruitId}_{publicationId}_s0.jpg` | `publication_fruit_images` |
+| `{fruitId}_{publicationId}.pdf` | `publication_descriptions` |
+
+- Collect unique `fruitId` values from both patterns → union = linked fruit set.
+- Look up each `fruitId` against `fruits.osdb_number` → skip if no match.
+- Link matched fruits via `publication_fruits`.
+
+---
+
+## 3. Necessary Refactorings
+
+- Script must run AFTER `import_fruits.py` (fruits table populated).
+- Idempotent: use `INSERT … ON CONFLICT (osdb_pub_id) DO UPDATE` for publications; delete cascade clears linked data before re-import if re-run.
+
+---
+
+## 4. Related Data Model
+
+Tables written (all created by story #4 migration):
+- `publications`
+- `publication_fruits`
+- `publication_descriptions`
+- `publication_fruit_images`
+
+---
+
+## 5. Affected Places / Related Components
+
+```
+scripts/
+  import_fruits.py          ← existing (story #3)
+  import_publications.py    ← new
+  README_import.md          ← update with run order and env var requirements
+```
+
+### Script: `scripts/import_publications.py`
+
+Steps:
+1. Parse `03-data/osws.xml`; iterate `<obj>` where `<osw>` = `1`
+2. For each publication:
+   a. Extract id, name, author (skip if `"."`)
+   b. Resolve cover image: `03-data/{img_tag_value}` — read bytes; skip if missing
+   c. `UPSERT` into `publications` (conflict on `osdb_pub_id`)
+   d. Scan `03-data/osdb/{id}/` for `*_s0.jpg` and `*.pdf` files
+   e. Collect unique fruit IDs from filenames
+   f. Resolve fruit IDs → `fruits.id` via `osdb_number`; skip unresolved
+   g. Delete existing `publication_fruits` for this pub; re-insert linked fruits
+   h. Delete existing `publication_descriptions` for this pub; import PDFs
+   i. Delete existing `publication_fruit_images` for this pub; import images
+3. Log: publications upserted, cover images loaded, fruits linked, PDFs imported, images imported, skips
+
+DB from `DATABASE_URL` env var.
+
+---
+
+## 6. Out of Scope
+
+- Modifying publication CRUD API
+- GUI import trigger
+- Fruit import (handled by script #3)
+
+---
+
+## 7. Verification
+
+| Check | Expected |
+|-------|---------|
+| Script runs to completion | Exit 0, logs summary counts |
+| Re-run is idempotent | Same final row counts |
+| Publication with `osw != 1` | Not imported |
+| Author `"."` | Stored as NULL |
+| Cover image file missing | Skipped, publication still imported |
+| Fruit ID from filename not in fruits table | Skipped with log warning |
+| `GET /api/v1/publications` after import | Returns populated list |
+| Spot-check: known publication | Correct title, linked fruits visible |
+
+---
+
+## 8. Open Questions
+
+| # | Question | Impact |
+|---|----------|--------|
+| 1 | Exact XML element structure in `osws.xml`? (tag names, nesting) | Parser field mapping |
+| 2 | Can `03-data/osdb/{id}/` be missing entirely for some publications? | Directory scan guard |
+| 3 | Re-import strategy: delete-then-insert vs upsert per file? | Idempotency |
+| 4 | Encoding of XML file (UTF-8 or Latin-1)? | `ElementTree` `encoding` param |