# Spec: 05 · Import Existing Publications

**Taiga Story:** #5 · `05-import-existing-publications`
**Status:** Ready
**Branch:** `feature/05-import-publications`
**Depends on:** `03-import-fruit-db` (fruits must exist), `04-fruit-publications` (publications tables)

---

## 1. Goal

A standalone Python script imports all publications from `03-data/osws.xml`, their cover images, linked fruits (determined by filesystem scan), PDFs, and fruit images into the database.

---

## 2. Background / Scope / Context / Constraints

- Separate file from the fruit import script — do not merge.
- Publication identified by XML `<obj>` element where `<osw>` = `1`.
- Fruit linking is NOT derived from `obstsorten.xml` tags (unreliable subset) but from filesystem scan of `03-data/osdb/{publicationId}/`.
- Image existence is not guaranteed — skip silently if file not found.
- `author` field: skip if value is `"."`.
- Cover image path given by `<img>` tag content (relative to `03-data/`).

### File patterns in `03-data/osdb/{publicationId}/`

| Pattern | Import target |
|---------|--------------|
| `{fruitId}_{publicationId}_s0.jpg` | `publication_fruit_images` |
| `{fruitId}_{publicationId}.pdf` | `publication_descriptions` |

- Collect unique `fruitId` values from both patterns → union = linked fruit set.
- Look up each `fruitId` against `fruits.osdb_number` → skip if no match.
- Link matched fruits via `publication_fruits`.

---

## 3. Necessary Refactorings

- Script must run AFTER `import_fruits.py` (fruits table populated).
- Idempotent: use `INSERT … ON CONFLICT (osdb_pub_id) DO UPDATE` for publications; delete cascade clears linked data before re-import if re-run.

---

## 4. Related Data Model

Tables written (all created by story #4 migration):
- `publications`
- `publication_fruits`
- `publication_descriptions`
- `publication_fruit_images`

---

## 5. Affected Places / Related Components

```
scripts/
  import_fruits.py          ← existing (story #3)
  import_publications.py    ← new
  README_import.md          ← update with run order and env var requirements
```

### Script: `scripts/import_publications.py`

Steps:
1. Parse `03-data/osws.xml`; iterate `<obj>` where `<osw>` = `1`
2. For each publication:
   a. Extract id, name, author (skip if `"."`)
   b. Resolve cover image: `03-data/{img_tag_value}` — read bytes; skip if missing
   c. `UPSERT` into `publications` (conflict on `osdb_pub_id`)
   d. Scan `03-data/osdb/{id}/` for `*_s0.jpg` and `*.pdf` files
   e. Collect unique fruit IDs from filenames
   f. Resolve fruit IDs → `fruits.id` via `osdb_number`; skip unresolved
   g. Delete existing `publication_fruits` for this pub; re-insert linked fruits
   h. Delete existing `publication_descriptions` for this pub; import PDFs
   i. Delete existing `publication_fruit_images` for this pub; import images
3. Log: publications upserted, cover images loaded, fruits linked, PDFs imported, images imported, skips

DB from `DATABASE_URL` env var.

---

## 6. Out of Scope

- Modifying publication CRUD API
- GUI import trigger
- Fruit import (handled by script #3)

---

## 7. Verification

| Check | Expected |
|-------|---------|
| Script runs to completion | Exit 0, logs summary counts |
| Re-run is idempotent | Same final row counts |
| Publication with `osw != 1` | Not imported |
| Author `"."` | Stored as NULL |
| Cover image file missing | Skipped, publication still imported |
| Fruit ID from filename not in fruits table | Skipped with log warning |
| `GET /api/v1/publications` after import | Returns populated list |
| Spot-check: known publication | Correct title, linked fruits visible |

---

## 8. Open Questions

| # | Question | Impact |
|---|----------|--------|
| 1 | Exact XML element structure in `osws.xml`? (tag names, nesting) | Parser field mapping |
| 2 | Can `03-data/osdb/{id}/` be missing entirely for some publications? | Directory scan guard |
| 3 | Re-import strategy: delete-then-insert vs upsert per file? | Idempotency |
| 4 | Encoding of XML file (UTF-8 or Latin-1)? | `ElementTree` `encoding` param |