julia/osdb-1-claude-opusplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: import fruit DB from XML (story #03) #3

Merged
julia merged 2 commits from feature/03-import-fruit-db into feature/02-manage-fruits 2026-06-17 12:46:05 +00:00
Conversation 0 Commits 2 Files Changed 5
+449 -1
julia commented 2026-06-17 11:38:41 +00:00
Owner
Copy Link
Copy Source

Summary

  • Add scripts/import_fruits.py — idempotent bulk-importer reading 03-data/obstsorten.xml, mapping 23 typ codes to 17 fruit_type enum values, splitting comma-separated synonyms, and discovering images across three directories (einzelfruechte, blueten, baume)
  • Add scripts/import_fruits_test.py — 43 unit tests covering map_typ, parse_synonyms, parse_date, and find_images (no DB required)
  • Update Makefile test target to include Python unit tests
  • Gitignore 03-data/ and obstsorten.zip (large binary data, not committed)

Test plan

  • cd scripts && python3 -m unittest import_fruits_test -v — all 43 tests pass
  • make test — all backend, frontend, and script tests pass
  • Manual: DATABASE_URL=... python3 scripts/import_fruits.py imports 3665 fruits with synonyms and images

🤖 Generated with Claude Code

## Summary - Add `scripts/import_fruits.py` — idempotent bulk-importer reading `03-data/obstsorten.xml`, mapping 23 `typ` codes to 17 fruit_type enum values, splitting comma-separated synonyms, and discovering images across three directories (`einzelfruechte`, `blueten`, `baume`) - Add `scripts/import_fruits_test.py` — 43 unit tests covering `map_typ`, `parse_synonyms`, `parse_date`, and `find_images` (no DB required) - Update `Makefile` test target to include Python unit tests - Gitignore `03-data/` and `obstsorten.zip` (large binary data, not committed) ## Test plan - [ ] `cd scripts && python3 -m unittest import_fruits_test -v` — all 43 tests pass - [ ] `make test` — all backend, frontend, and script tests pass - [ ] Manual: `DATABASE_URL=... python3 scripts/import_fruits.py` imports 3665 fruits with synonyms and images 🤖 Generated with [Claude Code](https://claude.com/claude-code)
julia added 2 commits 2026-06-17 11:38:41 +00:00
feat: import fruit DB from XML — scripts/import_fruits.py (story #03) 4d4c5f399f 
Adds a repeatable Python import script that populates fruits, fruit_synonyms,
and fruit_images from 03-data/obstsorten.xml and the three image directories.
Idempotent: deletes synonyms and images per fruit before re-inserting; upserts
fruit row by osdb_number. Reads DATABASE_URL from env.

- map_typ: 23 XML typ codes → 17-value fruit_type enum; aggregate types coerced
  with logged warning; typ='p' (2 entries) mapped to Pfirsiche
- parse_synonyms: comma-split with whitespace/newline strip, empty filter
- parse_date: YYYYMMDD → date, fallback None → DB default NOW()
- find_images: globs {id}_*_s0.* across einzelfruechte/blueten/baume dirs
- 42 unit tests (no DB required) cover all mapping, parsing, and glob logic
- Makefile test target extended to include Python test suite
- 03-data/ and obstsorten.zip added to .gitignore (large binary data)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fix: apply code-review findings to import_fruits script and tests 7fc8204924 
- find_images: verify exact fruit_id prefix (stem-parse before _s0.) to
  prevent cross-fruit image contamination (e.g. mitschurins steals
  mitschurins_fruchtbare images)
- DATA_DIR uses os.path.abspath to handle symlinks / relative invocation
- counts["images"] derived from IMAGE_DIRS constant, not hardcoded keys
- counts["warnings"] incremented on missing-id/name skip (was missing)
- UPSERT preserves existing comment (comment = fruits.comment, not NULL)
- Print summary and rollback moved inside try/except to avoid NameError
- Remove dead XML_PATH constant
- Tests: add tearDown to clean up tmpdir; fix vacuous _tn exclusion test
  to include positive-control _s0 file; add prefix-collision regression test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
julia merged commit 78892bf32d into feature/02-manage-fruits 2026-06-17 12:46:05 +00:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
julia (Julia Weber)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: julia/osdb-1-claude-opusplan#3