GKaszewski/k-photos
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

perf: scale fixes for 1M+ photo libraries

Browse Source 
Indexes: share_targets.target_id, duplicate_groups.status,
GIN on stacks members + duplicate candidates JSONB,
composite (owner_user_id, created_at DESC) on assets.

N+1 elimination: batch metadata loading via find_by_assets(ids)
using WHERE asset_id = ANY($1), used in timeline + sidecar export.

Visibility: cache find_targets_for_user per request via OnceCell,
extract filter_visible helper to reduce duplication.

Streaming: FileStoragePort.open_file() returns (DataStream, u64),
LocalFileStorage uses ReaderStream instead of loading full file.
serve_file/serve_derivative use Body::from_stream().

Unbounded queries: sidecar full_export/import batched in 500-row
chunks instead of u32::MAX. find_unresolved paginated with
limit/offset. list_duplicates API accepts pagination params.
This commit is contained in:
Gabriel Kaszewski
2026-05-31 22:40:25 +02:00
parent d879fd6437
commit bcaf49cc81
21 changed files with 263 additions and 123 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
Cargo.lock generated
View File

@@ -100,6 +100,7 @@ dependencies = [
"futures",
"object_store",
"tokio",
"tokio-util",
"tracing",
]
@@ -188,6 +189,7 @@ dependencies = [
"bytes",
"chrono",
"domain",
"futures",
"serde",
"serde_json",
"sha2",