Head to head.
Compare up to three provider runs using the same queries.
October 1, 2026 · v3.3.2
Selections and main filters are saved in the page URL for sharing.
Lumiere before & after Same 28 queries · two separate runs
Both runs include metadata enrichment. Different run times and evolving metadata caches prevent attributing timing differences solely to the Lumiere update. The previous build and database snapshot were not recorded. The updated build is identified by its image revision, not the mutable “latest” tag.
| Media | Original matches | Updated matches | Original median | Updated median | HTTP attempts · old / new |
|---|
Matches use the selected cutoff; medians and HTTP counts include all 14 searches per media type. This table always shows both media types and ignores category, timing and explorer filters. Other providers were not rerun.
Rerun report and version evidence ↗ · Updated detailed results ↓ · Updated query CSV ↓
Exploratory Stremio results; not measured Odin performance. Method & limitations ↗ Click a chart to inspect its data.
What kinds of searches were tested? 9 categories · coverage and examples
The same 28 queries ran on every provider: 14 movies and 14 shows. Counts below are distinct queries per provider, not 196 different titles. Variants can refer to the same title.
| Type | Movies / shows | What it checks · examples actually tested |
|---|
Year queries included both Blade Runner 2049 formats, Dark 2017, and Doctor Who (1963). Neither Goosebumps (1995) nor Goosebumps (2023) was tested. A proper paired test should search “Goosebumps”, both year-qualified titles, and each year with and without parentheses, then verify which version appears. That would be a new test, not a result shown here.
Find the right title
Target found / eligible searches, within your chosen position cutoff. Click a bar for matches and misses.
What does “top 10” mean?
This scores whether the expected title appears at position 10 or earlier. It does not limit collected results. “Anywhere in returned results” checks the entire saved first response, not the provider’s full database. Result limits differ between providers; no extra pages were fetched.
Time to a complete response
Median elapsed time, including enrichment. This selector affects only this chart and the timing dots.
Measured in milliseconds. Each bar shows its sample count. “Same successful queries” uses only queries where all displayed runs found the target within the selected cutoff.
Across search categories
Matches / eligible queries. Uses media and cutoff filters; always shows all categories. Click a cell for its queries.
Some cells contain just one query. Small differences are not conclusive.
Every response, not just the median
One dot per search. The vertical tick marks the median; all providers share one time scale.
Uses the timing population selected above. Click a dot to inspect its query. Keyboard: Tab to a dot, then Enter or Space.
Requests & metadata details Explore downstream work and returned fields
Where requests go
Outbound HTTP attempts by destination, including enrichment.
Counts, not cost. Click a colored segment for contributing queries. Uses media/category filters, not the timing selector.
Fields included in the response
Click a percentage to see entries with and without that field. Uses media/category filters, not the timing selector.
Counts include repeated titles across queries. A poster field can contain a fallback URL; a description field can contain attribution text.
Explore every query
Compare the same query across providers. Select a result to see the returned titles.
Supplemental scoring rules ↗ · Franchise counts respect the selected cutoff; use “Anywhere” for full-response coverage.
Methodology & caveats
Original-run methodology below. Both Lumiere runs are shown separately; see rerun conditions and version evidence.
Read the full report ↗Download detailed data ↓
What was measured?
168 catalog searches: the same 14 movie and 14 show queries for each provider. AIOMetadata’s Stremio search route, normal metadata enrichment, and complete responses were measured. This is an independent community test. Odin’s Jellyfin route requests light searches and merges enabled catalogs, so it can return different timings and rankings. Phone/network delay, client rendering, and image downloads were not measured.
How are accuracy and missing IDs handled?
Each media type has 11 eligible title queries. Alias, franchise, and negative cases have supplemental reviews in the explorer, using their own stated rules, and remain excluded from headline title accuracy. Matching uses expected IMDb IDs. TVDB has two show cases with unmapped IDs in the top five; confirmed-hit scores are lower bounds where marked with an asterisk. A result at rank 9 is a miss at top 5, but a hit at top 10. This measures target discovery, not the relevance of every returned title.
Were searches cold? What changed during the run?
Providers had isolated caches and equal four-query pilots. Every main query executed a provider lookup; metadata caches evolved naturally. These were not fully cold searches. All six test apps restarted after 25 queries per provider with disk/Redis retained; the last three queries per provider form a separate restart cohort. Scheduled background jobs were disabled in test copies. Production settings and caches were unchanged.
Can I remove trailers or enrichment from the timings?
No trailer entries or addon-trailer calls were observed. Stage spans overlap and instrumentation does not cover every enrichment step. You cannot subtract stage times to estimate a feature-disabled response. The timing selector changes which observed searches are included; it does not simulate disabling features. “Same successful queries” compares identical cases across all six providers, but excludes harder queries where any selected run missed the target.
How reliable are these results?
This is one block of a proposed 1,680-search corpus, with a single timing per query. It is exploratory, not a universal ranking. Four Lumiere responses matched the original search code byte-for-byte after canonical JSON hashing; that does not establish zero instrumentation overhead for every provider. All 168 searches returned HTTP 200, with two downstream TMDB 404s. Repeated runs and larger category samples would strengthen conclusions.
What was the total request budget?
1,295 observed outbound HTTP attempts: 809 main-search requests, 252 pilot/setup requests, 36 validation requests, and 198 startup/background requests. These include enrichment and dataset checks, not just billable search calls. There were 204 isolated catalog attempts, including pilot correction and validation. Trakt was excluded. Temporary test containers and private configuration copies were removed afterward.