sci-fi-analysis

Sci-Fi Lineage Tree

A directed graph of science fiction history — works, authors, and editors as nodes, documented influence as edges — with a contested-"firsts" policy: when a crown is disputed (Was the first time machine 1881 or 1895? Who coined "cyberspace"?), the graph keeps both claimants and marks the edge as contested instead of picking a winner.

The dataset is the single source of truth; both visualizations are generated from it.

Layout

data/scifi_lineage.json        # THE dataset — nodes, edges, eras, schema
data/scifi_lineage_nodes.csv   # generated from the JSON (scripts/export_csv.py)
data/scifi_lineage_edges.csv   # generated from the JSON
research/                      # research brief, fact-check report, expansion notes
web/                           # interactive D3 visualization (static, no build)
manim/scifi_lineage.py         # Manim scenes for the video
scripts/                       # dataset tooling

The data model

• Nodes: works / authors / editors / films / terms, each with year, era (1 = Proto & Foundational, 2 = Golden Age, 3 = New Wave → Modern), tags (first:*, coined:*, award:*, hub, contested-first, math-hook), and a one-line "why it matters".
• Edges: typed influence links — documented (there's a receipt: a review, a letter, an acknowledgment), influence / style / root (tradition or thematic), adaptation, coined, editor, published, response (a rebuttal, e.g. Starship Troopers → The Forever War), and feud (claimed and denied, e.g. We → Brave New World). Edges whose status is contested are flagged disputed: true and drawn dashed/red.

After editing the JSON, regenerate the CSVs:

python3 scripts/export_csv.py

Web visualization

Static site, no build step. D3 v7.9.0 is vendored at web/vendor/d3.min.js (no CDN dependency). Serve the repo root and open web/:

python3 -m http.server 8000
# → http://localhost:8000/web/

Features: timeline layout (x = year, piecewise scale), era color coding, node size by out/in-degree, hover tooltips, click-to-isolate a lineage (ego network), era/spotlight/search filters, and a "hide thematic/hedged" toggle that leaves only hard-documented edges.

Video (Manim)

Requires the 3blue1brown-style Manim Community toolchain (ffmpeg, cairo, pango, pkg-config via Homebrew on macOS):

python3.12 -m venv .venv
.venv/bin/pip install manim

.venv/bin/manim -ql manim/scifi_lineage.py LineageBuild      # quick preview
.venv/bin/manim -qh manim/scifi_lineage.py LineageBuild      # 1080p
.venv/bin/manim -qh manim/scifi_lineage.py ContestedBeats    # the five contested-crown cards

LineageBuild animates the tree growing node-by-node in chronological order (video hooks + hubs + their connectors) with a running year counter; ContestedBeats renders the five "contested crown" title cards.

Images & book links

data/images.json maps each node to cover art, an author portrait, and a book page link — resolved from Open Library and Wikipedia (both allow hotlinking, so the repo redistributes no copyrighted artwork):

python3 scripts/fetch_images.py             # resolve urls (re-runs fill gaps)
python3 scripts/fetch_images.py --download  # also cache copies into web/img/

Covers show in tooltips, and the detail panel shows cover + portrait + a "the book ↗" link. Every node is deep-linkable: web/index.html#node=<id>.

Video segments (narrative + links)

scripts/build_segments.py turns the dataset into the story structure for the short — cold open (contested crowns), the three eras, outro — and writes:

• data/segments.json — structured segments with node ids and book links
• video/segments.md — paste-ready video description: chapter placeholders with every book mentioned in that segment, linked

python3 scripts/build_segments.py

Influence analysis & themes

scripts/analyze.py computes a small battery of explainable influence metrics per node and writes data/analysis.json:

• cascade — descendant count with distance decay (λ=0.5): the headline "how much of the genre descends from this work" number
• pagerank — PageRank on reversed edges: prestige from influential descendants
• generations — longest descendant chain (influence depth)
• betweenness — bridge works between movements (ranks only, it's unstable on a sparse near-tree)
• influence — equal-weight mean of the four percentile ranks
• era_fair — cascade percentile within the node's own era, so ancient works don't win just for being ancient

data/themes.json adds the theme layer: 25 themes assigned to every node, each with an introduced_by / popularized_by pair, plus convergence works where 3+ theme streams meet. The analysis reports per-theme spread, momentum by period, and fidelity — how often a theme actually travels along lineage edges vs. being reinvented. Methodology, caveats, and the v2 external-signals plan (Open Library editions, Wikidata adaptations, OpenAlex, pageviews): research/influence_methodology.md.

python3 scripts/analyze.py

In the web viz, node size encodes influence, the detail panel shows the metric breakdown and theme chips, and the theme dropdown filters the graph to a single motif's spread.

Semantic clustering (summaries → embeddings → map)

A second, content-based view of the graph — independent of who-influenced-whom:

python3 scripts/fetch_summaries.py            # Wikipedia extracts / Open Library blurbs
.venv/bin/pip install sentence-transformers scikit-learn
.venv/bin/python scripts/embed_cluster.py     # embed → cluster → data/clusters.json

embed_cluster.py embeds each work's summary (all-MiniLM-L6-v2), picks the cluster count by silhouette, projects to 2D with PCA, and reports alignment: NMI between clusters and the curated themes, per-theme cluster concentration, and edge coherence (what fraction of lineage edges stay within one semantic cluster vs. share a theme vs. stay in one era). In the web viz, switch View → semantic map: nodes are placed by content similarity and colored by cluster; tooltips and the detail panel show the cluster's signature words.

Contributing

Additions and corrections welcome — the bar for an edge is a receipt or a flag. If an influence is documented (an author acknowledged it, a review exists), it's a solid edge. If it's tradition or vibes, it's a hedged edge and the note says so. Contested "firsts" keep all claimants.

A Math vs Vibes project — the dataset and both visualizations were built for a Math vs Vibes short about the lineage of science fiction.