Epic Games Open-Sourced Lore — A Version Control System Built for Massive Game Assets

rust dev.to

Epic Games just dropped something that could reshape how game studios handle code and assets. They've open-sourced Lore — a centralized version control system built from the ground up to solve one painful problem: managing enormous binary files alongside source code.

This isn't another Git wrapper. It's a completely new VCS, written in Rust, MIT-licensed, and battle-tested behind Fortnite's UEFN (Unreal Editor for Fortnite) toolkit.

Why Does the World Need Another VCS?

Git is brilliant for text-based code. But game development isn't just text. It's 4K textures, uncompressed audio, rigged 3D models, animation sequences, and massive world maps. These files can be hundreds of megabytes each.

Git wasn't designed for this. Git LFS (Large File Storage) helps, but it's a patch on top of a fundamentally text-oriented system. Perforce Helix Core has been the industry standard for game studios for decades — but it's proprietary, expensive, and closed-source.

Epic Games looked at this landscape and said: we can do better.

What Is Lore?

Lore is a centralized, content-addressed version control system optimized for:

  • Large binary assets — textures, meshes, audio, video
  • Massive teams — hundreds of developers working simultaneously
  • Hybrid projects — code + binaries in the same repository
  • Sparse checkouts — developers only download what they need

Think of it as Perforce's philosophy (centralized, binary-friendly) combined with Git's content-addressed storage model, wrapped in Rust's performance guarantees.

How It Works Under the Hood

Lore's architecture is built around a few key technical decisions:

Content-Addressed Storage

Every piece of data is stored and referenced by its content hash. This means:

  • Automatic deduplication — identical content is stored once
  • Integrity verification — any tampering changes the hash
  • Efficient caching — content can be cached anywhere in the pipeline

Merkle Trees & Immutable Revision Chain

Revision hashes are cryptographically derived from parent hashes and data hashes. This creates a tamper-evident chain of history — similar to how Git works internally, but applied to binary-heavy workflows.

Chunked Storage

Large files aren't stored as monolithic blobs. They're split into reusable chunks. When a 200MB texture changes 5% of its data, Lore only stores the changed chunks — not the entire file again.

On-Demand Hydration

Developers don't download the entire repository. Workspaces fetch file data only when needed. Working on the UI? You don't need the 50GB of audio files sitting on your disk.

Centralized with Caching

Lore uses a centralized server (like Perforce), but adds intelligent caching to scale throughput. This gives teams a single source of truth while avoiding the bottleneck of every request hitting the origin server.

Key Features

Fast and Efficient
Shared, reusable data chunks and on-demand downloads keep operations fast even with terabyte-scale repositories.

Free Branching
Lightweight, low-overhead branching for experimentation and release management. No more "I'm scared to branch because it'll take 20 minutes."

Trustworthy History
Verifiable, tamper-evident source of truth. Every revision is cryptographically signed.

Full CLI Interface
Complete functionality via command line. No GUI dependency — integrate with whatever workflow you want.

Multi-Language API
Extend Lore via bindings for C/C++, C#, Rust, Go, Python, or JavaScript. Build custom tooling, integrate with your pipeline, automate anything.

The SDK Ecosystem

Lore isn't just a CLI tool. It's a platform:

SDK Language Use Case
Core Rust Server, library, CLI
lore-js JavaScript Web tooling, CI/CD
lore-python Python Pipeline automation
lore-dotnet C# Unity/Unreal integration
lore-go Go Backend services

Getting Started

Lore ships with a demo mode for immediate experimentation:

# macOS / Linux
curl -fsSL https://raw.githubusercontent.com/EpicGames/lore/main/scripts/install.sh | bash -s -- --demo
Enter fullscreen mode Exit fullscreen mode
# Windows (PowerShell)$env:LORE_DEMO=1;irmhttps://raw.githubusercontent.com/EpicGames/lore/main/scripts/install.ps1|iex
Enter fullscreen mode Exit fullscreen mode

The latest release is v0.8.3 — still pre-1.0, but functional and under active development.

Lore vs Git vs Perforce

Feature Git Git LFS Perforce Lore
Large binary support Poor Better Excellent Excellent
Open source
Cost Free Free $$$ Free
Content-addressed Partial
Chunked dedup
Sparse checkout Partial Partial
On-demand hydration
Built for binary assets Somewhat
Merkle tree integrity

Who Should Care?

Game studios — obvious. If you're shipping AAA titles with massive asset pipelines, Lore is built for you.

Film & VFX — same problem. Huge binary assets, multiple contributors, tight deadlines. Lore's architecture fits.

Enterprise software — any project with large binaries (ML models, datasets, media files) could benefit.

Open-source enthusiasts — Epic just gave the world a free, MIT-licensed alternative to Perforce. That's significant.

Current Limitations

Lore is pre-1.0. The team explicitly warns that interfaces, on-disk formats, and APIs may change between releases. The UEFN build currently uses a proprietary compression format not available in the open-source version.

It's also centralized by design — no distributed model like Git. For some teams, that's a feature. For others, it's a constraint.

Why This Matters

Epic Games didn't open-source Lore out of charity. They built it because Git and Perforce weren't good enough for Fortnite's development pipeline — a game with terabytes of assets, hundreds of concurrent developers, and daily updates.

By open-sourcing it, they're:

  1. Attracting contributors to improve the tool they depend on
  2. Building ecosystem lock-in — if studios adopt Lore, they're more likely to stay in the Unreal ecosystem
  3. Undercutting Perforce — free alternative to an expensive competitor

It's a smart business move disguised as community goodwill.

What's Next?

The Lore team is actively developing toward 1.0. The roadmap likely includes:

  • Stabilizing APIs and on-disk formats
  • Expanding the SDK ecosystem
  • Building GUI tooling
  • Enterprise features (SSO, audit logs, permissions)

The GitHub repo already has 808 stars and a growing community on Discord. If you're in game dev or working with large binary assets, Lore is worth watching.


Links:


Based on EpicGames/lore on GitHub and Epic Games announcement

Source: dev.to

arrow_back Back to Tutorials