Skip to content

Commit

Permalink
Graphene documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
Keavon committed Mar 11, 2024
1 parent 343523a commit adee127
Show file tree
Hide file tree
Showing 8 changed files with 167 additions and 50 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ Starting from the left, the <img src="https://static.graphite.rs/content/learn/i
Next, that is fed into the <img src="https://static.graphite.rs/content/learn/introduction/features-and-limitations/circular-repeat-node__2.avif" onerror="this.onerror = null; this.src = this.src.replace('.avif', '.png')" onload="this.width /= 2" style="vertical-align: middle" alt="Circular Repeat" /> node which has several *parameters* you can modify and get different output data based on your choices, like in these examples:

<style class="table-1-style">
.table-1-style + table {
width: auto;
}

.table-1-style + table td {
vertical-align: middle;
text-align: center;
Expand Down
103 changes: 103 additions & 0 deletions website/content/volunteer/guide/graphene/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
+++
title = "Graphene"
template = "book.html"
page_template = "book.html"

[extra]
order = 3 # Chapter number
js = ["video-embed.js"]
+++

**Graphene** is the node graph engine that powers the Graphite editor.

It's hard to describe in one sentence precisely what Graphene is, because it's a technology that serves several roles when viewed from different angles. But to get a feel for what it encompasses, here is a list of some of its purposes:

- Render engine
- Runtime environment
- Procedural data processor
- Node-based scripting system
- Compiled programming language
- Compiler toolchain built around `rustc`

## Background

### Artwork as a program

Artwork created in Graphite is represented as a node graph that generates the graphical content authored by the user. This document is essentially source code for a program in the Graphene language. Modifying the graph (like adding a layer, changing a node's parameter, or updating a node's data every frame while interactively drawing a shape) changes the actual program that generates and renders the artwork. This program must be recompiled and executed every frame a change is made.

Nodes are functions that run algorithms related to graphical operations. Some may read bitmap images from disk, others may generate procedural patterns, and more may be used for compositing and blending. Vector nodes can also produce shapes, alter their geometry, and apply styling and effects. Put together, a full document is built from just its interconnected nodes— producing a complete work of art generated entirely with algorithms and data.

### Graph executors as programming languages

Every node-based application needs to run its node graph to compute the resulting data. Execution occurs in an order that depends on the shape of the graph so that every node has the data it needs to compute its output.

A procedural graph executor, in its basic form, is a simple system that executes functions in the appropriate order. It feeds information between nodes and caches that data for reuse between executions so that only changed branches of the graph have to be computed again. The system, as described, is the approach commonly used by virtually all node-based apps.

Crucially, the execution flow is handled at runtime so there is some overhead during every run. By analogy to programming languages, this traditional execution model acts like an interpreted language. But interpreted languages are famously slow, and we don't want Graphite leaving performance on the table.

In designing Graphene, we decided to take a more advanced approach that could yield many of the benefits of a compiled language— code inlining, compiler optimizations, and a philosophy of offloading invariant enforcement to the type system. Instead of building a simple graph interpreter, we designed a system that dynamically executes the graph with a variable degree of pre-compiled optimizations. Graphene can range between an interpreted language, a JIT-optimized language, and a fully compiled language.

## Technical overview

### The latency/performance tradeoff

While working in Graphite, multiple needs arise for speed in different contexts. While making interactive changes, the user needs feedback as quickly as possible. While panning and zooming the canvas or playing an animation, the user cares about smoothness and responsiveness. When procedural artwork is exported as a standalone program that processes data at runtime, performance is the sole concern.

This sliding scale of latency/performance concerns maps directly to programming language concepts. Interpreted languages run immediately, but with slow runtime performance. JIT-optimized languages also run nearly without delay, but with less overhead than an interpreter since it can dynamically balance its effort towards optimizing and executing code. Compiled languages take upfront time to compile, but run with less overhead. A choice of optimization levels can be applied to further trade initial compilation time for runtime performance.

We designed Graphene to operate in interpreted, JIT, and compiled regimes.

### Building upon the Rust compiler

Nodes are functions written in Rust and every function has precompiled bytecode that ships with Graphite for use in the interpreted regime. The graph `input``A``B``C``output` is equivalent to the Rust statement `let output = C(B(A(input)));`. Graphene can either execute `A`, `B`, and `C` sequentially in its interpreted regime, or its JIT and compiled regimes can generate that Rust statement and compile it with the Rust compiler, `rustc`. The inlined and optimized bytecode can then be substituted for the chain of nodes in the JIT regime.

Graphene figures out which branches of the graph to compile and substitute as part of the JIT process while the user is authoring content in Graphite. While editing the graph, as changes occur to specific nodes, their surrounding graph branches drop back down to using the slower interpreted nodes. Then the JIT system works its way back up to faster execution over time by gradually compiling and swapping in larger optimized parts of the overall graph.

The fully compiled regime is used only when the user exports the procedural artwork as a standalone program. For example, a CLI program may read a string input argument (like a name) and procedurally generate an output image file (like a birthday card).

The interpreted regime is currently the only mode implemented in Graphene. The node registry (in the file `node_registry.rs`) currently exists to allow the interpreted executor to find the Rust functions that correspond to each node with its appropriate type signature.

| Regime | Usage |
|:------------|:----------------------------------------------------------------------|
| Interpreted | While editing. Simple and currently the only mode that's implemented. |
| JIT | While editing. Dynamically bridges the gap between both regimes by selectively substituting branches of the graph with interpreted and compiled nodes to keep latency low and work towards higher execution performance. |
| Compiled | When exported. The entire graph is compiled as a standalone program. |

### GPU compute shaders

Further building upon the Rust compiler toolchain, we employ the [`rust-gpu`](https://github.com/EmbarkStudios/rust-gpu) compiler backend for `rustc` which generates compute shaders that get executed on the GPU. This means we can write the same code to implement nodes that run on both CPU and GPU. (Although in practice, some nodes may need GPU-specific versions suited for the architectural limitations of GPU programming.) And we don't have to use a separate shader language!

### A language within a language

While Graphene is a programming language, it is also foundationally built upon the Rust language. We don't just use the Rust compiler, but we also employ its type system, traits, data structures, standard library, and crate ecosystem. The data that flows between nodes are Rust types (like structs, enums, tuples, primitives, and collections). Graphene's generic type system uses Rust's trait definitions in its enforcement of type safety and type inference.

### Graphene language concepts

Since Graphene is fundamentally a programming language, throughout this documentation we will use analogies which correlate Graphene concepts with their counterparts from traditional programming language theory. Here is an at-a-glance overview:

| Graphene concept | Programming language concept |
|:------------------|:-------------------------------------|
| Node | Function |
| Graphite editor | IDE/text editor |
| Document | Source code |
| Graph/network | Abstract syntax tree (AST) |
| Graph compilation | Linking/JIT optimization/compilation |
| Graph execution | Program execution |

<!-- The philosophy of building your own language features in the language itself -->
<!-- Compose nodes and automatic/manual composition -->
<!-- Extract/inject nodes and metaprogramming -->
<!-- Cache nodes and stable node IDs -->
<!-- Graph rewriting step (currently used only to remove Identity nodes),
at various points in the compilation process,
based on rules akin to an optimizing compiler -->
<!-- Borrow tree -->
<!-- Document nodes, proto nodes, and networks (must be: acyclic) -->
<!-- Lambdas -->
<!-- Graph compilation process -->
<!-- The compilation server -->
<!-- Code structure overview -->
<!-- Guide for implementing a node -->
<!-- The `Node` trait -->
<!-- Generics, type inference, type erasure, and the node registry -->
<!-- Monitor nodes -->
22 changes: 22 additions & 0 deletions website/content/volunteer/guide/graphene/networks-and-nodes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
+++
title = "Networks and nodes"

[extra]
order = 1 # Page number after chapter intro
+++

In Graphite, users build their artwork by connecting nodes together in a graph. When they want to organize and reuse a complex group of nodes, those may be encapsulated together as a subgraph in which one parent node represents the functionality of its children. In fact, many of the nodes provided in Graphite are themselves subgraphs built out of other nodes.

Double-clicking on nodes backed by a subgraph will display the subgraph's interior. Double-clicking nodes that are, instead, backed directly by Rust source code will open a code editor.

Any (sub)graph can import/export data from/to the outside world. For example, a reusable subgraph may receive an imported image then use several nodes to process it and finally export the result. Or the root-level artwork graph may import the animation timestamp and render a frame of the artwork then export it to the canvas.

In the Graphite editor UI, here is an example graph of artwork that imports no data but exports its content to the canvas:

<img src="https://static.graphite.rs/content/index/gui-mockup-nodes__5.avif" onerror="this.onerror = null; this.src = this.src.replace('.avif', '.png')" alt="Node graph UI mockup" data-carousel-image />

The graph shown above represents the full artwork, meaning it's the root-level graph in its document. But there is nothing special about that graph compared to any subgraph. To avoid the confusion of calling it a graph or subgraph which comes with implications about user-facing concepts in the context of a document, we will use the less-ambiguous term **network** in the context of Graphene's internal concepts and code base.

## Networks

A node network can be thought of as a box containing a finite set of nodes that are connected together. The network is only concerned with its own node-to-node data flow. But to interact with the outside world, data can be imported into the network and exported out of it. From the inside, those imported/exported data sources/destinations are connected to the other nodes in the network. From the outside, a network can be considered a "black box" that is simply fed inputs and can be executed to produce outputs.
2 changes: 1 addition & 1 deletion website/content/volunteer/guide/product-design/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ template = "book.html"
page_template = "book.html"

[extra]
order = 4 # Chapter number
order = 5 # Chapter number
+++

**NOTE: Developers probably don't need to read this chapter.**
Expand Down
2 changes: 1 addition & 1 deletion website/content/volunteer/guide/projects/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ template = "book.html"
page_template = "book.html"

[extra]
order = 3 # Chapter number
order = 4 # Chapter number
+++

Graphite is built from a number of separate projects, each with a distinct focus. New developers may choose to specialize in one or more area without having to attain a working knowledge of the entire codebase. This chapter details each project's purpose, what knowledge or background is best suited for getting involved with it, and how to begin making useful contributions. Opportunities also exist for contributing to specific self-contained sub-projects for students interested in completing a Google Summer of Code or other internship program, a university capstone project, or another similar endeavor.
Expand Down
5 changes: 3 additions & 2 deletions website/content/volunteer/guide/projects/student-projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,10 @@ You are encouraged to reference the project idea list below to find several pote

When it comes to writing the proposal, which you will submit to the GSoC application website, we offer some guidelines below:

- **Proposal structure:** Please consult the [Blender GSoC application template](https://developer.blender.org/docs/programs/gsoc/application_template/) as reference for our desired format.
- **Proposal structure:** Please consult the [Blender GSoC application template](https://developer.blender.org/docs/programs/gsoc/application_template/) as reference for our desired format. Remember: don't waste your time restating information that we already know, like background information about Graphite or the underlying technologies; we want to hear your thoughts and plans about what you uniquely bring to the table and how you will execute the project.
- **Your background:** We're especially interested in your background and experience, so attaching a résumé or CV is optional but highly recommended and will help us understand your capabilities. If able, please also include links to evidence of past open source contributions or personal projects in the bio section of your proposal. Our goal is to help you learn and grow as a productive open source software engineer, not to help you learn to program from scratch, so any such evidence will help us understand your potential as a self-motivated contributor to the open source community.
- **Prior PRs:** If you have made any contributions to Graphite and/or similar open source projects, please include links to your pull requests in your proposal. We put significant extra weight towards applicants who have already made successful contributions to Graphite because this shows your ability to work in a professional capacity with our team and demonstrates your interest in Graphite in particular (as opposed to a shotgun approach to GSoC applications). You can also state that you'll submit your first PRs (we encourage at least two) after the application deadline on [April 2](https://developers.google.com/open-source/gsoc/timeline#april_2_-_1800_utc), but before we make our final selections a couple days prior to [April 24](https://developers.google.com/open-source/gsoc/timeline#april_24_-_1800_utc). We are very likely to reject applicants who have not made meaningful contributions to Graphite by that time.
- **Your own words:** Don't use AI to write your proposal. You may find it's a helpful resource by asking it to check specific paragraphs for grammar and copy editing, but it should not be used to generate the text of your proposal. It is extremely easy for us to tell and we will notice. Our goal is for us to understand your own thoughts, plans, motivation, determination, and communication skills; the effort you put into your written proposal is an indicator for those criteria which we will consider in determining if you're a good fit.

<!-- TODO: Explain how we want a proposal structured -->

Expand Down Expand Up @@ -63,7 +64,7 @@ This project involves diving into the LibRaw source code (~44,000 lines), unders

The Graphite concept is built around a node graph representation of layer stacks, while tools automatically generate and manipulate nodes. When a layer or node is inserted, deleted, moved, or referenced, the graph needs to be reorganized to maintain a clear and useful layout. Users can also interactively expand and collapse groups of nodes which occupies or frees up graph real estate.

Unlike other node editors that are centered around manual graph editing, where users are fully in charge of node placements within one large node network, Graphite's node UI is more oriented towards automatic layout management and viewing just parts of the graph at one time. This means the shown graph toplogy is constantly changing and the layout system needs to cooperatively organize the graph in concert with user actions.
Unlike other node editors that are centered around manual graph editing, where users are fully in charge of node placements within one large node network, Graphite's node UI is more oriented towards automatic layout management and viewing just parts of the graph at one time. This means the shown graph topology is constantly changing and the layout system needs to cooperatively organize the graph in concert with user actions.

While general graph layout algorithms are complex and struggle to produce good results in other node editors, Graphite's graph topology is more constrained and predictable, which makes it possible to design a layout system that can produce good results. Nodes tend to be organized into rows, and layers into columns (see the image in the project below). This turns the problem into more of a constraint-based, axis-aligned packing problem.

Expand Down
73 changes: 27 additions & 46 deletions website/sass/base.scss
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
--border-thickness: 1px;
}

@media screen and (max-width: 500px) {
@media print, screen and (max-width: 500px) {
--font-size-intro-heading: 32px;
--font-size-intro-body: 16px;
--font-size-link: 16px;
Expand Down Expand Up @@ -151,47 +151,19 @@ p {
-webkit-hyphens: auto;
hyphens: auto;
text-align: justify;
}

p ~ img,
p ~ iframe {
width: 100%;
height: auto;
}

p ~ p {
margin-top: 1.5em;
}

h1,
h2,
h3,
h4 {
& ~ p,
& ~ ol li p,
& ~ img {
margin-top: 20px;
& ~ img,
& ~ iframe {
width: 100%;
height: auto;
}
}

h1 ~ .informational-group,
p ~ h1,
p ~ h2,
p ~ h3,
p ~ h4,
p ~ details summary,
p ~ blockquote,
p ~ .informational-group,
p ~ .image-comparison,
p + .link,
p + section,
p + table,
p + style + table,
p ~ .video-background,
p ~ .video-embed,
.video-embed + p,
.video-embed + .link,
.video-embed + .button,
:is(h1, h2, h3, h4, article > :first-child) ~ :is(p, ol li p, img),
:is(h1, p) ~ .informational-group,
p ~ :is(h1, h2, h3, h4, details summary, blockquote, .image-comparison, .video-background, .video-embed),
.video-embed + :is(p, .link, .button),
p + :is(.link, section),
img + .link {
margin-top: 20px;
}
Expand All @@ -213,18 +185,24 @@ li {
margin-top: 0.5em;
}

img {
vertical-align: top;
}

table {
margin: -20px;
margin: 20px -20px;
width: calc(100% + 40px);

th,
td {
border: 20px solid transparent;
vertical-align: top;
margin: 0;
padding: 0;
}

td {
vertical-align: top;
border: 20px solid transparent;

th:empty {
border: none;
}
}

Expand Down Expand Up @@ -352,8 +330,6 @@ pre {
}

kbd {
background: var(--color-fog);
border-radius: calc(var(--variable-px) * 2);
outline: calc(var(--border-thickness) / 2) solid var(--color-navy);
padding: 0 8px;
margin: 0 4px;
Expand Down Expand Up @@ -412,6 +388,11 @@ summary {
margin-top: 0;
}
}

// Captions below images in blog posts
p:has(> img) + center:has(> em) {
margin-top: 10px;
}
}
}

Expand Down Expand Up @@ -1152,7 +1133,7 @@ body > .page {
}
}

@media screen and (max-width: 880px) {
@media print, screen and (max-width: 880px) {
gap: 20px;

a {
Expand Down
6 changes: 6 additions & 0 deletions website/sass/book.scss
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@
}
}

@media print {
aside {
display: none;
}
}

// Page contents right sidebar is removed on smaller screens
@media screen and (max-width: 1200px) {
.contents {
Expand Down

0 comments on commit adee127

Please sign in to comment.