Gaussian splatting reference

Image-to-3D models whose output is a gaussian splat (a point cloud of oriented 3D gaussians) rather than a mesh. Deliverable file: gaussian .ply (the INRIA convention — the interchange format everything reads) and optionally .splat (antimatter15 32-byte records; smaller, no view-dependent color).

TRELLIS 1 also exports gaussian .ply alongside its mesh — covered in 3d-models.md.

Verify against the live official Space before writing files. Facts below are as of 2026-07.

The one decision that shapes everything: rasterize server-side or not?

Rendering gaussians on the server needs a CUDA rasterizer (diff-gaussian-rasterization or gsplat). Prebuilt Blackwell (sm_120) wheel availability is narrow: the multimodalart/zerogpu-blackwell-wheels dataset (see requirements.md) ships diff_gaussian_rasterization with the upstream Inria API only (2-tuple return) — the ashawkey fork most LGM-family code imports (4-tuple with alpha+depth) has no wheel, and gsplat's own wheel index tops out at torch 2.4/cu124. Two viable shapes:

Shape A — no server-side rendering (default, strongly preferred). The model decodes gaussians as plain PyTorch tensors; you write a .ply and let the browser render it. No CUDA extension at all — requirements.txt can be as small as gradio, torch, torchvision, numpy, safetensors, pillow, tqdm (TripoSplat's, in full). This is how every modern splat Space works (TripoSplat, SHARP, Splatt3R, Splatter Image; Splatt3R's requirements literally comment out the rasterizer dep).

Shape B — a real rasterizer (only for server-rendered orbit videos). If the code uses the upstream Inria API, take the prebuilt wheel from requirements.md's dataset and be done. ashawkey/LGM needs its fork, so its Space JIT-builds it — the GPU-worker bootstrap documented in 3d-cuda-extensions.md Strategy 3: @spaces.GPU(duration=600) setup function at module import, CUDA_HOME=/cuda-image/usr/local/cuda-13.0, TORCH_CUDA_ARCH_LIST="12.0", sitecustomize no-op of _check_cuda_version, pip install --no-build-isolation --no-deps git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git, then ctypes.CDLL(".../libcudart.so.13", RTLD_GLOBAL). Copy it from dylanebert/LGM-mini verbatim. Don't take this on unless the user explicitly wants server-rendered video output.

Viewing splats

• gr.Model3D renders gaussian .ply and .splat natively (Babylon.js; supported at least since Gradio 4.25, current in 6.x). Return the file path like any other output. display_mode is ignored for splats; clear_color, camera_position, height still apply. This is the zero-effort default — use it.
• Custom JS viewers give better splat sorting and controls when the demo is the product: TripoSplat serves a Spark.js (@sparkjsdev/spark) viewer page in an iframe; ml-sharp vendors a static PlayCanvas SuperSplat build and serves it via gr.set_static_paths(...) + gr.HTML iframe with /gradio_api/file=... URLs. Only go here if gr.Model3D's rendering visibly undersells the model.
• Always add a gr.DownloadButton for the .ply — splat users take the file to their own viewer/engine.

Output writing

Gaussian .ply (INRIA convention): binary little-endian, vertex props x,y,z,nx,ny,nz,f_dc_0..2,(f_rest_*),opacity(logit),scale_0..2(log),rot_0..3(quat). Model codebases ship their own writer (save_ply in LGM/SHARP/TripoSplat) — use it; don't hand-roll unless porting.

Optional .splat conversion for lighter downloads (~32 bytes/gaussian, drops view-dependent SH): TripoSplat's triposplat.py::to_splat_bytes is a complete pure-numpy reference — position f32×3, scale f32×3 (linear), RGBA u8×4 (rgb = (f_dc·0.2820948 + 0.5)·255, alpha = sigmoid(opacity)), quaternion u8×4 (q·128+128), records sorted by opacity×volume.

Watch orientation here too: splat conventions differ from viewers' (TripoSplat applies [[1,0,0],[0,0,-1],[0,1,0]] on export). If the splat renders sideways in gr.Model3D, fix it at export, same as meshes.

ZeroGPU pickle discipline (bites harder here)

Gaussian objects in these codebases are custom classes (often holding CUDA tensors or locks) — they cannot cross the @spaces.GPU boundary. Two proven patterns:

• Do everything — preprocess, sample, decode, .ply write — inside one decorated function and return only path strings (TripoSplat, with an explicit comment to that effect).
• Never decorate bound methods; wrap module-level functions (spaces.GPU(duration=180)(predict_to_ply) — ml-sharp, whose ModelWrapper holds a non-picklable threading.RLock).

Durations: TripoSplat and LGM-mini run on the bare @spaces.GPU default 60s; SHARP uses 180. Splat decoding is fast — the multiview-diffusion stage (LGM) is what eats time.

Model-specific notes

• TripoSplat — the current default pick: MIT, running official ZeroGPU Space, fp16 weights well within 48 GB, ~262k gaussians per generation, BiRefNet background removal built in. Caveat when duplicating: the official Space uses a fully custom Gradio 6 gradio.Server + index.html frontend, not gr.Blocks — its HTTP API is bespoke, so either keep it wholesale or rebuild a plain gr.Blocks + gr.Model3D UI around TripoSplatPipeline (the pipeline code is self-contained in triposplat.py/model.py). Weights download at startup via hf download VAST-AI/TripoSplat --local-dir ckpts.
• LGM — image → 4 multiview images (ImageDream) → gaussians. dylanebert/LGM-mini is the clean diffusers-style duplicate target (both pipelines via from_pretrained(..., trust_remote_code=True), output straight into gr.Model3D); ashawkey/LGM is the video-rendering variant (Shape B). Both carry an xformers→SDPA monkeypatch for Blackwell — keep it when duplicating (the xformers 0.0.34 Blackwell wheel in requirements.md's dataset makes the patch unnecessary if you modernize the requirements instead, but the patch is harmless).
• SHARP — pip-installable (sharp @ git+https://github.com/apple/ml-sharp.git@<sha>), predicts a camera-space scene splat from one photo. Research-only license (apple-amlr) — tell the user before they build anything commercial. preload_from_hub in the frontmatter pre-bakes the checkpoint into the Space image.
• Splatter Image — the minimal skeleton (~CVPR 2024, per-pixel gaussians, MIT): good template bones, but its Space pins Gradio 4.27 and a cu113 torch via pre-requirements.txt — modernize rather than copy pins.