Differential Surfel Rasterization MCMC

This project extends the surfel-rasterization engine of 2D Gaussian Splatting, by integrating a relocation kernel based on Markov Chain Monte Carlo (3DGS-MCMC) principles.

This relocation strategy enhances handling Gaussian splat parameters, focusing on maintaining sample state probabilities during heuristic moves like 'move', 'split', 'clone', 'prune', and 'add'.

Except for the relocation kernel, the engine is exactly same as the original.

Installation

To use the engine, follow these steps:

• Clone the repository:

git clone https://github.com/hwanhuh/diff-surfel-rasterization.git
cd diff-surfel-rasterization

• Install the package

pip install . --no-cache

• Alternatively, you can set up the Python C++ extension project:

python setup.py build_ext --inplace

Example Usage

from diff_surfel_rasterization import compute_relocation
import torch
import math

N_MAX = 51
BINOMS = torch.zeros((N_MAX, N_MAX)).float().cuda()
for n in range(N_MAX):
    for k in range(n+1):
        BINOMS[n, k] = math.comb(n, k)

def compute_relocation_cuda(
    opacities,  # [N]
    scales,  # [N, 2]
    ratios,  # [N]
):
    """
    Computes new opacities and scales using the MCMC relocation kernel.

    Args:
        opacities (torch.Tensor): Array of opacities for each Gaussian splat.
        scales (torch.Tensor): Array of scales for each Gaussian splat.
        ratios (torch.Tensor): Array of ratios used in relocation computation.

    Returns:
        new_opacities (torch.Tensor): Updated opacities after relocation.
        new_scales (torch.Tensor): Updated scales after relocation.
    """
    N = opacities.shape[0]
    opacities = opacities.contiguous()
    scales = scales.contiguous()
    ratios.clamp_(min=1, max=N_MAX)
    ratios = ratios.int().contiguous()

    new_opacities, new_scales = compute_relocation(
        opacities, scales, ratios, BINOMS, N_MAX
    )
    return new_opacities, new_scales

Acknowledgments

This project builds upon the research and implementations detailed in the following papers:

• 3D Gaussian Splatting as Markov Chain Monte Carlo
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields