Huge cloud like object appears in the scene reconstructed with splatfacto

[barikata1984]

Hello everyone,

Does anybody know why a huge cloud-like artifact appears above the target object when reconstructing it with splatfacto and how to remove it?

I am a graduate student working on a 3D Gaussian Splatting project and have recently started using this library. As a starter, I tried reconstructing the lego bulldozer with splatfacto. The bulldozer itself was reconstructed pretty nicely, but a large cloud-like object appeared above the bulldozer, along with minor artifacts surrounding it. Is that an expected behaviour?

I noticed a similar topic has been discussed at #3379, but it is about an issue happening when the resultant splats are exported as point clouds. So, the solution proposed there was not helpful.

Following is my setup and a screenshot of the training result:

Hardware and software:

• OS: Ubuntu 22.04
• GPU:RTX3090
• Python: 3.10.15
• Cuda: 11.8
• PyTorch: 2.1.2
• NerfStudio: 1.1.5

Dataset pre-processing:
The following code was applied to the original lego dataset to accommodate it to nerfstudio.

import json
import os
import numpy as np
import tyro
from pathlib import Path


def main(
        transform_path: str,
        img_extension: str = ".png",
    ) -> None:
    """Add additional data and a file extension if file_path in a transform json file written in the 
    Neural Radiance Fields' dataset style does not have a file extension.

    Args:
        transform_path: Path to an original transform json file.
        img_extension: File extension to be added.
    """
    transform_path = Path.cwd() / transform_path

    with open(transform_path) as f:
        data = json.load(f)

    # Add camera intrinsics required by nerfstudio
    data["h"] = 800
    data["w"] = 800 
    data["cx"] = 400  # principal point may be defined w.r.t a corner in nerfstudio?
    data["cy"] = 400
    data["fl_x"] = (0.5 * data["w"]) / np.tan(0.5 * float(data["camera_angle_x"]))
    data["fl_y"] = data["fl_x"]

    # Add the file extension to image files
    for frame in data["frames"]:
        if 0 == len(Path(frame["file_path"]).suffix):
            frame["file_path"] += img_extension

    # Save the json object
    output_path = transform_path.with_stem(f"ns-{transform_path.stem}")
    with open(output_path, 'w') as f:
        json.dump(data, f)


if __name__ == "__main__":
    # Generate a CLI and call `main` with its two arguments: `foo` and `bar`.
    tyro.cli(main)

Training command:

ns-train splatfact --data path/to/lego/dataset

Screenshot:

Expected behavior
At least the gigantic cloud disappears. Hopefully, the minor ones also get eliminated.

Thanks is advance!