"create_graph.py" takes a lot of time!

[davidqiu1993]

It only runs for 15 iterations after 4 hours, which is quite unreasonable. Is it expected or there are some parameters to configure making it run faster?

It is taking an 4090 GPU up to 17GB GPU memory, and my flash memory up to about 45GB.

python application/create_graph.py main.dataset=hm3dsem main.dataset_path=data/hm3dsem_walks/val/00824-Dd4bFSTQ8gi/ main.save_path=data/scene_graphs/00824-Dd4bFSTQ8gi
[2024-08-20 15:24:57,570][root][INFO] - Loaded ViT-H-14 model config.
[2024-08-20 15:25:04,095][root][INFO] - Loading pretrained ViT-H-14 weights (checkpoints/laion2b_s32b_b79k.bin).
Creating RGB-D point cloud: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 226/226 [00:12<00:00, 18.73it/s]
Extracting features: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 226/226 [12:27<00:00,  3.31s/it]
Merging 3d masks sequentially
  7%|██████████▍                                        15/225 [4:19:08<204:03:45, 3498.22s/it]

[Zflying1]

Hi! Have you solved it? I came across the same problem.

[zRiverBird]

Hi! Have you solved it? I came across the same problem.

[davidqiu1993]

Nope. I am waiting for the authors' response as well.

[Benaidandan]

The same problem ! Author I need help ,please!!!!

[Tom-Huang]

Dear @davidqiu1993 @Zflying1 @Benaidandan,
Thank you for your interest in our project and sorry for the late reply. Currently, the major bottleneck of the pipeline is the merging of segments. More specifically, the heavy computation is from getting the overlapping ratio between two point clouds (See this line). So far the overlapping computation heavily relies on the faiss implementation of nearest neighbor search in the point cloud. Although the implementation already boost the speed of searching compared to other methods like KD-Tree etc., given the plethora of frames and globally maintained 3D segments, the computation still takes forever on normal computing resources. One way to further boost the speed is to maintain a 3D voxel array storing the indices of the global instances at indexed location (row, col, height). Such implementation can speed up the merging by simply looking up the 3D voxel array to retrieve global instance IDs with overlapping indexed locations O(N) (N is the number of points in the query point cloud) instead of finding nearest neighbors for each point in the query point cloud O(MN) (M is the number of points in the source point cloud). So far we are planning to integrate the implementation into the pipeline but we are currently at capacity. We will only have time to implement it after two weeks. Sorry for the inconvenience.

[sktalha68]

Hello. Can you please confirm if you have pushed the fix for this issue or not?

[abwerby]

Hello,
We are still working on improving the merge phase performance for both sequential and hierarchical merges.
For now you can change the hyperparameters (increase voxel size, skip more frames)

HOV-SG/config/create_graph.yaml

Line 30 in 6d9cd24

skip_frames: 10