Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make cugraph-ops optional for cugraph-gnn packages #99

Open
wants to merge 13 commits into
base: branch-25.02
Choose a base branch
from
2 changes: 0 additions & 2 deletions conda/environments/all_cuda-118_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,11 @@ dependencies:
- pandas
- pre-commit
- pydantic
- pylibcugraphops==25.2.*,>=0.0.0a0
- pytest
- pytest-benchmark
- pytest-cov
- pytest-forked
- pytest-xdist
- pytorch-cuda=11.8
- pytorch>=2.3
- pytorch_geometric>=2.5,<2.6
- rapids-build-backend>=0.3.0,<0.4.0.dev0
Expand Down
2 changes: 0 additions & 2 deletions conda/environments/all_cuda-121_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,11 @@ dependencies:
- pandas
- pre-commit
- pydantic
- pylibcugraphops==25.2.*,>=0.0.0a0
- pytest
- pytest-benchmark
- pytest-cov
- pytest-forked
- pytest-xdist
- pytorch-cuda=12.1
- pytorch>=2.3
- pytorch_geometric>=2.5,<2.6
- rapids-build-backend>=0.3.0,<0.4.0.dev0
Expand Down
2 changes: 0 additions & 2 deletions conda/environments/all_cuda-124_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,11 @@ dependencies:
- pandas
- pre-commit
- pydantic
- pylibcugraphops==25.2.*,>=0.0.0a0
- pytest
- pytest-benchmark
- pytest-cov
- pytest-forked
- pytest-xdist
- pytorch-cuda=12.4
- pytorch>=2.3
- pytorch_geometric>=2.5,<2.6
- rapids-build-backend>=0.3.0,<0.4.0.dev0
Expand Down
3 changes: 1 addition & 2 deletions conda/recipes/cugraph-dgl/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.

{% set version = environ['RAPIDS_PACKAGE_VERSION'].lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
{% set minor_version = version.split('.')[0] + '.' + version.split('.')[1] %}
Expand Down Expand Up @@ -29,7 +29,6 @@ requirements:
- numba >=0.57
- numpy >=1.23,<3.0a0
- pandas
- pylibcugraphops ={{ minor_version }}
- tensordict >=0.1.2
- python
- pytorch >=2.3
Expand Down
3 changes: 1 addition & 2 deletions conda/recipes/cugraph-pyg/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022-2024, NVIDIA CORPORATION.
# Copyright (c) 2022-2025, NVIDIA CORPORATION.

{% set version = environ['RAPIDS_PACKAGE_VERSION'].lstrip('v') + environ.get('VERSION_SUFFIX', '') %}
{% set minor_version = version.split('.')[0] + '.' + version.split('.')[1] %}
Expand Down Expand Up @@ -35,7 +35,6 @@ requirements:
- pytorch >=2.3
- cupy >=12.0.0
- cugraph ={{ minor_version }}
- pylibcugraphops ={{ minor_version }}
- tensordict >=0.1.2
- pytorch_geometric >=2.5,<2.6

Expand Down
46 changes: 0 additions & 46 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ files:
- depends_on_cugraph
- depends_on_cudf
- depends_on_dask_cudf
- depends_on_pylibcugraphops
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the dependencies in the conda recipes be removed as well?

- pylibcugraphops ={{ minor_version }}

- pylibcugraphops ={{ minor_version }}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tingyu66 could you please make this change (removing the dependencies on pylibcugraphops in conda recipes) in this PR?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jameslamb Done, thank you!

- depends_on_cupy
- depends_on_pytorch
- depends_on_dgl
Expand All @@ -43,7 +42,6 @@ files:
- cuda_version
- docs
- py_version
- depends_on_pylibcugraphops
test_cpp:
output: none
includes:
Expand Down Expand Up @@ -114,7 +112,6 @@ files:
table: project
includes:
- depends_on_cugraph
- depends_on_pylibcugraphops
- python_run_cugraph_dgl
py_test_cugraph_dgl:
output: pyproject
Expand All @@ -140,7 +137,6 @@ files:
table: project
includes:
- depends_on_cugraph
- depends_on_pylibcugraphops
- depends_on_pyg
- python_run_cugraph_pyg
py_test_cugraph_pyg:
Expand All @@ -164,7 +160,6 @@ files:
includes:
- checks
- depends_on_cugraph
- depends_on_pylibcugraphops
- depends_on_dgl
- depends_on_pytorch
- cugraph_dgl_dev
Expand All @@ -178,7 +173,6 @@ files:
- checks
- depends_on_cugraph
- depends_on_pyg
- depends_on_pylibcugraphops
- depends_on_pytorch
- cugraph_pyg_dev
- test_python_common
Expand Down Expand Up @@ -428,21 +422,6 @@ dependencies:
- *pytorch_pip
- *tensordict
- {matrix: null, packages: [*pytorch_pip, *tensordict]}
- output_types: [conda]
matrices:
- matrix: {cuda: "12.1"}
packages:
- pytorch-cuda=12.1
- matrix: {cuda: "12.4"}
packages:
- pytorch-cuda=12.4
- matrix: {cuda: "11.8"}
packages:
- pytorch-cuda=11.8
# pytorch only supports certain CUDA versions... skip
# adding pytorch-cuda pinning if any other CUDA version is requested
- matrix:
packages:

depends_on_dgl:
specific:
Expand Down Expand Up @@ -615,31 +594,6 @@ dependencies:
- pylibcugraph-cu11==25.2.*,>=0.0.0a0
- {matrix: null, packages: [*pylibcugraph_unsuffixed]}

depends_on_pylibcugraphops:
common:
- output_types: conda
packages:
- &pylibcugraphops_unsuffixed pylibcugraphops==25.2.*,>=0.0.0a0
- output_types: requirements
packages:
# pip recognizes the index as a global option for the requirements.txt file
- --extra-index-url=https://pypi.nvidia.com
- --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple
specific:
- output_types: [requirements, pyproject]
matrices:
- matrix:
cuda: "12.*"
cuda_suffixed: "true"
packages:
- pylibcugraphops-cu12==25.2.*,>=0.0.0a0
- matrix:
cuda: "11.*"
cuda_suffixed: "true"
packages:
- pylibcugraphops-cu11==25.2.*,>=0.0.0a0
- {matrix: null, packages: [*pylibcugraphops_unsuffixed]}

depends_on_cupy:
common:
- output_types: conda
Expand Down
2 changes: 0 additions & 2 deletions python/cugraph-dgl/conda/cugraph_dgl_dev_cuda-118.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,10 @@ dependencies:
- dglteam/label/th23_cu118::dgl>=2.4.0.th23.cu*
- pre-commit
- pydantic
- pylibcugraphops==25.2.*,>=0.0.0a0
- pytest
- pytest-benchmark
- pytest-cov
- pytest-xdist
- pytorch-cuda=11.8
- pytorch>=2.3
- tensordict>=0.1.2
- torchdata
Expand Down
12 changes: 1 addition & 11 deletions python/cugraph-dgl/cugraph_dgl/nn/conv/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
Expand All @@ -12,17 +12,7 @@
# limitations under the License.

from .base import SparseGraph
from .gatconv import GATConv
from .gatv2conv import GATv2Conv
from .relgraphconv import RelGraphConv
from .sageconv import SAGEConv
from .transformerconv import TransformerConv

__all__ = [
"SparseGraph",
"GATConv",
"GATv2Conv",
"RelGraphConv",
"SAGEConv",
"TransformerConv",
]
18 changes: 15 additions & 3 deletions python/cugraph-dgl/cugraph_dgl/nn/conv/base.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
Expand Down Expand Up @@ -279,6 +279,18 @@ def to(self, device: Union[torch.device, str, int]) -> "cugraph_dgl.nn.SparseGra
return sg


def conditional_class(import_name):
def decorator(cls):
try:
__import__(import_name)
return cls
except ImportError:
return None

return decorator


@conditional_class("pylibcugraphops")
class BaseConv(torch.nn.Module):
r"""An abstract base class for cugraph-ops nn module."""

Expand All @@ -298,7 +310,7 @@ def get_cugraph_ops_CSC(
g: Union[SparseGraph, dgl.DGLHeteroGraph],
is_bipartite: bool = False,
max_in_degree: Optional[int] = None,
) -> ops_torch.CSC:
) -> "ops_torch.CSC":
"""Create CSC structure needed by cugraph-ops."""

if not isinstance(g, (SparseGraph, dgl.DGLHeteroGraph)):
Expand Down Expand Up @@ -333,7 +345,7 @@ def get_cugraph_ops_HeteroCSC(
etypes: Optional[torch.Tensor] = None,
is_bipartite: bool = False,
max_in_degree: Optional[int] = None,
) -> ops_torch.HeteroCSC:
) -> "ops_torch.HeteroCSC":
"""Create HeteroCSC structure needed by cugraph-ops."""

if not isinstance(g, (SparseGraph, dgl.DGLHeteroGraph)):
Expand Down
8 changes: 7 additions & 1 deletion python/cugraph-dgl/cugraph_dgl/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2021-2024, NVIDIA CORPORATION.
# Copyright (c) 2021-2025, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
Expand All @@ -22,6 +22,12 @@
)


def pytest_ignore_collect(collection_path, config):
"""Return True to prevent considering this path for collection."""
if "nn" in collection_path.name:
return True


@pytest.fixture(scope="module")
def dask_client():
# start_dask_client will check for the SCHEDULER_FILE and
Expand Down
5 changes: 4 additions & 1 deletion python/cugraph-dgl/cugraph_dgl/tests/test_utils.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2022-2024, NVIDIA CORPORATION.
# Copyright (c) 2022-2025, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
Expand All @@ -11,6 +11,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import pytest
import cudf
import cupy as cp
import numpy as np
Expand Down Expand Up @@ -179,6 +180,7 @@ def test_get_source_destination_range():
assert output_d == expected_output


@pytest.mark.skip(reason="Skipping due to missing cugraph-ops backend.")
def test__create_homogeneous_cugraph_dgl_nn_sparse_graph():
Copy link
Contributor

@bdice bdice Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Looks like a typo with the double underscore here.

Suggested change
def test__create_homogeneous_cugraph_dgl_nn_sparse_graph():
def test_create_homogeneous_cugraph_dgl_nn_sparse_graph():

tensor_d = {
"sources_range": 1,
Expand All @@ -196,6 +198,7 @@ def test__create_homogeneous_cugraph_dgl_nn_sparse_graph():
assert isinstance(sparse_graph, cugraph_dgl.nn.SparseGraph)


@pytest.mark.skip(reason="Skipping due to missing cugraph-ops backend.")
def test_create_homogeneous_sampled_graphs_from_dataframe_csc():
df = get_dummy_sampled_df_csc()
batches = create_homogeneous_sampled_graphs_from_dataframe_csc(df)
Expand Down
1 change: 0 additions & 1 deletion python/cugraph-dgl/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ dependencies = [
"cugraph==25.2.*,>=0.0.0a0",
"numba>=0.57",
"numpy>=1.23,<3.0a0",
"pylibcugraphops==25.2.*,>=0.0.0a0",
] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit ../../dependencies.yaml and run `rapids-dependency-file-generator`.

[project.optional-dependencies]
Expand Down
2 changes: 0 additions & 2 deletions python/cugraph-pyg/conda/cugraph_pyg_dev_cuda-118.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@ dependencies:
- cugraph==25.2.*,>=0.0.0a0
- pre-commit
- pydantic
- pylibcugraphops==25.2.*,>=0.0.0a0
- pytest
- pytest-benchmark
- pytest-cov
- pytest-xdist
- pytorch-cuda=11.8
- pytorch>=2.3
- pytorch_geometric>=2.5,<2.6
- tensordict>=0.1.2
Expand Down
18 changes: 8 additions & 10 deletions python/cugraph-pyg/cugraph_pyg/examples/graph_sage_mg.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2023-2024, NVIDIA CORPORATION.
# Copyright (c) 2023-2025, NVIDIA CORPORATION.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
Expand Down Expand Up @@ -29,7 +29,7 @@
import torch
import numpy as np

from cugraph_pyg.nn import SAGEConv as CuGraphSAGEConv
from torch_geometric.nn import SAGEConv

import torch.nn as nn
import torch.nn.functional as F
Expand All @@ -41,22 +41,21 @@
from typing import List


class CuGraphSAGE(nn.Module):
class GraphSAGE(nn.Module):
def __init__(self, in_channels, hidden_channels, out_channels, num_layers):
super().__init__()

self.convs = torch.nn.ModuleList()
self.convs.append(CuGraphSAGEConv(in_channels, hidden_channels))
self.convs.append(SAGEConv(in_channels, hidden_channels))
for _ in range(num_layers - 1):
conv = CuGraphSAGEConv(hidden_channels, hidden_channels)
conv = SAGEConv(hidden_channels, hidden_channels)
self.convs.append(conv)

self.lin = nn.Linear(hidden_channels, out_channels)

def forward(self, x, edge, size):
edge_csc = CuGraphSAGEConv.to_csc(edge, (size[0], size[0]))
def forward(self, x, edge):
for conv in self.convs:
x = conv(x, edge_csc)[: size[1]]
x = conv(x, edge)
x = F.relu(x)
x = F.dropout(x, p=0.5)

Expand Down Expand Up @@ -262,7 +261,7 @@ def train(
)
td.barrier()
model = (
CuGraphSAGE(in_channels=128, hidden_channels=64, out_channels=349, num_layers=3)
GraphSAGE(in_channels=128, hidden_channels=64, out_channels=349, num_layers=3)
.to(torch.float32)
.to(device_id)
)
Expand Down Expand Up @@ -316,7 +315,6 @@ def train(
hetero_data.edge_index_dict[("paper", "cites", "paper")].to(
device_id
),
(len(y_true), len(y_true)),
)
end_time_forward = time.perf_counter_ns()
total_time_forward += (end_time_forward - start_time_forward) / 1e9
Expand Down
Loading
Loading