Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for SNAL attack #2440

Merged
merged 34 commits into from
Oct 22, 2024
Merged

Conversation

CNOCycle
Copy link
Contributor

Description

This pull request adds the support of the SNAL Attack proposed in [1].

[1]Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks. [Paper]
Fixes # (issue)

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • Notebook Example

Test Configuration:

  • OS: Ubuntu 20.04
  • Python version: 3.8.12
  • ART version or commit number: 900f470
  • PyTorch version: 1.13.1+cu116
  • cudnn version: 8302

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • My changes have been tested using both CPU and GPU devices

@beat-buesser beat-buesser self-requested a review May 6, 2024 09:21
@beat-buesser beat-buesser self-assigned this May 6, 2024
@beat-buesser beat-buesser added the enhancement New feature or request label May 6, 2024
@beat-buesser beat-buesser added this to the ART 1.18.0 milestone May 6, 2024
@beat-buesser beat-buesser changed the base branch from main to dev_1.18.0 May 13, 2024 21:57
Copy link

codecov bot commented May 13, 2024

Codecov Report

Attention: Patch coverage is 94.39655% with 26 lines in your changes missing coverage. Please review.

Project coverage is 85.38%. Comparing base (7983374) to head (ce3aec7).
Report is 35 commits behind head on dev_1.19.0.

Files with missing lines Patch % Lines
...n/steal_now_attack_later/steal_now_attack_later.py 95.30% 7 Missing and 13 partials ⚠️
...cks/evasion/steal_now_attack_later/drop_block2d.py 75.00% 2 Missing and 2 partials ⚠️
...mators/object_detection/pytorch_object_detector.py 84.61% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@              Coverage Diff               @@
##           dev_1.19.0    #2440      +/-   ##
==============================================
+ Coverage       85.25%   85.38%   +0.13%     
==============================================
  Files             330      333       +3     
  Lines           30470    30930     +460     
  Branches         5228     5294      +66     
==============================================
+ Hits            25977    26410     +433     
- Misses           3042     3053      +11     
- Partials         1451     1467      +16     
Files with missing lines Coverage Δ
art/attacks/evasion/__init__.py 100.00% <100.00%> (ø)
...attacks/evasion/steal_now_attack_later/bbox_ioa.py 100.00% <100.00%> (ø)
art/estimators/object_detection/pytorch_yolo.py 87.23% <ø> (ø)
...mators/object_detection/pytorch_object_detector.py 89.44% <84.61%> (-0.56%) ⬇️
...cks/evasion/steal_now_attack_later/drop_block2d.py 75.00% <75.00%> (ø)
...n/steal_now_attack_later/steal_now_attack_later.py 95.30% <95.30%> (ø)

... and 3 files with indirect coverage changes

Copy link
Collaborator

@beat-buesser beat-buesser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @CNOCycle Thank you very much for you pull request. I think it looks good, I have only found a few formatting and documentation issues. Could you please take a look and let me know if you have questions?

@@ -0,0 +1,747 @@
#
# Copyright (C) The Adversarial Robustness Toolbox (ART) Authors 2018
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Copyright (C) The Adversarial Robustness Toolbox (ART) Authors 2018
# Copyright (C) The Adversarial Robustness Toolbox (ART) Authors 2024

Comment on lines 18 to 21
"""
This module implements the paper: "Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks"
| Paper link: https://arxiv.org/abs/2304.05370
"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"""
This module implements the paper: "Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks"
| Paper link: https://arxiv.org/abs/2304.05370
"""
"""
This module implements the paper: "Steal Now and Attack Later: Evaluating Robustness of Object Detection against Black-box Adversarial Attacks"
| Paper link: https://arxiv.org/abs/2304.05370
"""

Comment on lines 25 to 26
import logging
from typing import Optional, Tuple, TYPE_CHECKING
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import logging
from typing import Optional, Tuple, TYPE_CHECKING
import logging
import random
from typing import Optional, Tuple, TYPE_CHECKING

from typing import Optional, Tuple, TYPE_CHECKING

import numpy as np
import random
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import random


x_out[:, :, y1:y2, x1:x2] = updated

return x_out, tile_mat
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add empty line at end of file

Suggested change
return x_out, tile_mat
return x_out, tile_mat

Comment on lines 633 to 634
import torch
TRIAL = 10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
import torch
TRIAL = 10
import torch
TRIAL = 10


def __init__(
self,
estimator: "torch.nn.Module",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
estimator: "torch.nn.Module",
estimator: PYTORCH_OBJECT_DETECTION_TYPE,

and please add

PYTORCH_OBJECT_DETECTION_TYPE  = Union[PyTorchObjectDetector]

to art/utils.py with the other type definitions at the tope of that file.

# pylint: disable=C0412
import torch

logger = logging.getLogger(__name__)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add docstrings to all functions in this module?


logger = logging.getLogger(__name__)

def _bbox_ioa(box1: "torch.tenosr",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please include the license text of the file yolov5/utils/metrics.py to this docstring?

block_size: int):
"""
=== NOTE ===
This function is modified from torchvision (torchvision/ops/drop_block.py)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please include the license text of the file yolov5/utils/metrics.py to this docstring?

@CNOCycle CNOCycle force-pushed the attack/snal branch 2 times, most recently from f0c4c5d to 87c5b9b Compare July 31, 2024 12:57
@CNOCycle CNOCycle changed the base branch from dev_1.18.0 to dev_1.19.0 August 6, 2024 13:14
@beat-buesser beat-buesser self-requested a review October 22, 2024 14:09
@beat-buesser
Copy link
Collaborator

Hi @CNOCycle Thank you very much for your contribution of the SNAL attack to ART!

@beat-buesser beat-buesser merged commit 35781a5 into Trusted-AI:dev_1.19.0 Oct 22, 2024
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants