Skip to content

Commit

Permalink
feat/docsite publishing (#93)
Browse files Browse the repository at this point in the history 
* feat: #comment updated mkdocs related configs, no code changes

* feat: #comment forcing LF line ending swap on requirements.txt file

* chore: #comment requirements.txt cleanup after line-ending fix

* feat: #comment adding additional docs

* feat: #comment updated index and contributing.md to align with root content

* feat: #comment updated docs and examples based on feedback, also applied the same index.md fixes to main README.md as that is where those incorrect values were sourced from
  • Loading branch information
@scrthq
scrthq authored Jun 20, 2024
1 parent 9c1b106 commit 290ae03
Show file tree
Hide file tree
Showing 11 changed files with 375 additions and 171 deletions.
14 changes: 5 additions & 9 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

- [ASH; The *A*utomated *S*ecurity *H*elper](#ash-the-automated-security-helper)
- [Description](#description)
- [ASH change advisory](#ash-change-advisory)
- [Supported frameworks](#supported-frameworks)
- [Prerequisites](#prerequisites)
- [Getting Started](#getting-started)
Expand All @@ -14,6 +13,7 @@
- [Synopsis](#synopsis)
- [FAQ](#faq)
- [Feedback](#feedback)
- [Contributing](#contributing)
- [Security](#security)
- [License](#license)

Expand All @@ -28,10 +28,6 @@ by providing a fast and easy tool to conduct preliminary security check as earl
- It uses light, open source tools to maintain its flexibility and ability to run from anywhere.
- ASH is cloning and running different open-source tools, such as: git-secrets, bandit, Semgrep, Grype, Syft, nbconvert, npm-audit, checkov, cdk-nag and cfn-nag. Please review the tools [LICENSE](license) before usage.

### ASH change advisory

We are currently working on a re-architecture of ASH targeting a single-container architecture as well as documentation to go along with it.

## Supported frameworks

The security helper supports the following vectors:
Expand Down Expand Up @@ -218,17 +214,17 @@ SYNOPSIS:
OPTIONS:
-v | --version Prints version number.
-p | --preserve-report Add timestamp to the final report file to avoid overriding it after multiple executions.
-p | --preserve-report Add timestamp to the final report file to avoid overwriting it after multiple executions.
--source-dir Path to the directory containing the code/files you wish to scan. Defaults to $(pwd)
--output-dir Path to the directory that will contain the report of the scans. Defaults to $(pwd)
--ext | -extension Force a file extension to scan. Defaults to identify files automatically.
--force Rebuild the Docker images of the scanning tools, to make sure software is up-to-date.
--no-cleanup Don't cleanup the work directory where temp reports are stored during scans.
--debug Print ASH debug log information where applicable.
-q | --quiet Don't print verbose text about the build process.
-c | --no-color Don't print colorized output.
-s | --single-process Run ash scanners serially rather than as separate, parallel sub-processes.
-o | --oci-runner Use the specified OCI runner instead of docker to run the containerized tools.
-f | --finch Use finch instead of docker to run the containerized tools.
WARNING: The '--finch|-f' option is deprecated and will be removed in a future
release. Please switch to using '--oci-runner finch' in scripts instead.
```

## FAQ
Expand Down
33 changes: 17 additions & 16 deletions docs/content/contributing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ documentation, we greatly value feedback and contributions from our community.
Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
information to effectively respond to your bug report or contribution.


## Reporting Bugs/Feature Requests

We welcome you to use the GitHub issue tracker to report bugs or suggest features.
Expand All @@ -19,43 +18,45 @@ reported the issue. Please try to include as much information as you can. Detail
* Any modifications you've made relevant to the bug
* Anything unusual about your environment or deployment


## Contributing via Pull Requests

Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:

1. You are working against the latest source on the *master* branch.
1. You are working against the latest source on the `main` branch.
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.

To send us a pull request, please:

1. Fork the repository.
2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
3. Ensure local tests pass.
4. Commit to your fork using clear commit messages.
5. Send us a pull request, answering any default questions in the pull request interface.
6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.

GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
2. Create a branch in your fork where the branch name is something meaningful. We encourage
the use of `feature/<short-description>`, `bugfix/<short-description>`, `hotfix/<short-description>`, and so on for branch naming.
3. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code,
it will be hard for us to focus on your change.
4. Ensure local tests pass.
5. Commit to your fork using clear commit messages.
6. Send us a pull request, answering any default questions in the pull request interface.
7. Pay attention to any automated continuous integration (CI) failures reported in the pull request, and stay involved in the conversation.

GitHub provides additional documentation on [forking a repository](https://help.github.com/articles/fork-a-repo/) and
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).


## Finding contributions to work on
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.

Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start.

## Code of Conduct

This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct).
For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact
[email protected] with any additional questions or comments.

<[email protected]> with any additional questions or comments.

## Security issue notifications
If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.

## Licensing

See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
See the [LICENSE](https://github.com/awslabs/automated-security-helper/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.

We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.
172 changes: 152 additions & 20 deletions docs/content/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,13 @@

- [ASH; The *A*utomated *S*ecurity *H*elper](#ash-the-automated-security-helper)
- [Description](#description)
- [ASH change advisory](#ash-change-advisory)
- [Getting started with ASH](#getting-started-with-ash)
- [Supported frameworks](#supported-frameworks)
- [Prerequisites](#prerequisites)
- [Getting Started](#getting-started)
- [Getting Started - Linux or MacOS](#getting-started---linux-or-macos)
- [Getting Started - Windows](#getting-started---windows)
- [Cloud9 Quickstart Guide](#cloud9-quickstart-guide)
- [Using `ash` with `pre-commit`](#using-ash-with-pre-commit)
- [Examples](#examples)
- [Synopsis](#synopsis)
- [FAQ](#faq)
Expand All @@ -24,18 +28,6 @@ by providing a fast and easy tool to conduct preliminary security check as earl
- It uses light, open source tools to maintain its flexibility and ability to run from anywhere.
- ASH is cloning and running different open-source tools, such as: git-secrets, bandit, Semgrep, Grype, Syft, nbconvert, npm-audit, checkov, cdk-nag and cfn-nag. Please review the tools [LICENSE](license) before usage.

### ASH change advisory

We are currently working on a re-architecture of ASH targeting a single-container architecture as well as documentation to go along with it.

## Getting started with ASH

For details on using ASH, please see the relevant guides below:

* [Running ASH Locally](./tutorials/running-ash-locally.md)
* [Running ASH in CI](./tutorials/running-ash-in-ci.md)
* [Cloud9 Quickstart](./tutorials/cloud9-quickstart.md)

## Supported frameworks

The security helper supports the following vectors:
Expand Down Expand Up @@ -75,6 +67,122 @@ The security helper supports the following vectors:
* Dockerfile
* **[checkov](https://github.com/bridgecrewio/checkov)**

## Prerequisites

To start using `ash` please make sure to install and configure the following:

- Tools installed to run Linux containers, such as [Finch](https://github.com/runfinch/finch), [Rancher Desktop](https://rancherdesktop.io/), [Podman Desktop](https://podman-desktop.io/), or [Docker Desktop](https://docs.docker.com/get-docker/).
- This can be any command-line interface (CLI) + container engine combination; there is nothing in ASH that requires a specific container runtime.
- If on Windows, you will also likely need Windows Subsystem for Linux (WSL) installed as a prerequisite for the listed container engine tools. Please see the specific instructions for the tool of choice regarding Windows-specific prerequisites.

## Getting Started

### Getting Started - Linux or MacOS

Clone the git repository into a folder. For example:

```bash
# Set up some variables
REPO_DIR="${HOME}"/Documents/repos/reference
REPO_NAME=automated-security-helper

# Create a folder to hold reference git repositories
mkdir -p ${REPO_DIR}

# Clone the repository into the reference area
git clone https://github.com/awslabs/automated-security-helper "${REPO_DIR}/${REPO_NAME}"

# Set the repo path in your shell for easier access
#
# Add this (and the variable settings above) to
# your ~/.bashrc, ~/.bash_profile, ~/.zshrc, or similar
# start-up scripts so that the ash tool is in your PATH
# after re-starting or starting a new shell.
#
export PATH="${PATH}:${REPO_DIR}/${REPO_NAME}"

# Execute the ash tool
ash --version
```

### Getting Started - Windows

**ASH** uses containers, `bash` shell scripts, and multiple background processes running in parallel to run the multiple
source code security scanning tools that it uses. Because of this, running `ash` from either a `PowerShell` or `cmd`
shell on Windows is not possible. Furthermore, due to reliance on running containers, usually with Docker Desktop
when running on Windows, there is an implicit dependency on having installed, configured, and operational a
Windows Subsystem for Linux (WSL) 2 environment on the Windows machine where `ash` will be run.

To use `ash` on Windows:

- Install, configure, and test the [WSL 2 environment on Windows](https://learn.microsoft.com/en-us/windows/wsl/install)
- Install, configure, and test [Docker Desktop for Windows](https://docs.docker.com/desktop/install/windows-install/), using the WSL 2 environment
- Use the [Windows Terminal](https://learn.microsoft.com/en-us/windows/terminal/install) program and open a command-line window to interact with the WSL 2 environment
- Install and/or update the `git` client in the WSL 2 environment. This should be pre-installed, but you may need to update the version
using the `apt-get update` command.

Once the WSL2 command-line window is open, follow the steps above in [Getting Started - Linux or MacOS](#getting-started---linux-or-macos)
to install and run `ash` in WSL 2 on the Windows machine.

To run `ash`, open a Windows Terminal shell into the WSL 2 environment and use that command-line shell to run the `ash` command.

**Note**: when working this way, be sure to `git clone` any git repositories to be scanned into the WSL 2 filesystem.
Results are un-predictable if repositories or file sub-trees in the Windows filesystem are scanned using `ash`
that is running in the WSL 2 environment.

**Tip**: If you are using Microsoft VSCode for development, it is possible to configure a "remote" connection
[using VSCode into the WSL2 environment](https://learn.microsoft.com/en-us/windows/wsl/tutorials/wsl-vscode).
By doing this, you can host your git repositories in WSL 2 and still
work with them as you have in the past when they were in the Windows filesystem of your Windows machine.

### Cloud9 Quickstart Guide

Follow the instruction in the [quickstart page](/quickstart/README.md) to deploy an AWS Cloud9 Environment with ASH pre-installed.

## Using `ash` with `pre-commit`

The `ash` tool can be used interactively on a workstation or run using the [`pre-commit`](https://pre-commit.com/) command.
If `pre-commit` is used to run `ash`, then the `pre-commit` processing takes care of installing
a copy of the `ash` git repository and setting up to run the `ash` program from that installed
repository. Using `pre-commit` still requires usage of WSL 2 when running on Windows.

Using `ash` as a [`pre-commit`](https://pre-commit.com/) hook enables development teams to use the `ash` tool
in two ways. First, developers can use `ash` as a part of their local development process on whatever
development workstation or environment they are using. Second, `ash` can be run in a build automation stage
by running `pre-commit run --hook-stage manual ash` in build automation stage.
When using `pre-commit`, run the `pre-commit` commands while in a folder/directory within the git repository that is
configured with `pre-commit` hooks.

Refer to the [pre-commit-hooks](./.pre-commit-hooks.yaml) file for information about the `pre-commit`
hook itself.

To configure a git repository to use the `ash` hook, start with the following `pre-commit-config` configuration:

```yaml
- repo: [email protected]:awslabs/automated-security-helper.git
rev: '1.1.0-e-01Dec2023' # update with the latest tagged version in the repository
hooks:
- id: ash
name: scan files using ash
stages: [ manual ]
# uncomment the line below if using "finch" on MacOS
# args: [ "-f" ]
```

Once the `.pre-commit-hooks.yaml` file is updated, the `ash` tool can be run using the following command:

```bash
pre-commit run --hook-stage manual ash
```

Results from the run of the `ash` tool can be found in the `aggregated_results.txt` file
the `--output-dir` folder/directory.

When ASH converts CloudFormation files into CDK and runs cdk-nag on them,
the output of the cdk-nag check results are preserved in a 'ash_cf2cdk_output'
folder/directory under `--output-dir` after the ASH scan is run. This folder/directory is
in addition to the `aggregated_results.txt` file found in `--output-dir`.

## Examples

```bash
Expand Down Expand Up @@ -117,25 +225,49 @@ OPTIONS:
-c | --no-color Don't print colorized output.
-s | --single-process Run ash scanners serially rather than as separate, parallel sub-processes.
-o | --oci-runner Use the specified OCI runner instead of docker to run the containerized tools.
For more information please visit https://github.com/awslabs/automated-security-helper
```

## FAQ
Please see the [FAQs](./faq.md) page.

- Q: How to run `ash` on a Windows machine

A: ASH on a windows machine

- Install a Windows Subsystem for Linux (WSL) 2 environment with a [Ubuntu distribution](https://docs.microsoft.com/en-us/windows/wsl/install). Be sure to use the WSL 2.
- Install Docker Desktop for windows and activate the [integration the WSL 2](https://docs.docker.com/desktop/windows/wsl/)
- Clone this git repo from a windows terminal via VPN (while in vpn it'll not connect to the repo directly from Ubuntu WSL 2).
- Execute the helper tool from the folder downloaded in the previous step from the Ubuntu WSL.

- Q: How to run `ash` in a Continuous Integration/Continuous Deployment (CI/CD) pipline?

A: Check the [ASH Pipeline solution](https://github.com/aws-samples/automated-security-helper-pipeline)

- Q: How to run `ash` with [finch](https://aws.amazon.com/blogs/opensource/introducing-finch-an-open-source-client-for-container-development/)
or another Open Container Initiative (OCI) compatible tool.

A: You can configure the OCI compatible tool to use with by using the environment variable `ASH_OCI_RUNNER`

- Q: How to exclude files from scanning.

A: `ash` will scan all the files in the folder specified in `--source-dir`, or the current directory if invoked without parameters. If the folder is a git repository,
then `ash` will use the exclusions in your `.gitignore` configuration file. If you want to exclude any specific folder, it **must** be added to your git ignore list before invoking `ash`.

- Q: `ash` reports there are not files to scan or you see a message stating `warning: You appear to have cloned an empty repository.`

A: Ensure you're running ASH inside the folder you intend to scan or using the `--source-dir` parameter. If the folder where the files reside is part of a git repository, ensure the files are added (committed) before running ASH.

## Feedback

Create an issue [here](https://github.com/awslabs/automated-security-helper/issues).

## Contributing

Please see the [Contributing](./contributing.md) page.
See [CONTRIBUTING](contributing.md#contributing-guidelines) for information on how to contribute to this project.

## Security

See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.
See [CONTRIBUTING](contributing.md#security-issue-notifications) for more information.

## License

This library is licensed under the Apache 2.0 License. See the LICENSE file.
This library is licensed under the Apache 2.0 License. See the LICENSE file.
Loading

0 comments on commit 290ae03

Please sign in to comment.