Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding start to batch examples page #16

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions src/docs/getting-started/submitting.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,10 +151,10 @@ Congratulations, you've submitted your first batch job on Sherlock!
Actually, quite a lot. Although you now know how to submit a simple batch job,
there are many other options and areas to explore in the next sections:

* [Data transfer][url_transfer]
* [Storage][url_storage]
* [Running jobs][url_runningjobs]

* [Data transfer][url_transfer]
* [Storage][url_storage]
* [Running jobs][url_runningjobs]
* [Batch Job Examples](/docs/user-guide/examples/batch-job-examples/)


[comment]: # (link URLs -----------------------------------------------------)
Expand Down
78 changes: 78 additions & 0 deletions src/docs/user-guide/examples/batch-job-examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
The following are several examples for batch jobs to run on the Sherlock cluster.
If you would like to contribute an example, please [share it here](https://www.github.com/stanford-rc/www.sherlock.stanford.edu/issues).

> _This post is a living post, meaning that it will be updated with more examples_

## Basic
The job below asks for 16GB and can run up to 48 hours.
```bash
#!/bin/bash
#SBATCH --job-name=allthesmallthings
#SBATCH --time=48:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=20
#SBATCH --mem=16GB
```

## Using GPUs
To modify your sbatch script for a GPU, you need to add both `--gres` and `--partition`

```bash
#!/bin/bash
#SBATCH --job-name=cron
##SBATCH --begin=now+7days
##SBATCH --dependency=singleton
#SBATCH --time=00:02:00
#SBATCH --mail-type=FAIL
#SBATCH -p gpu
#SBATCH -c 1
#SBATCH --gres gpu:1

# Load modules here
ml load gromacs/2016.3

# Run your script
/bin/bash /scratch/users/pancakes/runMe.sh
```
For more information, see the [gpu](https://www.sherlock.stanford.edu/docs/user-guide/gpu/) and [running jobs](https://www.sherlock.stanford.edu/docs/user-guide/running-jobs/) documentation.

## Long Running Jobs
Let's say you have a long running job with low memory, and the memory grows over time. Here is an example that would make sure that the job goes for 3 days. The QOS (quality of service) directive tells the job manager that the job will be running longer than a day.
```
#!/bin/bash
#SBATCH --job-name=normaljob
#SBATCH -p normal
#SBATCH --qos=long
#SBATCH --time=3-00:00:00
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=32
#SBATCH --mem=300000
ml load system devel
srun /bin/bash /scratch/users/minniemouse/myJob.sh
```

## Without a Job File
You can submit a job directly (on the command line) without a job file. The same SBATCH directives at the top become the parameter.

```bash
sbatch --job-name=$job_name -o $job_name.%j.out -e $job_name.%j.err /scratch/users/smiley/scripts/makeSmiles.sh ${file1} ${file2}
```
## Google Drive Sync

```bash
#!/bin/bash
#SBATCH --job-name=gdrive
#SBATCH --output=/home/users/julienc/logs/gdrive.out
#SBATCH -p agitler
#SBATCH --time=7-0 ## To be used with --qos=long
#SBATCH --cpus-per-task=1
#SBATCH --begin=now+48hour
#SBATCH --dependency=singleton

module load system gdrive
date

# Example of delete, where the string at the end is the file id on Google Drive
gdrive sync upload --keep-local --delete-extraneous $OAK/Shared/Potatoes/ sdhfshds3u39ur93rioneksfser
```
2 changes: 1 addition & 1 deletion src/docs/user-guide/running-jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ account.
[comment]: #_



For quick examples, see the [batch job examples page](/docs/user-guide/examples/batch-job-examples/).

## Recurring jobs

Expand Down