From 0875c9596fae6684150379aa6556459a1dff99b2 Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Wed, 24 Jan 2024 11:49:07 -0500 Subject: [PATCH 1/6] and source files for hello world in C, C++ and fortran. add stream example test and hello world compilation --- tutorials/compilation_examples/hello.c | 6 ++++++ tutorials/compilation_examples/hello.cpp | 6 ++++++ tutorials/compilation_examples/hello.f | 3 +++ .../hello_world_compilation.yml | 12 ++++++++++++ tutorials/compilation_examples/stream.yml | 17 +++++++++++++++++ 5 files changed, 44 insertions(+) create mode 100644 tutorials/compilation_examples/hello.c create mode 100644 tutorials/compilation_examples/hello.cpp create mode 100644 tutorials/compilation_examples/hello.f create mode 100644 tutorials/compilation_examples/hello_world_compilation.yml create mode 100644 tutorials/compilation_examples/stream.yml diff --git a/tutorials/compilation_examples/hello.c b/tutorials/compilation_examples/hello.c new file mode 100644 index 000000000..293499a94 --- /dev/null +++ b/tutorials/compilation_examples/hello.c @@ -0,0 +1,6 @@ +#include + +int main() { + printf("Hello, World in C\n"); + return 0; +} diff --git a/tutorials/compilation_examples/hello.cpp b/tutorials/compilation_examples/hello.cpp new file mode 100644 index 000000000..91bdf38d9 --- /dev/null +++ b/tutorials/compilation_examples/hello.cpp @@ -0,0 +1,6 @@ +#include + +int main() { + std::cout << "Hello, World in C++" << std::endl; + return 0; +} diff --git a/tutorials/compilation_examples/hello.f b/tutorials/compilation_examples/hello.f new file mode 100644 index 000000000..c00784f93 --- /dev/null +++ b/tutorials/compilation_examples/hello.f @@ -0,0 +1,3 @@ +PROGRAM HelloWorld + PRINT *, 'Hello, World in Fortran' +END PROGRAM HelloWorld diff --git a/tutorials/compilation_examples/hello_world_compilation.yml b/tutorials/compilation_examples/hello_world_compilation.yml new file mode 100644 index 000000000..372236f81 --- /dev/null +++ b/tutorials/compilation_examples/hello_world_compilation.yml @@ -0,0 +1,12 @@ +buildspecs: + hello_world_c_cpp: + type: script + executor: generic.local.bash + description: Hello world compilation in C and C++ + compilers: + name: ['builtin_gcc'] + run: | + $BUILDTEST_CC hello.c -o hello_c + $BUILDTEST_CXX hello.cpp -o hello_cpp + ./hello_c + ./hello_cpp diff --git a/tutorials/compilation_examples/stream.yml b/tutorials/compilation_examples/stream.yml new file mode 100644 index 000000000..aeb2b82c2 --- /dev/null +++ b/tutorials/compilation_examples/stream.yml @@ -0,0 +1,17 @@ +buildspecs: + stream_openmp_c: + executor: generic.local.bash + type: script + description: "STREAM Microbenchmark C Test with OpenMP" + tags: ["benchmark"] + compilers: + name: ['(builtin_gcc)'] + default: + gcc: + cflags: -fopenmp -O2 + env: + OMP_NUM_THREADS: 8 + run: | + curl https://www.cs.virginia.edu/stream/FTP/Code/stream.c -O stream.c + $BUILDTEST_CC $BUILDTEST_CFLAGS stream.c -o stream + ./stream From 7b9ed20806f06bfeacb3c52f157d3a7600df9c01 Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Wed, 24 Jan 2024 13:24:50 -0500 Subject: [PATCH 2/6] add documentation page for compilation under "Writing Buildspecs" with example on Hello World example and STREAM example --- docs/writing_buildspecs.rst | 1 + docs/writing_buildspecs/compilation.rst | 57 +++++++++++++++++++ .../hello.c | 0 .../hello.cpp | 0 .../hello.f | 0 .../hello_world_compilation.yml | 0 .../stream.yml | 0 7 files changed, 58 insertions(+) create mode 100644 docs/writing_buildspecs/compilation.rst rename tutorials/{compilation_examples => compilation}/hello.c (100%) rename tutorials/{compilation_examples => compilation}/hello.cpp (100%) rename tutorials/{compilation_examples => compilation}/hello.f (100%) rename tutorials/{compilation_examples => compilation}/hello_world_compilation.yml (100%) rename tutorials/{compilation_examples => compilation}/stream.yml (100%) diff --git a/docs/writing_buildspecs.rst b/docs/writing_buildspecs.rst index 18aa0bb21..b581e76cf 100644 --- a/docs/writing_buildspecs.rst +++ b/docs/writing_buildspecs.rst @@ -9,6 +9,7 @@ Writing Buildspecs :caption: Buildspec Reference writing_buildspecs/global + writing_buildspecs/compilation writing_buildspecs/performance_checks writing_buildspecs/test_dependency writing_buildspecs/multi_executor diff --git a/docs/writing_buildspecs/compilation.rst b/docs/writing_buildspecs/compilation.rst new file mode 100644 index 000000000..8f99009fd --- /dev/null +++ b/docs/writing_buildspecs/compilation.rst @@ -0,0 +1,57 @@ +Compilation Example +================== + +Hello World Example +------------------- + +In this section, we will show to compile source code with compiler and compiler flags. To get started, +let's start with a simple hello world example we have available in C and C++ as shown below + +.. literalinclude:: ../../tutorials/compilation/hello_world.c + :language: c + :linenos: + +.. literalinclude:: ../../tutorials/compilation/hello_world.cpp + :language: c++ + :linenos: + +Shown below is an example buildspec file that will compile the above source code with the gcc compiler. +The ``compilers`` section is used to specify the compiler to use that is selected via the ``name`` property which applies +a regular expression to search for compiler. The compilers are defined in buildtest configuration file see :ref:`compilers` +for more details. + +.. literalinclude:: ../../tutorials/compilation/hello_world_buildspec.yml + :language: yaml + :linenos: + :emphasize-lines: 6-7, 9-10 + +Buildtest will define environment variables like ``BUILDTEST_CC`` and ``BUILDTEST_CXX`` that point to compiler wrapper +for the selected compiler. + +Let's try to run the code and inspect the test output and test file. + +.. dropdown:: ``buildtest build -b tutorials/compilation_examples/hello_world_buildspec.yml`` + + .. command-output:: buildtest build -b tutorials/compilation_examples/hello_world_buildspec.yml + + .. command-output:: buildtest inspect query -o -t hello_world_c_cpp + +STREAM Benchmark Example +------------------------ + +In this next section, we will run the `STREAM `_ memory benchmark. In order to run this example, +we will need to download the source code and compile the source code. We will use the ``curl`` command to download the source code. The +``default`` section is used to specify default compiler settings for compiler, this may include compiler options and environment variables. +The `cflags` option is responsible for setting C compiler flags which is set to environment variable ``BUILDTEST_CFLAGS`` that we can access in +the ``run`` section. The ``env`` section is used to set environment variables that are used in the ``run`` section. + +.. literalinclude:: ../../tutorials/compilation/stream.yml + :language: c + :linenos: + :emphasize-lines: 9-13,16 + +.. dropdown:: ``buildtest build -b tutorials/compilation_examples/stream.yml`` + + .. command-output:: buildtest build -b tutorials/compilation_examples/stream.yml + + .. command-output:: buildtest inspect query -t stream_openmp_c \ No newline at end of file diff --git a/tutorials/compilation_examples/hello.c b/tutorials/compilation/hello.c similarity index 100% rename from tutorials/compilation_examples/hello.c rename to tutorials/compilation/hello.c diff --git a/tutorials/compilation_examples/hello.cpp b/tutorials/compilation/hello.cpp similarity index 100% rename from tutorials/compilation_examples/hello.cpp rename to tutorials/compilation/hello.cpp diff --git a/tutorials/compilation_examples/hello.f b/tutorials/compilation/hello.f similarity index 100% rename from tutorials/compilation_examples/hello.f rename to tutorials/compilation/hello.f diff --git a/tutorials/compilation_examples/hello_world_compilation.yml b/tutorials/compilation/hello_world_compilation.yml similarity index 100% rename from tutorials/compilation_examples/hello_world_compilation.yml rename to tutorials/compilation/hello_world_compilation.yml diff --git a/tutorials/compilation_examples/stream.yml b/tutorials/compilation/stream.yml similarity index 100% rename from tutorials/compilation_examples/stream.yml rename to tutorials/compilation/stream.yml From ef7e6b8b3135dca48442bee1baf9020d19e07f27 Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Wed, 24 Jan 2024 13:45:08 -0500 Subject: [PATCH 3/6] fix typos in file path --- docs/writing_buildspecs/compilation.rst | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/docs/writing_buildspecs/compilation.rst b/docs/writing_buildspecs/compilation.rst index 8f99009fd..b9268a84d 100644 --- a/docs/writing_buildspecs/compilation.rst +++ b/docs/writing_buildspecs/compilation.rst @@ -7,13 +7,11 @@ Hello World Example In this section, we will show to compile source code with compiler and compiler flags. To get started, let's start with a simple hello world example we have available in C and C++ as shown below -.. literalinclude:: ../../tutorials/compilation/hello_world.c +.. literalinclude:: ../tutorials/compilation/hello_world.c :language: c - :linenos: -.. literalinclude:: ../../tutorials/compilation/hello_world.cpp +.. literalinclude:: ../tutorials/compilation/hello_world.cpp :language: c++ - :linenos: Shown below is an example buildspec file that will compile the above source code with the gcc compiler. The ``compilers`` section is used to specify the compiler to use that is selected via the ``name`` property which applies @@ -22,7 +20,6 @@ for more details. .. literalinclude:: ../../tutorials/compilation/hello_world_buildspec.yml :language: yaml - :linenos: :emphasize-lines: 6-7, 9-10 Buildtest will define environment variables like ``BUILDTEST_CC`` and ``BUILDTEST_CXX`` that point to compiler wrapper @@ -30,9 +27,9 @@ for the selected compiler. Let's try to run the code and inspect the test output and test file. -.. dropdown:: ``buildtest build -b tutorials/compilation_examples/hello_world_buildspec.yml`` +.. dropdown:: ``buildtest build -b tutorials/compilation/hello_world_buildspec.yml`` - .. command-output:: buildtest build -b tutorials/compilation_examples/hello_world_buildspec.yml + .. command-output:: buildtest build -b tutorials/compilation/hello_world_buildspec.yml .. command-output:: buildtest inspect query -o -t hello_world_c_cpp @@ -46,12 +43,11 @@ The `cflags` option is responsible for setting C compiler flags which is set to the ``run`` section. The ``env`` section is used to set environment variables that are used in the ``run`` section. .. literalinclude:: ../../tutorials/compilation/stream.yml - :language: c - :linenos: + :language: yaml :emphasize-lines: 9-13,16 -.. dropdown:: ``buildtest build -b tutorials/compilation_examples/stream.yml`` +.. dropdown:: ``buildtest build -b tutorials/compilation/stream.yml`` - .. command-output:: buildtest build -b tutorials/compilation_examples/stream.yml + .. command-output:: buildtest build -b tutorials/compilation/stream.yml .. command-output:: buildtest inspect query -t stream_openmp_c \ No newline at end of file From c99e5a869a942a78477ac652ea4d3bd43955416a Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Wed, 24 Jan 2024 16:04:28 -0500 Subject: [PATCH 4/6] fix documentation issues with path names add curl to .readthedocs.yaml to install curl package in image --- .readthedocs.yaml | 1 + docs/writing_buildspecs/compilation.rst | 22 ++++++++++++++-------- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 11e325ff6..7b830fc61 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -14,6 +14,7 @@ build: apt_packages: - iputils-ping - build-essential + - curl - gfortran - csh - zsh diff --git a/docs/writing_buildspecs/compilation.rst b/docs/writing_buildspecs/compilation.rst index b9268a84d..5ba337da1 100644 --- a/docs/writing_buildspecs/compilation.rst +++ b/docs/writing_buildspecs/compilation.rst @@ -1,16 +1,16 @@ Compilation Example -================== +==================== Hello World Example -------------------- +-------------------- In this section, we will show to compile source code with compiler and compiler flags. To get started, let's start with a simple hello world example we have available in C and C++ as shown below -.. literalinclude:: ../tutorials/compilation/hello_world.c +.. literalinclude:: ../tutorials/compilation/hello.c :language: c -.. literalinclude:: ../tutorials/compilation/hello_world.cpp +.. literalinclude:: ../tutorials/compilation/hello.cpp :language: c++ Shown below is an example buildspec file that will compile the above source code with the gcc compiler. @@ -18,12 +18,12 @@ The ``compilers`` section is used to specify the compiler to use that is selecte a regular expression to search for compiler. The compilers are defined in buildtest configuration file see :ref:`compilers` for more details. -.. literalinclude:: ../../tutorials/compilation/hello_world_buildspec.yml +.. literalinclude:: ../tutorials/compilation/hello_world_compilation.yml :language: yaml :emphasize-lines: 6-7, 9-10 -Buildtest will define environment variables like ``BUILDTEST_CC`` and ``BUILDTEST_CXX`` that point to compiler wrapper -for the selected compiler. +Buildtest will define environment variables like ``BUILDTEST_CC`` and ``BUILDTEST_CXX`` that point to the C and C++ +compiler wrapper for the selected compiler. Let's try to run the code and inspect the test output and test file. @@ -33,6 +33,12 @@ Let's try to run the code and inspect the test output and test file. .. command-output:: buildtest inspect query -o -t hello_world_c_cpp +Please note that the compiler definition for ``builtin_gcc`` is a canonical name to reference to system GNU +compiler that is set to `/usr/bin/gcc`, `/usr/bin/g++`. The compiler details can be extracted from the configuration +file via ``buildtest config compilers list`` command. Shown below is the YAML output of the compiler details. + +.. command-output:: buildtest config compilers list --yaml + STREAM Benchmark Example ------------------------ @@ -42,7 +48,7 @@ we will need to download the source code and compile the source code. We will us The `cflags` option is responsible for setting C compiler flags which is set to environment variable ``BUILDTEST_CFLAGS`` that we can access in the ``run`` section. The ``env`` section is used to set environment variables that are used in the ``run`` section. -.. literalinclude:: ../../tutorials/compilation/stream.yml +.. literalinclude:: ../tutorials/compilation/stream.yml :language: yaml :emphasize-lines: 9-13,16 From 5502219c599fda175e78265dccb27dac181b55ac Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Wed, 24 Jan 2024 16:15:26 -0500 Subject: [PATCH 5/6] fix issue in curl command its '-o' for output file fix file path issues in compilation.rst --- docs/writing_buildspecs/compilation.rst | 4 ++-- tutorials/compilation/stream.yml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/writing_buildspecs/compilation.rst b/docs/writing_buildspecs/compilation.rst index 5ba337da1..31c0ef475 100644 --- a/docs/writing_buildspecs/compilation.rst +++ b/docs/writing_buildspecs/compilation.rst @@ -27,9 +27,9 @@ compiler wrapper for the selected compiler. Let's try to run the code and inspect the test output and test file. -.. dropdown:: ``buildtest build -b tutorials/compilation/hello_world_buildspec.yml`` +.. dropdown:: ``buildtest build -b tutorials/compilation/hello_world_compilation.yml`` - .. command-output:: buildtest build -b tutorials/compilation/hello_world_buildspec.yml + .. command-output:: buildtest build -b tutorials/compilation/hello_world_compilation.yml .. command-output:: buildtest inspect query -o -t hello_world_c_cpp diff --git a/tutorials/compilation/stream.yml b/tutorials/compilation/stream.yml index aeb2b82c2..8a01d6d89 100644 --- a/tutorials/compilation/stream.yml +++ b/tutorials/compilation/stream.yml @@ -12,6 +12,6 @@ buildspecs: env: OMP_NUM_THREADS: 8 run: | - curl https://www.cs.virginia.edu/stream/FTP/Code/stream.c -O stream.c + curl https://www.cs.virginia.edu/stream/FTP/Code/stream.c -o stream.c $BUILDTEST_CC $BUILDTEST_CFLAGS stream.c -o stream ./stream From e3cfb4d143c8ae171260b13bc326bf35d17ac001 Mon Sep 17 00:00:00 2001 From: Shahzeb Siddiqui Date: Thu, 25 Jan 2024 16:02:27 -0500 Subject: [PATCH 6/6] minor format changes in docs --- docs/writing_buildspecs/compilation.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/writing_buildspecs/compilation.rst b/docs/writing_buildspecs/compilation.rst index 31c0ef475..470d1ea25 100644 --- a/docs/writing_buildspecs/compilation.rst +++ b/docs/writing_buildspecs/compilation.rst @@ -34,7 +34,7 @@ Let's try to run the code and inspect the test output and test file. .. command-output:: buildtest inspect query -o -t hello_world_c_cpp Please note that the compiler definition for ``builtin_gcc`` is a canonical name to reference to system GNU -compiler that is set to `/usr/bin/gcc`, `/usr/bin/g++`. The compiler details can be extracted from the configuration +compiler that is set to **/usr/bin/gcc**, **/usr/bin/g++**. The compiler details can be extracted from the configuration file via ``buildtest config compilers list`` command. Shown below is the YAML output of the compiler details. .. command-output:: buildtest config compilers list --yaml @@ -56,4 +56,4 @@ the ``run`` section. The ``env`` section is used to set environment variables th .. command-output:: buildtest build -b tutorials/compilation/stream.yml - .. command-output:: buildtest inspect query -t stream_openmp_c \ No newline at end of file + .. command-output:: buildtest inspect query -o -t stream_openmp_c \ No newline at end of file