Update github CI workflow file to test compilation of oneMKL with hip…

…SYCL on Ubuntu 18.04 and 20.04

Squashed commit of the following:

commit 5ee7dd1
Author: Nils Friess <[email protected]>
Date:   Thu Jun 30 11:44:57 2022 +0200

    Also test on Ubuntu 18.04 (requires installing boost from source)

commit 4cfe3aa
Author: Nils Friess <[email protected]>
Date:   Thu Jun 23 22:49:30 2022 +0200

    Renamed workflow file

commit 911b977
Author: Nils Friess <[email protected]>
Date:   Thu Jun 23 22:22:39 2022 +0200

    Run also on pull requests

commit 18670fa
Author: Nils Friess <[email protected]>
Date:   Thu Jun 23 22:08:56 2022 +0200

    Revert "Also test on Ubuntu 22.04 LTS"

    This reverts commit 1912951.
    ROCm currently cannot be installed on 22.04.

commit 1912951
Author: Nils Friess <[email protected]>
Date:   Thu Jun 23 22:02:05 2022 +0200

    Also test on Ubuntu 22.04 LTS

commit 2aca44b
Author: Nils Friess <[email protected]>
Date:   Thu Jun 23 22:01:41 2022 +0200

    Format and remove unused flags etc.

commit c976018
Author: Nils Friess <[email protected]>
Date:   Tue Jun 21 11:13:31 2022 +0200

    Fix typo in CMake command

commit 716a3ff
Author: Nils Friess <[email protected]>
Date:   Tue Jun 21 11:04:53 2022 +0200

    Update apt sources before installing boost

commit 78c0b74
Author: Nils Friess <[email protected]>
Date:   Tue Jun 21 10:51:38 2022 +0200

    Remove commands to install tbb and mkl since they are not necessary

commit c3b1abd
Author: Nils Friess <[email protected]>
Date:   Mon Jun 20 14:25:29 2022 +0200

    Run cmake install as sudo

commit 7f4ddbd
Author: Nils Friess <[email protected]>
Date:   Mon Jun 20 14:15:49 2022 +0200

    Add path to lapack in cmake command

commit 33d23a5
Author: Nils Friess <[email protected]>
Date:   Mon Jun 20 14:12:42 2022 +0200

    Install LAPACK from source

commit f87e040
Author: Nils Friess <[email protected]>
Date:   Mon Jun 20 13:53:46 2022 +0200

    Add command to build oneMKL

commit 159db62
Author: Nils Friess <[email protected]>
Date:   Mon Jun 20 13:53:18 2022 +0200

    Fix typo in cmake flag

commit 4d50f8b
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 13:58:20 2022 +0200

    Add command to install CBLAS (for simplicity we install libatlas)

commit 3527cd8
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 13:43:03 2022 +0200

    Add command to install cblas

commit 4a6906c
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 13:28:53 2022 +0200

    Add path to cmake config of hsa-runtime64 as env variable

commit f6f62a9
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 13:15:35 2022 +0200

    Add path to amd_comgr cmake config as env variable

commit 6fd04a0
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 12:52:27 2022 +0200

    Add path to cmake config for AMDDeviceLibs as env variable

commit d9fc104
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 12:41:25 2022 +0200

    Add path to hip cmake config as environment variable

commit 5125368
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 12:32:52 2022 +0200

    Install rocblas together with rocm

commit 483bdc6
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 12:23:05 2022 +0200

    Add rocblas_DIR to cmake command

commit fa58d4b
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 12:13:32 2022 +0200

    Add rocBLAS CMake config path as environment variable

commit 52065e9
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 11:00:08 2022 +0200

    Fix hipSYCL build paths

commit bd79ccf
Author: Nils Friess <[email protected]>
Date:   Sun Jun 19 10:45:27 2022 +0200

    Add commands for building oneMKL

commit 4b44a68
Merge: be11ced 6b95323
Author: Nils Friess <[email protected]>
Date:   Wed Jun 8 23:26:20 2022 +0200

    Merge branch 'github-ci-test' of github.com:nilsfriess/oneMKL into github-ci-test

commit be11ced
Author: Nils Friess <[email protected]>
Date:   Wed Jun 8 23:23:11 2022 +0200

    Add hipsycl build (including setup for CUDA and ROCm)

commit 6b95323
Author: Nils Friess <[email protected]>
Date:   Wed Jun 8 23:20:41 2022 +0200

    Remove sudo and add hipsycl build

commit 8378aba
Author: Nils Friess <[email protected]>
Date:   Wed Jun 8 23:03:34 2022 +0200

    Add github workflow test file

.github/workflows/linux.yml

@@ -1,17 +1,16 @@
-name: Linux build and test
+name: Build tests with hipSYCL
 
 on: [push, pull_request]
 
 jobs:
-  test-clang-based:
-    name: clang ${{ matrix.clang_version }}, ${{ matrix.os }}, CUDA ${{matrix.cuda}}
-    runs-on: ubuntu-20.04
+  test-with-hipsycl:
+    name: Run tests with hipSYCL
     strategy:
       matrix:
         clang_version: [13]
         rocm_version: ['5.1.1']
-        os: [ubuntu-20.04]
-        cuda: [11.0]
+        os: [ubuntu-20.04, ubuntu-18.04]
+    runs-on: ${{matrix.os}}
     steps:
     - uses: actions/checkout@v2
       with:
@@ -22,90 +21,72 @@ jobs:
         wget -q -O - https://repo.radeon.com/rocm/rocm.gpg.key | sudo apt-key add -
         echo 'deb [arch=amd64] https://repo.radeon.com/rocm/apt/${{matrix.rocm_version}}/ ubuntu main' | sudo tee /etc/apt/sources.list.d/rocm.list
         sudo apt update
-        sudo apt install rocm-dev
-    - name: install CUDA 11.0
-      if: matrix.cuda == 11.0
-      run: |
-        mkdir -p /opt/hipSYCL/cuda
-        wget -q -O cuda.sh http://developer.download.nvidia.com/compute/cuda/11.0.2/local_installers/cuda_11.0.2_450.51.05_linux.run
-        sudo sh ./cuda.sh --override --silent --toolkit --no-man-page --no-drm --no-opengl-libs --installpath=/opt/hipSYCL/cuda || true
-        echo "CUDA Version 11.0.0" | sudo tee /opt/hipSYCL/cuda/version.txt
+        sudo apt install rocm-dev rocblas
     - name: install LLVM
       run: |
         wget https://apt.llvm.org/llvm.sh
         chmod +x llvm.sh
         sudo ./llvm.sh ${{matrix.clang_version}}
         sudo apt install libclang-${{matrix.clang_version}}-dev clang-tools-${{matrix.clang_version}} libomp-${{matrix.clang_version}}-dev
-    - name: install boost (from source)
+    - name: install boost from apt
+      if: matrix.os == 'ubuntu-20.04'
+      run: |
+        sudo apt update
+        sudo apt install libboost-all-dev
+    - name: install boost from source
       if: matrix.os == 'ubuntu-18.04'
       run: |
+        cd
         wget -q https://boostorg.jfrog.io/artifactory/main/release/1.70.0/source/boost_1_70_0.zip
         unzip boost_1_70_0.zip
         cd boost_1_70_0
         ./bootstrap.sh --prefix=/usr
-        sudo ./b2 link=static,shared install -j$(nproc) || true
-        cd ..
-    - name: install boost (from apt)
-      if: matrix.os == 'ubuntu-20.04'
-      run: |
-        sudo apt install libboost-all-dev
+        ./b2
+        sudo ./b2 install -j4
     - name: build hipSYCL
       run: |
+        cd
         git clone https://github.com/illuhad/hipSYCL.git
         cd hipSYCL
         mkdir build && cd build
-        cmake -DCMAKE_CXX_COMPILER=/usr/bin/clang++-${{matrix.clang_version}} -DCLANG_EXECUTABLE_PATH=/usr/bin/clang++-${{matrix.clang_version}} -DLLVM_DIR=/usr/lib/llvm-${{matrix.clang_version}}/cmake -DWITH_CUDA_BACKEND=ON -DWITH_ROCM_BACKEND=ON -DWITH_LEVEL_ZERO_BACKEND=OFF -DCMAKE_INSTALL_PREFIX=`pwd`/install -DCUDA_TOOLKIT_ROOT_DIR=/opt/hipSYCL/cuda -DROCM_PATH=/opt/rocm ..
-        make -j2 install
-        cp /opt/hipSYCL/cuda/lib64/stubs/libcuda.so `pwd`/install/lib/libcuda.so
-        cp /opt/hipSYCL/cuda/lib64/stubs/libcuda.so `pwd`/install/lib/libcuda.so.1
-    - name: build CPU tests
-      run: |
-        mkdir /home/runner/work/hipSYCL/hipSYCL/build/tests-cpu
-        cd /home/runner/work/hipSYCL/hipSYCL/build/tests-cpu
-        cmake -DHIPSYCL_TARGETS="omp" -DhipSYCL_DIR=/home/runner/work/hipSYCL/hipSYCL/build/install/lib/cmake/hipSYCL /home/runner/work/hipSYCL/hipSYCL/tests
-        make -j2
-    - name: build CUDA tests
-      run: |
-        mkdir /home/runner/work/hipSYCL/hipSYCL/build/tests-cuda
-        cd /home/runner/work/hipSYCL/hipSYCL/build/tests-cuda
-        cmake -DHIPSYCL_TARGETS="cuda:sm_60" -DhipSYCL_DIR=/home/runner/work/hipSYCL/hipSYCL/build/install/lib/cmake/hipSYCL /home/runner/work/hipSYCL/hipSYCL/tests
-        make -j2
-    - name: build ROCm tests
-      run: |
-        mkdir /home/runner/work/hipSYCL/hipSYCL/build/tests-rocm
-        cd /home/runner/work/hipSYCL/hipSYCL/build/tests-rocm
-        cmake -DHIPSYCL_PLATFORM=rocm -DHIPSYCL_GPU_ARCH=gfx906 -DhipSYCL_DIR=/home/runner/work/hipSYCL/hipSYCL/build/install/lib/cmake/hipSYCL /home/runner/work/hipSYCL/hipSYCL/tests
-        make -j2
-    - name: run CPU tests
+        cmake -DCMAKE_CXX_COMPILER=/usr/bin/clang++-${{matrix.clang_version}} \
+              -DCLANG_EXECUTABLE_PATH=/usr/bin/clang++-${{matrix.clang_version}} \
+              -DLLVM_DIR=/usr/lib/llvm-${{matrix.clang_version}}/cmake \
+              -DWITH_CUDA_BACKEND=OFF \
+              -DWITH_ROCM_BACKEND=ON \
+              -DWITH_LEVEL_ZERO_BACKEND=OFF \
+              -DCMAKE_INSTALL_PREFIX=/opt/hipSYCL \
+              -DROCM_PATH=/opt/rocm-${{matrix.rocm_version}} ..
+        make -j4 install
+    - name: install LAPACK (for CBLAS)
       run: |
-        cd /home/runner/work/hipSYCL/hipSYCL/build/tests-cpu
-        LD_LIBRARY_PATH=/home/runner/work/hipSYCL/hipSYCL/build/install/lib ./sycl_tests
-
-  test-dpcpp-based:
-    name: Test dpcpp based compilation
-    runs-on: ubuntu-20.04
-    steps:
-    - uses: actions/checkout@v2
-      with:
-        submodules: 'recursive'
-    - name: Install dpcpp
-      run: |
-        wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18236/l_BaseKit_p_2021.4.0.3422.sh
-        bash l_BaseKit_p_2021.4.0.3422.sh \
-          -a --action install  \
-          -s --eula accept  \
-          --components=\
-          intel.oneapi.lin.tbb.devel:\
-          intel.oneapi.lin.dpcpp-cpp-compiler:\
-          intel.oneapi.lin.mkl.devel \
-          --install-dir /opt/intel/ \
-          --download-dir /opt/intel/dw/ \
-          --download-cache /opt/intel/dw/ \
-          --package-path /opt/intel/intel
-
-    - name: Clone oneMKL
+        cd
+        git clone https://github.com/Reference-LAPACK/lapack.git
+        cd lapack
+        mkdir build && cd build
+        cmake -DCMAKE_INSTALL_LIBDIR=/opt/lapack -DBUILD_SHARED_LIBS=ON -DCBLAS=ON ..
+        sudo cmake --build . -j --target install
+    - name: clone and build oneMKL
+      env:
+        rocblas_DIR: /opt/rocm-${{matrix.rocm_version}}/lib/cmake/rocblas/
+        hip_DIR: /opt/rocm-${{matrix.rocm_version}}/lib/cmake/hip/
+        AMDDeviceLibs_DIR: /opt/rocm-${{matrix.rocm_version}}/lib/cmake/AMDDeviceLibs/
+        amd_comgr_DIR: /opt/rocm-${{matrix.rocm_version}}/lib/cmake/amd_comgr/
+        hsa-runtime64_DIR: /opt/rocm-${{matrix.rocm_version}}/lib/cmake/hsa-runtime64/
       run: |
+        cd
         git clone https://github.com/oneapi-src/oneMKL.git
-
-
-
+        cd oneMKL
+        mkdir build
+        cd build
+        cmake -DENABLE_CUBLAS_BACKEND=False \
+              -DENABLE_MKLGPU_BACKEND=False \
+              -DENABLE_MKLCPU_BACKEND=False \
+              -DENABLE_NETLIB_BACKEND=False \
+              -DENABLE_ROCBLAS_BACKEND=True \
+              -DTARGET_DOMAINS=blas \
+              -DHIPSYCL_TARGETS=omp\;hip:gfx906 \
+              -DONEMKL_SYCL_IMPLEMENTATION=hipSYCL \
+              -DREF_BLAS_ROOT=/opt/lapack/ ..
+        make -j8
+