fds-pyrosim-examples

Collection of PyroSim models created by Thunderhead Engineering

• Atrium with fans

Run FDS in parallel using Open MPI

FDS 6.3.X requires Open MPI 1.8.4 to run in parallel mode. NIST provides information about requirements as well as pre-compiled binaries. This was tested on Ubuntu 14.04 x64 LTS.

Clone this repository. This repository ships with FDS 6.3.0 and pre-compiled OpenMPI 1.8.4.

$ git clone https://github.com/saschagottfried/fds-pyrosim-examples

Extract pre-compiled archive in dist folder.

$ cd fds-pyrosim-examples
$ cd dist
$ tar -xvzf openmpi_1.8.4_linux_64.tar.gz
$ cd ..

Add FDS libraries in bin folder to shared libraries environment. Adjust absolute path to your environment.

$ export LD_LIBRARY_PATH=bin/LIB64

Increase stack size. Otherwise a larger FDS model runs into a segmentation fault.

$ ulimit -s unlimited

Limit number of OpenMP threads

$ export OMP_NUM_THREADS=1

Run model with FDS 6.3.0

Since the given FDS model has 4 meshes we run the simulation using the instruction shown below.

$ dist/openmpi_64/bin/mpiexec -np 4 bin/fds atrium_with_fans.fds 
 Mesh   1 is assigned to MPI Process   0
 Mesh   2 is assigned to MPI Process   1
 Mesh   3 is assigned to MPI Process   2
 Mesh   4 is assigned to MPI Process   3
 OpenMP thread   0 of   0 assigned to MPI process   0 of   3
 OpenMP thread   0 of   0 assigned to MPI process   1 of   3
 OpenMP thread   0 of   0 assigned to MPI process   3 of   3
 OpenMP thread   0 of   0 assigned to MPI process   2 of   3

 Fire Dynamics Simulator

 Current Date     : November 18, 2015  22:09:05
 Version          : FDS 6.3.0
 Revision         : Git-r1-12-gf7f4148
 Revision Date    : Wed Sep 30 23:20:47 2015 -0400
 Compilation Date : Oct 01, 2015  08:23:41

 MPI Enabled; Number of MPI Processes:     4
 OpenMP Enabled; Number of OpenMP Threads:   1

 Job TITLE        : 
 Job ID string    : atrium_with_fans

 Time Step:      1,    Simulation Time:      0.06 s
 Time Step:      2,    Simulation Time:      0.11 s
 Time Step:      3,    Simulation Time:      0.17 s
 Time Step:      4,    Simulation Time:      0.22 s
 Time Step:      5,    Simulation Time:      0.28 s
 Time Step:      6,    Simulation Time:      0.34 s
 Time Step:      7,    Simulation Time:      0.39 s

Run model with FDS 6.3.1

FDS 6.3.1 is a bit less verbose but shows useful information about MPI version in use. All you have to do is provide FDS 6.3.1 to bin folder.

$ dist/openmpi_64/bin/mpiexec -np 4 bin/fds atrium_with_fans.fds 
 Mesh      1 is assigned to MPI Process      0
 Mesh      2 is assigned to MPI Process      1
 Mesh      3 is assigned to MPI Process      2
 Mesh      4 is assigned to MPI Process      3
 Completed Initialization Step  1
 Completed Initialization Step  2
 Completed Initialization Step  3
 Completed Initialization Step  4

 Fire Dynamics Simulator

 Current Date     : November 18, 2015  22:25:35
 Version          : FDS 6.3.1
 Revision         : Git-r4-19-g352eda9
 Revision Date    : Mon Nov 9 17:25:52 2015 -0500
 Compilation Date : Nov 09, 2015  21:38:33

 MPI Enabled; Number of MPI Processes:     4
 OpenMP Enabled; Number of OpenMP Threads:   1

 MPI version: 3.0
 MPI library version: Open MPI v1.8.4, package: Open MPI gforney@blaze Distribution, ident: 1.8.4, repo rev: v1.8.3-330-g0344f04, Dec 19, 2014

 Job TITLE        : 
 Job ID string    : atrium_with_fans

 Time Step:      1, Simulation Time:      0.06 s
 Time Step:      2, Simulation Time:      0.11 s
 Time Step:      3, Simulation Time:      0.17 s
 Time Step:      4, Simulation Time:      0.22 s
 Time Step:      5, Simulation Time:      0.28 s

Run model with FDS 6.4.0

Extract FDS 6.4.0 in dist folder to bin folder. Run the simulation with this command

$ dist/openmpi_64/bin/mpiexec -np 4 bin/fds atrium_with_fans.fds
 Mesh      1 is assigned to MPI Process      0
 Mesh      2 is assigned to MPI Process      1
 Mesh      3 is assigned to MPI Process      2
 Mesh      4 is assigned to MPI Process      3
 Completed Initialization Step  1
 Completed Initialization Step  2
 Completed Initialization Step  3
 Completed Initialization Step  4

 Fire Dynamics Simulator

 Current Date     : May 23, 2016  21:59:30
 Version          : FDS 6.4.0
 Revision         : Git-r10-48-g5996289
 Revision Date    : Thu Apr 7 11:24:39 2016 -0400
 Compilation Date : Apr 07, 2016  11:43:03

 MPI Enabled; Number of MPI Processes:     4
 OpenMP Enabled; Number of OpenMP Threads:   1

 MPI version: 3.0
 MPI library version: Open MPI v1.8.4, package: Open MPI gforney@blaze Distribution, ident: 1.8.4, repo rev: v1.8.3-330-g0344f04, Dec 19, 2014

 Job TITLE        : 
 Job ID string    : atrium_with_fans

 Time Step:      1, Simulation Time:      0.06 s
 Time Step:      2, Simulation Time:      0.11 s
 Time Step:      3, Simulation Time:      0.17 s
 Time Step:      4, Simulation Time:      0.22 s

Validate working Open MPI environment.

This only works if you extract pre-compiled Open MPI to /shared folder.

$ /shared/openmpi_64/bin/mpiexec -version
mpiexec (OpenRTE) 1.8.4

Report bugs to http://www.open-mpi.org/community/help/