diff --git a/CITATION.cff b/CITATION.cff index 9e1880f03f..91bf036a1d 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -4,7 +4,7 @@ type: software authors: - given-names: "FastML Team" title: "hls4ml" -version: "v0.8.1" +version: "v1.0.0" doi: 10.5281/zenodo.1201549 repository-code: "https://github.com/fastmachinelearning/hls4ml" url: "https://fastmachinelearning.org/hls4ml" diff --git a/README.md b/README.md index 606e824d09..fd96763476 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,9 @@ If you have any questions, comments, or ideas regarding hls4ml or just want to s # Documentation & Tutorial -For more information visit the webpage: [https://fastmachinelearning.org/hls4ml/](https://fastmachinelearning.org/hls4ml/) +For more information visit the webpage: [https://fastmachinelearning.org/hls4ml/](https://fastmachinelearning.org/hls4ml/). + +For introductory material on FPGAs, HLS and ML inferences using hls4ml, check out the [video](https://www.youtube.com/watch?v=2y3GNY4tf7A&ab_channel=SystemsGroupatETHZ%C3%BCrich). Detailed tutorials on how to use `hls4ml`'s various functionalities can be found [here](https://github.com/hls-fpga-machine-learning/hls4ml-tutorial). @@ -49,8 +51,8 @@ hls_model = hls4ml.converters.keras_to_hls(config) hls4ml.utils.fetch_example_list() ``` -### Building a project with Xilinx Vivado HLS (after downloading and installing from [here](https://www.xilinx.com/products/design-tools/vivado/integration/esl-design.html)) -Note: Vitis HLS is not yet supported. Vivado HLS versions between 2018.2 and 2020.1 are recommended. +### Building a project. +We will build the project using Xilinx Vivado HLS, which can be downloaded and installed from [here](https://www.xilinx.com/products/design-tools/vivado/integration/esl-design.html). Alongside Vivado HLS, hls4ml also supports Vitis HLS, Intel HLS, Catapult HLS and has some experimental support dor Intel oneAPI. The target back-end can be changed using the argument backend when building the model. ```Python # Use Vivado HLS to synthesize the model @@ -61,15 +63,19 @@ hls_model.build() hls4ml.report.read_vivado_report('my-hls-test') ``` +# FAQ + +List of frequently asked questions and common HLS synthesis can be found [here](https://fastmachinelearning.org/hls4ml/faq.html) + # Citation If you use this software in a publication, please cite the software ```bibtex @software{fastml_hls4ml, author = {{FastML Team}}, title = {fastmachinelearning/hls4ml}, - year = 2023, + year = 2024, publisher = {Zenodo}, - version = {v0.8.1}, + version = {v1.0.0}, doi = {10.5281/zenodo.1201549}, url = {https://github.com/fastmachinelearning/hls4ml} } diff --git a/docs/advanced/auto.rst b/docs/advanced/auto.rst new file mode 100644 index 0000000000..f944a11e54 --- /dev/null +++ b/docs/advanced/auto.rst @@ -0,0 +1,22 @@ +============================= +Automatic precision inference +============================= + +The automatic precision inference (implemented in :py:class:`~hls4ml.model.optimizer.passes.infer_precision.InferPrecisionTypes`) attempts to infer the appropriate +widths for a given precision. It is initiated by setting a precision in the configuration as ``'auto'``. (Note, only layer-level precisions can be set to ``'auto'``, +not model-level.) Functions like :py:class:`~hls4ml.utils.config.config_from_keras_model`, :py:class:`~hls4ml.utils.config.config_from_onnx_model`, +and :py:class:`~hls4ml.utils.config.config_from_pytorch_model` automatically set most precisions to ``'auto'`` if the ``'name'`` granularity is used. + +.. note:: + It is recommended to pass the backend to the ``config_from_*`` functions so that they can properly extract all the configurable precisions. + +The approach taken by the precision inference is to set accumulator (the internal variable used to accumulate values in the matrix multiplications) and other precisions +to never truncate, using only the bitwidths of the inputs (not the values). This is quite conservative, especially in cases where post-training quantization is used, or +if the bit widths were set fairly loosely. The recommended action in that case is to edit the configuration and explicitly set some widths in it, potentially in an iterative process +after profiling the data. Another option is to pass a maximum precision using the ``max_precison`` parameter of the ``config_form_*`` functions. Then the automatic precision +inference will never set a bitwdith larger than the bitwidth of the ``max_precision`` or an integer part larger than the integer part of the ``max_precision`` that is passed. +(The bitwidth and integer parts of the ``max_precision`` are treated separately.) + +When manually setting bitdwidths, the accumulator can overflow, and the precision may need to be reduced. For the accumulator, it is usually a bad idea to explicitly +enable rounding or saturation modes since it dramatically increases the execution time. For other types (e.g. output types or weight types), however, rounding and saturation handling +can be enabled as needed. diff --git a/docs/advanced/bramfactor.rst b/docs/advanced/bramfactor.rst new file mode 100644 index 0000000000..37fe766060 --- /dev/null +++ b/docs/advanced/bramfactor.rst @@ -0,0 +1,42 @@ +================================== +Loading weights from external BRAM +================================== + +.. note:: + This feature is being evaluated for re-implementation. We welcome feedback from users how to make the implementation more flexible. + +``hls4ml`` can optionally store weights in BRAMs external to the design. This is supported in Vivado/Vitis and Catapult backends. It is the responsibility of the user to ensure the weights are properly loaded during the operation of the design. + +The feature works as a threshold, exposed through a ``BramFactor`` config parameter. Layers with more weights above the threshold will be exposed as BRAM interface. Consider the following code: + +.. code-block:: Python + + model = tf.keras.models.Sequential() + model.add(Dense(10, activation="relu", input_shape=(12,), name="dense_1")) + model.add(Dense(20, activation="relu", name="dense_2")) + model.add(Dense(5, activation="softmax", name="dense_3")) + model.compile(optimizer='adam', loss='mse') + + config = hls4ml.utils.config_from_keras_model(model) + config["Model"]["Strategy"] = "Resource" + config["Model"]["BramFactor"] = 100 + + hls_model = hls4ml.converters.convert_from_keras_model( + model, hls_config=config, output_dir=output_dir, io_type=io_type, backend=backend + ) + +Having set ``BramFactor=100``, only layers with more than 100 weights will be exposed as external BRAM, in this case layers ``dense_1`` and ``dense_2``. ``BramFactor`` can currently be only set at the model level. The generated code will now have weights as part of the interface. + +.. code-block:: C++ + + void myproject( + hls::stream &dense_1_input, + hls::stream &layer7_out, + model_default_t w2[120], + model_default_t w4[200] + ) { + #pragma HLS INTERFACE axis port=dense_1_input,layer7_out + #pragma HLS INTERFACE bram port=w2,w4 + ... + +When integrating the design, users can use the exposed interface to implement weight reloading scheme. diff --git a/docs/advanced/hgq.rst b/docs/advanced/hgq.rst index cf8f53d4d0..dd0faad7dc 100644 --- a/docs/advanced/hgq.rst +++ b/docs/advanced/hgq.rst @@ -9,7 +9,7 @@ High Granularity Quantization (HGQ) .. image:: https://img.shields.io/badge/arXiv-2405.00645-b31b1b.svg :target: https://arxiv.org/abs/2405.00645 -`High Granularity Quantization (HGQ) `_ is a library that performs gradient-based automatic bitwidth optimization and quantization-aware training algorithm for neural networks to be deployed on FPGAs. By laveraging gradients, it allows for bitwidth optimization at arbitrary granularity, up to per-weight and per-activation level. +`High Granularity Quantization (HGQ) `_ is a library that performs gradient-based automatic bitwidth optimization and quantization-aware training algorithm for neural networks to be deployed on FPGAs. By leveraging gradients, it allows for bitwidth optimization at arbitrary granularity, up to per-weight and per-activation level. .. image:: https://calad0i.github.io/HGQ/_images/overview.svg :alt: Overview of HGQ diff --git a/docs/advanced/model_optimization.rst b/docs/advanced/model_optimization.rst index 41132ab619..302d646023 100644 --- a/docs/advanced/model_optimization.rst +++ b/docs/advanced/model_optimization.rst @@ -124,8 +124,8 @@ Finally, optimizing Vivado DSPs is possible, given a hls4ml config: acc_optimized = accuracy_score(np.argmax(y_test, axis=1), np.argmax(y_optimized, axis=1)) print(f'Optimized Keras accuracy: {acc_optimized}') -There are two more Vivado "optimizers" - VivadoFFEstimator, aimed at reducing register utilisation and VivadoMultiObjectiveEstimator, aimed at optimising BRAM and DSP utilisation. -Note, to ensure DSPs are optimized, "unrolled" Dense multiplication must be used before synthesing HLS, by modifying the config: +There are two more Vivado "optimizers" - VivadoFFEstimator, aimed at reducing register utilization and VivadoMultiObjectiveEstimator, aimed at optimizing BRAM and DSP utilization. +Note, to ensure DSPs are optimized, "unrolled" Dense multiplication must be used before synthesizing HLS, by modifying the config: .. code-block:: Python diff --git a/docs/api/profiling.rst b/docs/advanced/profiling.rst similarity index 100% rename from docs/api/profiling.rst rename to docs/advanced/profiling.rst diff --git a/docs/command.rst b/docs/api/command.rst similarity index 97% rename from docs/command.rst rename to docs/api/command.rst index cb9d346e31..1f821b7f35 100644 --- a/docs/command.rst +++ b/docs/api/command.rst @@ -50,7 +50,7 @@ hls4ml config hls4ml config [-h] [-m MODEL] [-w WEIGHTS] [-o OUTPUT] -This creates a conversion configuration file. Visit Configuration section of the :doc:`Setup ` page for more details on how to write a configuration file. +This creates a conversion configuration file. Visit Configuration section of the :doc:`Setup <../intro/setup>` page for more details on how to write a configuration file. **Arguments** diff --git a/docs/api/concepts.rst b/docs/api/concepts.rst new file mode 100644 index 0000000000..9087470cf3 --- /dev/null +++ b/docs/api/concepts.rst @@ -0,0 +1,78 @@ +======== +Concepts +======== + +How it Works +---------------------- + +.. image:: ../img/nn_map_paper_fig_2.png + :width: 70% + :align: center + + +Consider a multilayer neural network. At each neuron in a layer :math:`m` (containing :math:`N_m` neurons), we calculate an output value (part of the output vector :math:`\mathbf{x}_m` of said layer) using the sum of output values of the previous layer multiplied by independent weights for each of these values and a bias value. An activation function is performed on the result to get the final output value for the neuron. Representing the weights as a :math:`N_m` by :math:`N_{m-1}` matrix :math:`W_{m,m-1}`, the bias values as :math:`\mathbf{b}_m`, and the activation function as :math:`g_m`, we can express this compactly as: + + +.. math:: + + \mathbf{x}_m = g_m (W_{m,m-1} \mathbf{x}_{m-1} +\mathbf{b}_m) + +With hls4ml, each layer of output values is calculated independently in sequence, using pipelining to speed up the process by accepting new inputs after an initiation interval. +The activations, if nontrivial, are precomputed. + +To ensure optimal performance, the user can control aspects of their model, principally: + + +* **Size/Compression** - Though not explicitly part of the ``hls4ml`` package, this is an important optimization to efficiently use the FPGA resources +* **Precision** - Define the :doc:`precision <../advanced/profiling>` of the calculations in your model +* **Dataflow/Resource Reuse** - Control parallel or streaming model implementations with varying levels of pipelining +* **Quantization Aware Training** - Achieve best performance at low precision with tools like QKeras, and benefit automatically during inference with ``hls4ml`` parsing of QKeras models + + +.. image:: ../img/reuse_factor_paper_fig_8.png + :width: 70% + :align: center + + +Often, these decisions will be hardware dependent to maximize performance. +Of note is that simplifying the input network must be done before using ``hls4ml`` to generate HLS code, for optimal compression to provide a sizable speedup. +Also important to note is the use of fixed point arithmetic in ``hls4ml``. +This improves processing speed relative to floating point implementations. +The ``hls4ml`` package also offers the functionality of configuring binning and output bit width of the precomputed activation functions as necessary. With respect to parallelization and resource reuse, ``hls4ml`` offers a "reuse factor" parameter that determines the number of times each multiplier is used in order to compute a layer of neuron's values. Therefore, a reuse factor of one would split the computation so each multiplier had to only perform one multiplication in the computation of the output values of a layer, as shown above. Conversely, a reuse factor of four, in this case, uses a single multiplier four times sequentially. Low reuse factor achieves the lowest latency and highest throughput but uses the most resources, while high reuse factor save resources at the expense of longer latency and lower throughput. + + +Frontends and Backends +---------------------- + +``hls4ml`` has a concept of a **frontend** that parses the input NN into an internal model graph, and a **backend** that controls +what type of output is produced from the graph. Frontends and backends can be independently chosen. Examples of frontends are the +parsers for Keras or ONNX, and examples of backends are Vivado HLS, Intel HLS, and Vitis HLS. See :ref:`Status and Features` for the +currently supported frontends and backends or the dedicated sections for each frontend/backend. + + +I/O Types +--------- + +``hls4ml`` supports multiple styles for handling data transfer to/from the network and between layers, known as the ``io_type``. + +io_parallel +^^^^^^^^^^^ +In this processing style, data is passed in parallel between the layers. Conceptually this corresponds to the C/C++ array where all elements can be accessed ay any time. This style allows for maximum parallelism and is well suited for MLP networks and small CNNs which aim for lowest latency. Due to the impact of parallel processing on resource utilization on FPGAs, the synthesis may fail for larger networks. + +io_stream +^^^^^^^^^ +As opposed to the parallel processing style, in ``io_stream`` mode data is passed one "pixel" at a time. Each pixel is an array of channels, which are always sent in parallel. This method for sending data between layers is recommended for larger CNN and RNN networks. For one-dimensional ``Dense`` layers, all the inputs are streamed in parallel as a single array. + +With the ``io_stream`` IO type, each layer is connected with the subsequent layer through first-in first-out (FIFO) buffers. +The implementation of the FIFO buffers contribute to the overall resource utilization of the design, impacting in particular the BRAM or LUT utilization. +Because the neural networks can have complex architectures generally, it is hard to know a priori the correct depth of each FIFO buffer. +By default ``hls4ml`` choses the most conservative possible depth for each FIFO buffer, which can result in a an unnecessary overutilization of resources. + +In order to reduce the impact on the resources used for FIFO buffer implementation, we have a FIFO depth optimization flow. This is described +in the :ref:`FIFO Buffer Depth Optimization` section. + + +Strategy +--------- + +**Strategy** in ``hls4ml`` refers to the implementation of core matrix-vector multiplication routine, which can be latency-oriented, resource-saving oriented, or specialized. Different strategies will have an impact on overall latency and resource consumption of each layer and users are advised to choose based on their design goals. The availability of particular strategy for a layer varies across backends, see the :doc:`Attributes <../ir/attributes>` section for a complete list of available strategies per-layer and per-backend. diff --git a/docs/api/configuration.rst b/docs/api/configuration.rst index f0db50a9b6..1bc8f0676c 100644 --- a/docs/api/configuration.rst +++ b/docs/api/configuration.rst @@ -34,20 +34,46 @@ Using hls4ml, you can quickly generate a simple configuration dictionary from a import hls4ml config = hls4ml.utils.config_from_keras_model(model, granularity='model') -This python dictionary can be edited as needed. A more advanced configuration can be generated by, for example: +This python dictionary can be edited as needed. More advanced configuration can be generated by, for example for ONNX models: .. code-block:: python import hls4ml - config = hls4ml.utils.config_from_keras_model( + config = hls4ml.utils.config_from_onnx_model( model, granularity='name', default_precision='fixed<16,6>', backend='Vitis') -This will include per-layer configuration based on the model. Including the backend is recommended because some configation options depend on the backend. Note, the precisions at the -higher granularites usually default to 'auto', which means that ``hls4ml`` will try to set it automatically. Note that higher granularity settings take precendence -over model-level settings. See :py:class:`~hls4ml.utils.config.config_from_keras_model` for more information on the various options. +for Keras models: + +.. code-block:: python + + import hls4ml + config = hls4ml.utils.config_from_keras_model( + model, + granularity='name', + default_precision='fixed<16,6>', + backend='oneAPI') + +or for PyTorch models: + +.. code-block:: python + + import hls4ml + config = hls4ml.utils.config_from_pytorch_model( + model, + granularity='name', + default_precision='fixed<16,6>', + backend='Catapult') + + +The ``name`` granularity includes per-layer configuration based on the model. A ``'name'`` granularity is generally recommended because it allows for more turning, and also because it allows +for automatic setting of precisions. The layer-level precisions with the ``'name'`` granularity default to ``'auto'``, which means that hls4ml will try to set it automatically +(see :ref:`Automatic precision inference`). Note that layer-level settings take precedence over model-level settings. A ``'name'`` granularity is required for QKeras +and QONNX model parsing. Passing the backend to these functions is recommended because some configuration options depend on the backend. See :py:class:`~hls4ml.utils.config.config_from_keras_model` +and similar for more information on the various options. Note specifically the documentation of :py:class:`~hls4ml.utils.config.config_from_pytorch_model` on how to handle differences in input data +formats between pytorch and keras (hls4ml follows keras conventions internally). One can override specific values before using the configuration: @@ -59,7 +85,7 @@ Or to set the precision of a specific layer's weight: .. code-block:: python - config['LayerName']['fc1']['Precision']['weight'] = 'ap_fixed<8,4>' + config['LayerName']['fc1']['Precision']['weight'] = 'fixed<8,4>' To better understand how the configuration hierachy works, refer to the next section for more details. @@ -75,7 +101,7 @@ Finally, one then uses the configuration to create an hls model: backend='Vitis' ) -See :py:class:`~hls4ml.converters.convert_from_keras_model` for more information on the various options. +See :py:class:`~hls4ml.converters.convert_from_keras_model` for more information on the various options. Similar functions exist for ONNX and PyTorch. ---- @@ -85,7 +111,7 @@ See :py:class:`~hls4ml.converters.convert_from_keras_model` for more information 2.1 Top Level Configuration --------------------------- -Configuration files are YAML files in hls4ml (\ ``*.yml``\ ). An example configuration file is `here `__. +One can also use YAML configuration files in hls4ml (\ ``*.yml``\ ). An example configuration file is `here `__. It looks like this: @@ -108,7 +134,7 @@ It looks like this: HLSConfig: Model: - Precision: ap_fixed<16,6> + Precision: fixed<16,6> ReuseFactor: 1 Strategy: Latency LayerType: @@ -124,7 +150,7 @@ There are a number of configuration options that you have. Let's go through the * **ProjectName**\ : the name of the HLS project IP that is produced * **KerasJson/KerasH5**\ : for Keras, the model architecture and weights are stored in a ``json`` and ``h5`` file. The path to those files are required here. We also support keras model's file obtained just from ``model.save()``. In this case you can just supply the ``h5`` file in ``KerasH5:`` field. -* **InputData/OutputPredictions**\ : path to your input/predictions of the model. If none is supplied, then hls4ml will create aritificial data for simulation. The data used above in the example can be found `here `__. We also support ``npy`` data files. We welcome suggestions on more input data types to support. +* **InputData/OutputPredictions**\ : path to your input/predictions of the model. If none is supplied, then hls4ml will create artificial data for simulation. The data used above in the example can be found `here `__. We also support ``npy`` data files. We welcome suggestions on more input data types to support. The backend-specific section of the configuration depends on the backend. You can get a starting point for the necessary settings using, for example `hls4ml.templates.get_backend('Vivado').create_initial_config()`. For Vivado backend the options are: @@ -134,13 +160,13 @@ For Vivado backend the options are: Then you have some optimization parameters for how your algorithm runs: * **IOType**\ : your options are ``io_parallel`` or ``io_stream`` which defines the type of data structure used for inputs, intermediate activations between layers, and outputs. For ``io_parallel``, arrays are used that, in principle, can be fully unrolled and are typically implemented in RAMs. For ``io_stream``, HLS streams are used, which are a more efficient/scalable mechanism to represent data that are produced and consumed in a sequential manner. Typically, HLS streams are implemented with FIFOs instead of RAMs. For more information see `here `__. * **HLSConfig**\: the detailed configuration of precision and parallelism, including: + * **ReuseFactor**\ : in the case that you are pipelining, this defines the pipeline interval or initiation interval * **ParallelizationFactor**\ : The number of output "pixels" to compute in parallel in convolutional layers. Increasing this parameter results in significant increase in resources required on the FPGA. * **Strategy**\ : Optimization strategy on FPGA, either "Latency", "Resource" or "Unrolled". If none is supplied then hl4ml uses "Latency" as default. Note that a reuse factor larger than 1 should be specified when using "resource" or "unrolled" strategy. An example of using larger reuse factor can be found `here. `__ * **PipelineStyle**\ : Set the top level pipeline style. Valid options are "auto", "pipeline" and "dataflow". If unspecified, it defaults to "auto". * **PipelineInterval**\ : Optionally override the desired initiation interval of the design. Only valid in combination with "pipeline" style. If unspecified, it is left to the compiler to decide, ideally matching the largest reuse factor of the network. - * **Precision**\ : this defines the precsion of your inputs, outputs, weights and biases. It is denoted by ``ap_fixed``\ , where ``Y`` is the number of bits representing the signed number above the binary point (i.e. the integer part), and ``X`` is the total number of bits. - Additionally, integers in fixed precision data type (\ ``ap_int``\ , where ``N`` is a bit-size from 1 to 1024) can also be used. You have a chance to further configure this more finely with per-layer configuration described below. + * **Precision**\ : this defines the precision of your inputs, outputs, weights and biases. It is denoted by ``fixed``\ , where ``Y`` is the number of bits representing the signed number above the binary point (i.e. the integer part), and ``X`` is the total number of bits. Additionally, integers in the type (\ ``int``\ , where ``N`` is a bit-size from 1 to 1024) can also be used. The format follows ``ap_fixed`` and ``ap_int`` conventions. You have a chance to further configure this more finely with per-layer configuration described below. In the per-layer configuration (but not globally) one can also use ``'auto'`` precision. 2.2 Per-Layer Configuration --------------------------- @@ -153,10 +179,10 @@ Under the ``HLSConfig`` heading, these can be set for the ``Model``\ , per ``Lay HLSConfig: Model: - Precision: ap_fixed<16,6> + Precision: fixed<16,6> ReuseFactor: 1 -This configuration use ``ap_fixed<16,6>`` for every variable and a ReuseFactor of 1 throughout. +This configuration use ``fixed<16,6>`` for every variable and a ReuseFactor of 1 throughout. Specify all ``Dense`` layers to use a different precision like this: @@ -164,13 +190,13 @@ Specify all ``Dense`` layers to use a different precision like this: HLSConfig: Model: - Precision: ap_fixed<16,6> + Precision: fixed<16,6> ReuseFactor: 1 LayerType: Dense: - Precision: ap_fixed<14,5> + Precision: fixed<14,5> -In this case, all variables in any ``Dense`` layers will be represented with ``ap_fixed<14,5>`` while any other layer types will use ``ap_fixed<16,6>``. +In this case, all variables in any ``Dense`` layers will be represented with ``fixed<14,5>`` while any other layer types will use ``fixed<16,6>``. A specific layer can be targeted like this: @@ -178,18 +204,18 @@ A specific layer can be targeted like this: HLSConfig: Model: - Precision: ap_fixed<16,6> + Precision: fixed<16,6> ReuseFactor: 16 LayerName: dense1: Precision: - weight: ap_fixed<14,2> - bias: ap_fixed<14,4> - result: ap_fixed<16,6> + weight: fixed<14,2> + bias: fixed<14,4> + result: fixed<16,6> ReuseFactor: 12 Strategy: Resource -In this case, the default model configuration will use ``ap_fixed<16,6>`` and a ``ReuseFactor`` of 16. The layer named ``dense1`` (defined in the user provided model architecture file) will instead use different precision for the ``weight``\ , ``bias``\ , and ``result`` (output) variables, a ``ReuseFactor`` of 12, and the ``Resource`` strategy (while the model default is ``Latency`` strategy. +In this case, the default model configuration will use ``fixed<16,6>`` and a ``ReuseFactor`` of 16. The layer named ``dense1`` (defined in the user provided model architecture file) will instead use different precision for the ``weight``\ , ``bias``\ , and ``result`` (output) variables, a ``ReuseFactor`` of 12, and the ``Resource`` strategy (while the model default is ``Latency`` strategy. More than one layer can have a configuration specified, e.g.: @@ -206,7 +232,7 @@ More than one layer can have a configuration specified, e.g.: dense2: ... -For more information on the optimization parameters and what they mean, you can visit the :doc:`Concepts <../concepts>` chapter. +For more information on the optimization parameters and what they mean, you can visit the :doc:`Concepts <../api/concepts>` section. ---- @@ -235,7 +261,7 @@ In your project, the file ``/firmware/.cpp`` is your top nnet::sigmoid(layer4_out, layer5_out); -You can see, for the simple 1-layer DNN, the computation (\ ``nnet::dense_latency``\ ) and activation (\ ``nnet::relu``\ /\ ``nnet::sigmoid``\ ) caluclation for each layer. For each layer, it has its own additional configuration parameters, e.g. ``config2``. +You can see, for the simple 1-layer DNN, the computation (\ ``nnet::dense_latency``\ ) and activation (\ ``nnet::relu``\ /\ ``nnet::sigmoid``\ ) calculation for each layer. For each layer, it has its own additional configuration parameters, e.g. ``config2``. In your project, the file ``/firmware/parameters.h`` stores all the configuration options for each neural network library. An example is `here `__. So for example, the detailed configuration options for an example DNN layer is: diff --git a/docs/advanced/accelerator.rst b/docs/backend/accelerator.rst similarity index 95% rename from docs/advanced/accelerator.rst rename to docs/backend/accelerator.rst index 7a79d9dbdc..187bccaa2c 100644 --- a/docs/advanced/accelerator.rst +++ b/docs/backend/accelerator.rst @@ -1,8 +1,8 @@ -========================= -VivadoAccelerator Backend -========================= +================= +VivadoAccelerator +================= -The ``VivadoAccelerator`` backend of ``hls4ml`` leverages the `PYNQ `_ software stack to easily deploy models on supported devices. +The **VivadoAccelerator** backend of ``hls4ml`` leverages the `PYNQ `_ software stack to easily deploy models on supported devices. Currently ``hls4ml`` supports the following boards: * `pynq-z2 `_ (part: ``xc7z020clg400-1``) @@ -13,7 +13,7 @@ Currently ``hls4ml`` supports the following boards: * `alveo-u280 `_ (part: ``xcu280-fsvh2892-2L-e``) but, in principle, support can be extended to `any board supported by PYNQ `_. -For the Zynq-based boards, there are two components: an ARM-based processing system (PS) and FPGA-based programmable logic (PL), with various intefaces between the two. +For the Zynq-based boards, there are two components: an ARM-based processing system (PS) and FPGA-based programmable logic (PL), with various interfaces between the two. .. image:: ../img/zynq_interfaces.png :height: 300px diff --git a/docs/backend/catapult.rst b/docs/backend/catapult.rst new file mode 100644 index 0000000000..00cf0fb98b --- /dev/null +++ b/docs/backend/catapult.rst @@ -0,0 +1,7 @@ +======== +Catapult +======== + +Support for Siemens Catapult HLS compiler has been added in ``hls4ml`` version 1.0.0. + +*TODO expand this section* diff --git a/docs/advanced/oneapi.rst b/docs/backend/oneapi.rst similarity index 58% rename from docs/advanced/oneapi.rst rename to docs/backend/oneapi.rst index ae0e0bc56b..585bfc27cb 100644 --- a/docs/advanced/oneapi.rst +++ b/docs/backend/oneapi.rst @@ -1,25 +1,24 @@ -============== -oneAPI Backend -============== +====== +oneAPI +====== -The ``oneAPI`` backend of hls4ml is designed for deploying NNs on Intel/Altera FPGAs. It will eventually -replace the ``Quartus`` backend, which should really have been called the Intel HLS backend. (The actual Quartus -program continues to be used with IP produced by the ``oneAPI`` backend.) -This section discusses details of the ``oneAPI`` backend. +The **oneAPI** backend of hls4ml is designed for deploying NNs on Intel/Altera FPGAs. It will eventually +replace the **Quartus** backend, which targeted Intel HLS. (Quartus continues to be used with IP produced by the +**oneAPI** backend.) This section discusses details of the **oneAPI** backend. -The ``oneAPI`` code uses SYCL kernels to implement the logic that is deployed on FPGAs. It naturally leads to the -accelerator style of programming. In the IP Component flow, which is currently the only flow supported, the +The **oneAPI** code uses SYCL kernels to implement the logic that is deployed on FPGAs. It naturally leads to the +accelerator style of programming. In the SYCL HLS (IP Component) flow, which is currently the only flow supported, the kernel becomes the IP, and the "host code" becomes the testbench. An accelerator flow, with easier deployment on PCIe accelerator boards, is planned to be added in the future. The produced work areas use cmake to build the projects in a style based `oneAPI-samples `_. -The standard ``fpga_emu``, ``report``, ``fpga_sim``, and ``fpga`` are supported. Additionally, ``make lib`` +The standard ``fpga_emu``, ``report``, ``fpga_sim``, and ``fpga`` make targets are supported. Additionally, ``make lib`` produces the library used for calling the ``predict`` function from hls4ml. The ``compile`` and ``build`` commands in hls4ml interact with the cmake system, so one does not need to manually use the build system, but it there if desired. -The ``oneAPI`` backend, like the ``Quartus`` backend, only implements the ``Resource`` strategy for the layers. There +The **oneAPI** backend, like the **Quartus** backend, only implements the ``Resource`` strategy for the layers. There is no ``Latency`` implementation of any of the layers. Note: currently tracing and external weights (i.e. setting BramFactor) are not supported. @@ -30,6 +29,7 @@ io_parallel and io_stream As mentioned in the :ref:`I/O Types` section, ``io_parallel`` is for small models, while ``io_stream`` is for larger models. In ``oneAPI``, there is an additional difference: ``io_stream`` implements each layer on its own ``task_sequence``. Thus, the layers run in parallel, with pipes connecting the inputs and outputs. This -is similar in style to the `dataflow` implementation on Vitis, but more explicit. On the other hand, ``io_parallel`` -always uses a single task, relying on pipelining within the task for good performance. In contrast, the Vitis -backend sometimes uses dataflow with ``io_parallel``. +is similar in style to the `dataflow` implementation on Vitis HLS, but more explicit. It is also a change +relative to the Intel HLS-based ``Quartus`` backend. On the other hand, ``io_parallel`` always uses a single task, +relying on pipelining within the task for good performance. In contrast, the Vitis backend sometimes uses dataflow +with ``io_parallel``. diff --git a/docs/backend/quartus.rst b/docs/backend/quartus.rst new file mode 100644 index 0000000000..8cde5f97b2 --- /dev/null +++ b/docs/backend/quartus.rst @@ -0,0 +1,12 @@ +======= +Quartus +======= + +.. warning:: + The **Quartus** backend is deprecated and will be removed in a future version. Users should migrate to the **oneAPI** backend. + +The **Quartus** backend of hls4ml is designed for deploying NNs on Intel/Altera FPGAs. It uses the discontinued Intel HLS compiler. The **oneAPI** backend should be preferred for new projects. +The **oneAPI** backend contains the migrated the HLS code from this backend, with significantly better io_stream support, though the **oneAPI** backend does not yet support profiling, tracing, +or the BramFactor option supported by the **Quartus** backend. Nevertheless, little or no further development is expected for this backend. + +The **Quartus** backend only implements the ``Resource`` strategy for the layers. There is no ``Latency`` implementation of any of the layers. diff --git a/docs/backend/sr.rst b/docs/backend/sr.rst new file mode 100644 index 0000000000..93a247b63d --- /dev/null +++ b/docs/backend/sr.rst @@ -0,0 +1,7 @@ +================== +SymbolicExpression +================== + +This backend can be used to implement expressions obtained through symbolic regression tools such as `PySR `_ or `SymbolNet `_. The backend targets Vivado/Vitis HLS and relies on HLS math libraries provided with a licensed installation of these tools. + +*TODO expand this section* diff --git a/docs/backend/vitis.rst b/docs/backend/vitis.rst new file mode 100644 index 0000000000..9528e89a93 --- /dev/null +++ b/docs/backend/vitis.rst @@ -0,0 +1,11 @@ +============ +Vivado/Vitis +============ + +The **Vivado** and **Vitis** backends are aimed for use with AMD/Xilinx FPGAs. The **Vivado** backend targets the discontinued ``Vivado HLS`` compiler, while +the **Vitis** backend targets the ``Vitis HLS`` compiler. Both are designed to produce IP for incorporation in ``Vivado`` designs. (See :doc:`VivadoAccelerator ` +for generating easily-deployable models with ``Vivado HLS``.) The ``Vitis`` accelerator flow is not directly supported, though HLS produced with the **Vitis** +backend can be easily incorporated into Vitis kernel. + +Users should generally use the **Vitis** backend for new designs that target AMD/Xilinx FPGAs; new ``hls4ml`` developments will not necessarily be backported to +the **Vivado** backend. diff --git a/docs/concepts.rst b/docs/concepts.rst deleted file mode 100644 index b788d5ba5d..0000000000 --- a/docs/concepts.rst +++ /dev/null @@ -1,69 +0,0 @@ -======== -Concepts -======== - -The goal of ``hls4ml`` is to provide an efficient and fast translation of machine learning models from open-source packages (like Keras and PyTorch) for training machine learning algorithms to high level synthesis (HLS) code that can then be transpiled to run on an FPGA. The resulting HLS project can be then used to produce an IP which can be plugged into more complex designs or be used to create a kernel for CPU co-processing. The user has freedom to define many of the parameters of their algorithm to best suit their needs. - -The ``hls4ml`` package enables fast prototyping of a machine learning algorithm implementation in FPGAs, -greatly reducing the time to results and giving the user intuition for how to best design a machine learning algorithm for their application while balancing performance, resource utilization and latency requirements. - -The Inspiration -=============== - -The inspiration for the creation of the ``hls4ml`` package stems from the high energy physics community at the CERN Large Hadron Collider (LHC). -While machine learning has already been proven to be extremely useful in analysis of data from detectors at the LHC, it is typically performed in an "offline" environment after the data is taken and agglomerated. -However, one of the largest problems at detectors on the LHC is that collisions, or "events", generate too much data for everything to be saved. -As such, filters called "triggers" are used to determine whether a given event should be kept. -Using FPGAs allows for significantly lower latency so machine learning algorithms can essentially be run "live" at the detector level for event selection. As a result, more events with potential signs of new physics can be preserved for analysis. - -The Solution: ``hls4ml`` -======================== - -.. image:: img/overview.jpg - - -With this in mind, let's take a look at how ``hls4ml`` helps to achieve such a goal. First, it's important to realize the architecture differences between an FPGA and a CPU or GPU. -An FPGA can be specifically programmed to do a certain task, in this case evaluate neural networks given a set of inputs, and as such can be highly optimized for the task, with tricks like pipelining and parallel evaluation. However, this means dynamic remapping while running isn't really a possibility. -FPGAs also often come at a comparatively low power cost with respect to CPUs and GPUs. This allows ``hls4ml`` to build HLS code from compressed neural networks that results in predictions on the microsecond scale for latency. -The ``hls4ml`` tool saves the time investment needed to convert a neural network to a hardware design language or even HLS code, thus allowing for rapid prototyping. - -How it Works -============= - -.. image:: img/nn_map_paper_fig_2.png - :width: 70% - :align: center - - -Consider a multilayer neural network. At each neuron in a layer :math:`m` (containing :math:`N_m` neurons), we calculate an output value (part of the output vector :math:`\mathbf{x}_m` of said layer) using the sum of output values of the previous layer multiplied by independent weights for each of these values and a bias value. An activation function is performed on the result to get the final output value for the neuron. Representing the weights as a :math:`N_m` by :math:`N_{m-1}` matrix :math:`W_{m,m-1}`, the bias values as :math:`\mathbf{b}_m`, and the activation function as :math:`g_m`, we can express this compactly as: - - -.. math:: - - \mathbf{x}_m = g_m (W_{m,m-1} \mathbf{x}_{m-1} +\mathbf{b}_m) - -With hls4ml, each layer of output values is calculated independently in sequence, using pipelining to speed up the process by accepting new inputs after an initiation interval. -The activations, if nontrivial, are precomputed. - -To ensure optimal performance, the user can control aspects of their model, principally: - - -* **Size/Compression** - Though not explicitly part of the ``hls4ml`` package, this is an important optimization to efficiently use the FPGA resources -* **Precision** - Define the :doc:`precision ` of the calculations in your model -* **Dataflow/Resource Reuse** - Control parallel or streaming model implementations with varying levels of pipelining -* **Quantization Aware Training** - Achieve best performance at low precision with tools like QKeras, and benefit automatically during inference with ``hls4ml`` parsing of QKeras models - - -.. image:: img/reuse_factor_paper_fig_8.png - :width: 70% - :align: center - - -Often, these decisions will be hardware dependent to maximize performance. -Of note is that simplifying the input network must be done before using ``hls4ml`` to generate HLS code, for optimal compression to provide a sizable speedup. -Also important to note is the use of fixed point arithmetic in ``hls4ml``. -This improves processing speed relative to floating point implementations. -The ``hls4ml`` package also offers the functionality of configuring binning and output bit width of the precomputed activation functions as necessary. With respect to parallelization and resource reuse, ``hls4ml`` offers a "reuse factor" parameter that determines the number of times each multiplier is used in order to compute a layer of neuron's values. Therefore, a reuse factor of one would split the computation so each multiplier had to only perform one multiplication in the computation of the output values of a layer, as shown above. Conversely, a reuse factor of four, in this case, uses a single multiplier four times sequentially. Low reuse factor achieves the lowest latency and highest throughput but uses the most resources, while high reuse factor save resources at the expense of longer latency and lower throughput. -The reuse factor can be set using the configuration options defined on the :doc:`Setup ` page. - -Thereby, the ``hls4ml`` package builds efficient HLS code to implement neural networks on FPGAs for microsecond-scale latency on predictions. For more detailed information, take a look at our :doc:`References ` page. All figures on this page are taken from the following paper: `JINST 13 P07027 (2018) `_. diff --git a/docs/details.rst b/docs/details.rst deleted file mode 100644 index 750833001d..0000000000 --- a/docs/details.rst +++ /dev/null @@ -1,33 +0,0 @@ -================ -Software Details -================ - -Frontends and Backends ----------------------- - -In ``hls4ml`` there is a a concept of a *frontend* to parse the input NN into an internal model graph, and a *backend* that controls -what type of output is produced from the graph. Frontends and backends can be independently chosen. Examples of frontends are the -parsers for Keras or ONNX, and examples of backends are Vivado HLS, Intel HLS, and Vitis HLS. See :ref:`Status and Features` for the -currently supported frontends and backends. - -I/O Types ---------- - -``hls4ml`` supports multiple styles for handling data between layers, known as the ``io_type``. - -io_parallel -^^^^^^^^^^^ -Data is passed in parallel between the layers. This is good for MLP networks and small CNNs. Synthesis may fail for larger networks. - -io_stream -^^^^^^^^^ -Data is passed one "pixel" at a time. Each pixel is an array of channels, which are always sent in parallel. This method for sending -data between layers is recommended for larger CNNs. For ``Dense`` layers, all the inputs are streamed in parallel as a single array. - -With the ``io_stream`` IO type, each layer is connected with the subsequent layer through first-in first-out (FIFO) buffers. -The implementation of the FIFO buffers contribute to the overall resource utilization of the design, impacting in particular the BRAM or LUT utilization. -Because the neural networks can have complex architectures generally, it is hard to know a priori the correct depth of each FIFO buffer. -By default ``hls4ml`` choses the most conservative possible depth for each FIFO buffer, which can result in a an unnecessary overutilization of resources. - -In order to reduce the impact on the resources used for FIFO buffer implementation, we have a FIFO depth optimization flow. This is described -in the :ref:`FIFO Buffer Depth Optimization` section. diff --git a/docs/frontend/keras.rst b/docs/frontend/keras.rst new file mode 100644 index 0000000000..d6d42cb4b8 --- /dev/null +++ b/docs/frontend/keras.rst @@ -0,0 +1,11 @@ +================ +Keras and QKeras +================ + +Keras and the quantization library QKeras are well supported in ``hls4ml``. Currently, the Keras v2 (``tf.keras``) is the preferred version, and the future versions of ``hls4ml`` will expand support for Keras v3. The frontend is based on the parsing the serialized json representation of the model. + +Currently, ``hls4ml`` can parse most Keras layers, including core layers, convolutional layers, pooling layers, recurrent layers, merging/reshaping layers and activation layers, implemented either via sequential or functional API. Notably missing are the attention and normalization layers. The equivalent QKeras API and quantizers are also supported. The ``Lambda`` layers don't save their state in the serialized format and are thus impossible to parse. In this case, the ``Lambda`` layers can be implemented as custom layers and parsed via the :ref:`Extension API`. + +The ``data_format='channels_first'`` parameter of Keras layers is supported, but not extensively tested. All HLS implementations in ``hls4ml`` are based on ``channels_last`` data format and need to be converted to that format before the HLS code can be emitted. We encourage users of ``channels_first`` to report their experiences to developers on GitHub. + +The development team of ``hls4ml`` is currently exploring options for QKeras alternative and will provide a drop-in replacement API compatible with Keras v3. diff --git a/docs/frontend/pytorch.rst b/docs/frontend/pytorch.rst new file mode 100644 index 0000000000..6e91d0c44e --- /dev/null +++ b/docs/frontend/pytorch.rst @@ -0,0 +1,20 @@ +==================== +PyTorch and Brevitas +==================== + +The PyTorch frontend in ``hls4ml`` is implemented by parsing the symbolic trace of the ``torch.fx`` framework. This ensures the proper execution graph is captured. Therefore, only models that can be traced with the FX framework can be parsed by ``hls4ml``. + +Provided the underlying operation is supported in ``hls4ml``, we generally aim to support the use of both ``torch.nn`` classes and ``torch.nn.functional`` functions in the construction of PyTorch models. Generally, the use of classes is more thoroughly +tested. Please reach out if you experience any issues with either case. + +The PyTorch/Brevitas parser is under heavy development and doesn't yet have the same feature set of the Keras parsers. Feel free to reach out to developers if you find a missing feature that is present in Keras parser and would like it implemented. + +.. note:: + The direct ingestion of models quantized with brevitas is not supported currently. Instead, brevitas models shoud be exported in the ONNX format (see `here `_) and read with the ``hls4ml`` + QONNX frontend. Issues may arise, for example when non power-of-2 or non-scalar quantization scales are used. Please reach out if you encounter any problems with this workflow. + +For multi-dimensional tensors, ``hls4ml`` follows the channels-last convention adopted by Keras, whereas PyTorch uses channels-first. By default, ``hls4ml`` will automaticlly transpose any tensors associated with weights and biases of the internal layers +of the model. If the ``io_parallel`` I/O type (see :ref:`Concepts`) is used, a transpose node will be added to the model that also adjusts the input tensors. This is not available in the ``io_stream`` case and inputs must be transposed by the user. +Outputs are not transposed back by default, but in ``io_parallel`` case, a transpose node can be added. If not needed, these adjustments can also be switched off. See :py:class:`~hls4ml.utils.config.config_from_pytorch_model` for details. + +The equivalent of Keras extension API is not yet available for PyTorch parser, and will be provided in the future. diff --git a/docs/advanced/qonnx.rst b/docs/frontend/qonnx.rst similarity index 100% rename from docs/advanced/qonnx.rst rename to docs/frontend/qonnx.rst diff --git a/docs/index.rst b/docs/index.rst index 335650d6dc..ff92a3d543 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,33 +2,64 @@ :hidden: :caption: Introduction - concepts - status - setup - release_notes - details - flows - command - reference + intro/introduction + intro/status + intro/setup + intro/faq + intro/release_notes + intro/reference .. toctree:: :hidden: :glob: - :caption: Quick API Reference + :caption: User Guide - api/* + api/concepts + api/configuration + api/command + +.. toctree:: + :hidden: + :glob: + :caption: Frontends + + frontend/keras + frontend/pytorch + frontend/qonnx + +.. toctree:: + :hidden: + :glob: + :caption: Backends + + backend/vitis + backend/accelerator + backend/oneapi + backend/catapult + backend/quartus + backend/sr .. toctree:: :hidden: :caption: Advanced Features + advanced/profiling + advanced/auto advanced/hgq - advanced/qonnx advanced/fifo_depth advanced/extension - advanced/oneapi - advanced/accelerator advanced/model_optimization + advanced/bramfactor + +.. toctree:: + :hidden: + :glob: + :caption: Internals + + ir/ir + ir/modelgraph + ir/flows + ir/attributes .. toctree:: :hidden: @@ -62,6 +93,4 @@ For the latest status including current and planned features, see the :ref:`Stat Tutorials ================================= -Detailed tutorials on how to use ``hls4ml``'s various functionalities can be found at: - -https://github.com/fastmachinelearning/hls4ml-tutorial +Detailed tutorials on how to use ``hls4ml``'s various functionalities can be found `here `_. diff --git a/docs/intro/faq.rst b/docs/intro/faq.rst new file mode 100644 index 0000000000..22b4c6c99a --- /dev/null +++ b/docs/intro/faq.rst @@ -0,0 +1,52 @@ +Frequently asked questions +========================== + +**What is hls4ml?** + +``hls4ml`` is a tool for converting neural network models into FPGA firmware. hls4ml is aimed at low-latency applications, such as triggering at the Large Hadron Collider (LHC) at CERN, but is applicable to other domains requiring microsecond latency. See the full documentation for more details. + +**How does hls4ml work?** + +``hls4ml`` takes the models from Keras, PyTorch and ONNX (optionally quantized with the respective quantization libraries) and produces high-level synthesis code (based on C++) that can be converted to FPGA firmware using the HLS compilers from different vendors (AMD/Xilinx, Intel/Altera, Catapult...). + +**How is hls4ml so fast?** + +``hls4ml`` stores all weights on-chip for fast access and has tuneable parallelism. As a consequence, the size of the model that can be successfully converted into firmware with hls4ml largely depends on the amount of available resources on the target FPGA. Therefore it is highly recommended to compress the model with quantization (via QKeras or HGQ for Keras or Brevitas for PyTorch) and pruning. Additionally, ``hls4ml`` exploits the parallelism available in an FPGA or ASIC by implementing a spatial dataflow architecture. + +**Will my model work with hls4ml?** + +``hls4ml`` supports many common layers found in MLP, CNN and RNN architectures, however some seldom-used features of these layers may not be supported. Novel architectures such as graph networks or transformers are in various stages of development and are currently not stable for end-users. See the status and features page for more information. Models with custom layers can be supported through extension API. If you encounter a feature not yet supported, open a new issue. + +**Will my model with X parameters fit an FPGA model Y?** + +It depends. ``hls4ml`` has been successfully used with quantized models with `O` (10k) parameters, while for some architectures going beyond `O` (1000) parameters is not doable even on the largest FPGAs. The number of parameters of a model is generally not a good estimate of the performance on an FPGA as the computational complexity of different types of NN layers has big effects on the resource consumption on an FPGA. For example, a CNN or GNN may reuse the same parameter in many operations. Furthermore, model compression in the form of quantization and pruning can significantly change the footprint of the model on the FPGA. For these reasons, we discourage the use of this metric for estimating performance. + +If you're looking for a quick estimate of the resource usage and latency for a given model without synthesis, look into `rule4ml `_ and `wa-hls4ml `_ projects. + +LLMs and large vision transformers are not supported nor planned. + +**How do I get started with hls4ml?** + +We strongly recommend interested users unfamiliar with FPGAs or model compression techniques to review the `hls4ml tutorials `_ to get an overview of the features and conversion workflow. + +**How do I contribute to hls4ml development?** + +We're always welcoming new contributions. If you have an interesting feature in mind feel free to start a new discussion thread with your proposal. We also have regular meetings online to discuss the status of developments where you can be invited to present your work. To receive announcements, `request to be added to our CERN e-group `_. Furthermore, check the `CONTRIBUTING `_ document for a set of technical requirements for making contributions to the hls4ml project. + + +Common HLS synthesis issues +*************************** + +**Stop unrolling loop ... because it may cause large runtime and excessive memory usage due to increase in code size.** + +This error is common with models that are too large to fit on the FPGA given the ``IOType`` used. If you are using ``io_parallel``, consider switching to ``io_stream``, which prevents unrolling all arrays. It may help to also use the ``Resource`` strategy. Pruning or quantizing the model may not help as it is related to the size of the loops. If possible, try to reduce the number of neurons/filters of your model to reduce the size of the activation tensors and thus number of iterations of loops. + +**cannot open shared object file ...: No such file or directory.** + +This is usually an indication that the compilation failed due to incorrect HLS code being produced. It is most likely a bug in hls4ml. Please open a bug report. Note that the displayed error message may be the same but the cause can be different. Unless you're sure that the existing bug reports show the same underlying issue, it is better to open a separate bug report. + +**My hls4ml predictions don't match the original Keras/PyTorch/ONNX ones** + +``hls4ml`` uses fixed-point precision types to represent internal data structures, unlike the floating-point precision types used for computation in upstream ML toolkits. If the used bit width is not sufficiently wide, you may encounter issues with computation accuracy that propagates through the layers. This is especially true for models that are not fully quantized, or models with insufficient ``accum_t`` bitwidth. Look into automatic precision inference and profiling tools to resolve the issue. + +Note that bit-exact behavior is not always possible, as many math functions (used by activation functions) are approximated with lookup tables. diff --git a/docs/intro/introduction.rst b/docs/intro/introduction.rst new file mode 100644 index 0000000000..8d603bd78f --- /dev/null +++ b/docs/intro/introduction.rst @@ -0,0 +1,30 @@ +============ +Introduction +============ + +The goal of ``hls4ml`` is to provide an efficient and fast translation of machine learning models from open-source packages (like Keras and PyTorch) for training machine learning algorithms to high level synthesis (HLS) code that can then be transpiled to run on an FPGA. The resulting HLS project can be then used to produce an IP which can be plugged into more complex designs or be used to create a kernel for CPU co-processing. The user has freedom to define many of the parameters of their algorithm to best suit their needs. + +The ``hls4ml`` package enables fast prototyping of a machine learning algorithm implementation in FPGAs, +greatly reducing the time to results and giving the user intuition for how to best design a machine learning algorithm for their application while balancing performance, resource utilization and latency requirements. + +The Inspiration +=============== + +The inspiration for the creation of the ``hls4ml`` package stems from the high energy physics community at the CERN Large Hadron Collider (LHC). +While machine learning has already been proven to be extremely useful in analysis of data from detectors at the LHC, it is typically performed in an "offline" environment after the data is taken and agglomerated. +However, one of the largest problems at detectors on the LHC is that collisions, or "events", generate too much data for everything to be saved. +As such, filters called "triggers" are used to determine whether a given event should be kept. +Using FPGAs allows for significantly lower latency so machine learning algorithms can essentially be run "live" at the detector level for event selection. As a result, more events with potential signs of new physics can be preserved for analysis. + +The Solution: ``hls4ml`` +======================== + +.. image:: ../img/overview.jpg + + +With this in mind, let's take a look at how ``hls4ml`` helps to achieve such a goal. First, it's important to realize the architecture differences between an FPGA and a CPU or GPU. +An FPGA can be specifically programmed to do a certain task, in this case evaluate neural networks given a set of inputs, and as such can be highly optimized for the task, with tricks like pipelining and parallel evaluation. However, this means dynamic remapping while running isn't really a possibility. +FPGAs also often come at a comparatively low power cost with respect to CPUs and GPUs. This allows ``hls4ml`` to build HLS code from compressed neural networks that results in predictions on the microsecond scale for latency. +The ``hls4ml`` tool saves the time investment needed to convert a neural network to a hardware design language or even HLS code, thus allowing for rapid prototyping. + +For more detailed information on technical details of ``hls4ml``, read the "Internals" section of our documentation or our :doc:`References ` page. All figures on this page are taken from the following paper: `JINST 13 P07027 (2018) `_. diff --git a/docs/reference.rst b/docs/intro/reference.rst similarity index 99% rename from docs/reference.rst rename to docs/intro/reference.rst index f271679620..0bd5912bb1 100644 --- a/docs/reference.rst +++ b/docs/intro/reference.rst @@ -12,9 +12,9 @@ If you use this software in a publication, please cite the software @software{fastml_hls4ml, author = {{FastML Team}}, title = {fastmachinelearning/hls4ml}, - year = 2023, + year = 2024, publisher = {Zenodo}, - version = {v0.8.1}, + version = {v1.0.0}, doi = {10.5281/zenodo.1201549}, url = {https://github.com/fastmachinelearning/hls4ml} } diff --git a/docs/release_notes.rst b/docs/intro/release_notes.rst similarity index 100% rename from docs/release_notes.rst rename to docs/intro/release_notes.rst diff --git a/docs/setup.rst b/docs/intro/setup.rst similarity index 50% rename from docs/setup.rst rename to docs/intro/setup.rst index a735281c3f..6ba0c4ce0e 100644 --- a/docs/setup.rst +++ b/docs/intro/setup.rst @@ -14,7 +14,7 @@ The latest release of ``hls4ml`` can be installed with ``pip``: pip install hls4ml -If you want to use our :doc:`profiling ` toolbox, you might need to install extra dependencies: +If you want to use our :doc:`profiling <../advanced/profiling>` toolbox, you might need to install extra dependencies: .. code-block:: @@ -43,29 +43,36 @@ version can be installed directly from ``git``: Dependencies ============ -The ``hls4ml`` library depends on a number of Python packages and external tools for synthesis and simulation. Python dependencies are automatically managed +The ``hls4ml`` library requires python 3.10 or later, and depends on a number of Python packages and external tools for synthesis and simulation. Python dependencies are automatically managed by ``pip`` or ``conda``. -* `TensorFlow `_ (version 2.4 and newer) and `QKeras `_ are required by the Keras converter. +* `TensorFlow `_ (version 2.8 to 2.14) and `QKeras `_ are required by the Keras converter. One may want to install newer versions of QKeras from GitHub. Newer versions of TensorFlow can be used, but QKeras and hl4ml do not currently support Keras v3. + * `ONNX `_ (version 1.4.0 and newer) is required by the ONNX converter. + * `PyTorch `_ package is optional. If not installed, the PyTorch converter will not be available. Running C simulation from Python requires a C++11-compatible compiler. On Linux, a GCC C++ compiler ``g++`` is required. Any version from a recent -Linux should work. On MacOS, the *clang*-based ``g++`` is enough. +Linux should work. On MacOS, the *clang*-based ``g++`` is enough. For the oneAPI backend, one must have oneAPI installed, along with the FPGA compiler, +to run C/SYCL simulations. To run FPGA synthesis, installation of following tools is required: -* Xilinx Vivado HLS 2018.2 to 2020.1 for synthesis for Xilinx FPGAs +* Xilinx Vivado HLS 2018.2 to 2020.1 for synthesis for Xilinx FPGAs using the ``Vivado`` backend. + +* Vitis HLS 2022.2 or newer is required for synthesis for Xilinx FPGAs using the ``Vitis`` backend. - * Vitis HLS 2022.2 or newer is required for synthesis for Xilinx FPGAs using the ``Vitis`` backend. +* Intel Quartus 20.1 to 21.4 for the synthesis for Intel/Altera FPGAs using the ``Quartus`` backend. -* Intel Quartus 20.1 to 21.4 for the synthesis for Intel FPGAs +* oneAPI 2024.1 to 2025.0 with the FPGA compiler and recent Intel/Altera Quartus for Intel/Altera FPGAs using the ``oneAPI`` backend. + +Catapult HLS 2024.1_1 or 2024.2 can be used to synthesize both for ASICs and FPGAs. Quick Start ============= -For basic concepts to understand the tool, please visit the :doc:`Concepts ` chapter. +For basic concepts to understand the tool, please visit the :doc:`Concepts <../api/concepts>` chapter. Here we give line-by-line instructions to demonstrate the general workflow. .. code-block:: python @@ -98,78 +105,79 @@ After that, you can use :code:`Vivado HLS` to synthesize the model: Done! You've built your first project using ``hls4ml``! To learn more about our various API functionalities, check out our tutorials `here `__. -If you want to configure your model further, check out our :doc:`Configuration ` page. +If you want to configure your model further, check out our :doc:`Configuration <../api/configuration>` page. -Apart from our main API, we also support model conversion using a command line interface, check out our next section to find out more: +.. + Apart from our main API, we also support model conversion using a command line interface, check out our next section to find out more: -Getting started with hls4ml CLI (deprecated) --------------------------------------------- + Getting started with hls4ml CLI (deprecated) + -------------------------------------------- -As an alternative to the recommended Python PI, the command-line interface is provided via the ``hls4ml`` command. + As an alternative to the recommended Python PI, the command-line interface is provided via the ``hls4ml`` command. -To follow this tutorial, you must first download our ``example-models`` repository: + To follow this tutorial, you must first download our ``example-models`` repository: -.. code-block:: bash + .. code-block:: bash - git clone https://github.com/fastmachinelearning/example-models + git clone https://github.com/fastmachinelearning/example-models -Alternatively, you can clone the ``hls4ml`` repository with submodules + Alternatively, you can clone the ``hls4ml`` repository with submodules -.. code-block:: bash + .. code-block:: bash - git clone --recurse-submodules https://github.com/fastmachinelearning/hls4ml + git clone --recurse-submodules https://github.com/fastmachinelearning/hls4ml -The model files, along with other configuration parameters, are defined in the ``.yml`` files. -Further information about ``.yml`` files can be found in :doc:`Configuration ` page. + The model files, along with other configuration parameters, are defined in the ``.yml`` files. + Further information about ``.yml`` files can be found in :doc:`Configuration ` page. -In order to create an example HLS project, first go to ``example-models/`` from the main directory: + In order to create an example HLS project, first go to ``example-models/`` from the main directory: -.. code-block:: bash + .. code-block:: bash - cd example-models/ + cd example-models/ -And use this command to translate a Keras model: + And use this command to translate a Keras model: -.. code-block:: bash + .. code-block:: bash - hls4ml convert -c keras-config.yml + hls4ml convert -c keras-config.yml -This will create a new HLS project directory with an implementation of a model from the ``example-models/keras/`` directory. -To build the HLS project, do: + This will create a new HLS project directory with an implementation of a model from the ``example-models/keras/`` directory. + To build the HLS project, do: -.. code-block:: bash + .. code-block:: bash - hls4ml build -p my-hls-test -a + hls4ml build -p my-hls-test -a -This will create a Vivado HLS project with your model implementation! + This will create a Vivado HLS project with your model implementation! -**NOTE:** For the last step, you can alternatively do the following to build the HLS project: + **NOTE:** For the last step, you can alternatively do the following to build the HLS project: -.. code-block:: Bash + .. code-block:: Bash - cd my-hls-test - vivado_hls -f build_prj.tcl + cd my-hls-test + vivado_hls -f build_prj.tcl -``vivado_hls`` can be controlled with: + ``vivado_hls`` can be controlled with: -.. code-block:: bash + .. code-block:: bash - vivado_hls -f build_prj.tcl "csim=1 synth=1 cosim=1 export=1 vsynth=1" + vivado_hls -f build_prj.tcl "csim=1 synth=1 cosim=1 export=1 vsynth=1" -Setting the additional parameters from ``1`` to ``0`` disables that step, but disabling ``synth`` also disables ``cosim`` and ``export``. + Setting the additional parameters from ``1`` to ``0`` disables that step, but disabling ``synth`` also disables ``cosim`` and ``export``. -Further help -^^^^^^^^^^^^ + Further help + ^^^^^^^^^^^^ -* For further information about how to use ``hls4ml``\ , do: ``hls4ml --help`` or ``hls4ml -h`` -* If you need help for a particular ``command``\ , ``hls4ml command -h`` will show help for the requested ``command`` -* We provide a detailed documentation for each of the command in the :doc:`Command Help <../command>` section + * For further information about how to use ``hls4ml``\ , do: ``hls4ml --help`` or ``hls4ml -h`` + * If you need help for a particular ``command``\ , ``hls4ml command -h`` will show help for the requested ``command`` + * We provide a detailed documentation for each of the command in the :doc:`Command Help ` section Existing examples ----------------- -* Examples of model files and weights can be found in `example_models `_ directory. * Training codes and examples of resources needed to train the models can be found in the `tutorial `__. +* Examples of model files and weights can be found in `example_models `_ directory. Uninstalling ------------ diff --git a/docs/status.rst b/docs/intro/status.rst similarity index 81% rename from docs/status.rst rename to docs/intro/status.rst index 4ff4d33282..5d3f3591f2 100644 --- a/docs/status.rst +++ b/docs/intro/status.rst @@ -18,8 +18,8 @@ A list of supported ML frameworks, HLS backends, and neural network architecture ML framework support: * (Q)Keras -* PyTorch (limited) -* (Q)ONNX (in development) +* PyTorch +* (Q)ONNX Neural network architectures: @@ -32,7 +32,9 @@ HLS backends: * Vivado HLS * Intel HLS -* Vitis HLS (experimental) +* Vitis HLS +* Catapult HLS +* oneAPI (experimental) A summary of the on-going status of the ``hls4ml`` tool is in the table below. @@ -46,35 +48,44 @@ A summary of the on-going status of the ``hls4ml`` tool is in the table below. - Vivado HLS - Intel HLS - Vitis HLS + - Catapult HLS + - oneAPI * - MLP - ``supported`` - - ``limited`` - - ``in development`` + - ``supported`` + - ``supported`` + - ``supported`` + - ``supported`` - ``supported`` - ``supported`` - ``experimental`` * - CNN - ``supported`` - - ``limited`` - - ``in development`` + - ``supported`` + - ``supported`` + - ``supported`` + - ``supported`` - ``supported`` - ``supported`` - ``experimental`` * - RNN (LSTM) + - ``supported`` - ``supported`` - ``N/A`` - - ``in development`` - ``supported`` - ``supported`` - - ``N/A`` + - ``supported`` + - ``supported`` + - ``experimental`` * - GNN (GarNet) - ``supported`` + - ``in development`` + - ``N/A`` - ``N/A`` - ``N/A`` - ``N/A`` - ``N/A`` - ``N/A`` - Other feature notes: @@ -82,6 +93,9 @@ Other feature notes: * Vivado HLS versions 2018.2 to 2020.1 * Intel HLS versions 20.1 to 21.4 * Vitis HLS versions 2022.2 to 2024.1 + * Catapult HLS versions 2024.1_1 to 2024.2 + * oneAPI versions 2024.1 to 2025.0 + * Windows and macOS are not supported * BDT support has moved to the `Conifer `__ package diff --git a/docs/ir/attributes.rst b/docs/ir/attributes.rst new file mode 100644 index 0000000000..dfbec51b1c --- /dev/null +++ b/docs/ir/attributes.rst @@ -0,0 +1,2802 @@ +================ +Layer attributes +================ + + +Input +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Constant +======== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* value: ndarray + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Activation +========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +ParametrizedActivation +====================== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* param_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +* n_in: int + +* activation: str + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* param_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +PReLU +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* param_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +* n_in: int + +* activation: str + +Weight attributes +----------------- +* param: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* param_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Softmax +======= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* implementation: list [latency,stable,argmax,legacy] (Default: stable) + + * Choice of implementation of softmax function. "latency" provides good latency at the expense of extra resources. performs well on small number of classes. "stable" may require extra clock cycles but has better accuracy. "legacy" is the older implementation which has bad accuracy, but is fast and has low resource use. It is superseded by the "latency" implementation for most applications. "argmax" is a special implementation that can be used if only the output with the highest probability is important. Using this implementation will save resources and clock cycles. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* skip: bool (Default: False) + + * If enabled, skips the softmax node and returns the raw outputs. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* exp_table_t: NamedType (Default: fixed<18,8,RND,SAT,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* inv_table_t: NamedType (Default: fixed<18,8,RND,SAT,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +TernaryTanh +=========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +HardActivation +============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* slope_t: NamedType + +* shift_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* activation: str + +* slope: float (Default: 0.2) + +* shift: float (Default: 0.5) + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* slope_t: NamedType + +* shift_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Reshape +======= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* target_shape: Sequence + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Dense +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_out: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Conv +==== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Conv1D +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_width: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +Conv2D +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +Conv2DBatchnorm +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +SeparableConv1D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* depthwise_t: NamedType + +* pointwise_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_width: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* depth_multiplier: int (Default: 1) + +* filt_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* depthwise: WeightVariable + +* pointwise: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* depthwise_t: NamedType + +* pointwise_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* depthwise_accum_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* pointwise_accum_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* depthwise_result_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* depthwise_reuse_factor: int (Default: 1) + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* pointwise_reuse_factor: int (Default: 1) + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +* dw_output_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * Available in: Catapult + +DepthwiseConv1D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_width: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +* in_width: int + +* out_width: int + +* n_chan: int + +* depth_multiplier: int (Default: 1) + +* n_filt: int + +* filt_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +SeparableConv2D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* depthwise_t: NamedType + +* pointwise_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* depth_multiplier: int (Default: 1) + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* depthwise: WeightVariable + +* pointwise: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* depthwise_t: NamedType + +* pointwise_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* depthwise_accum_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* pointwise_accum_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* depthwise_result_t: NamedType + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* depthwise_reuse_factor: int (Default: 1) + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* pointwise_reuse_factor: int (Default: 1) + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +* dw_output_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * Available in: Catapult + +DepthwiseConv2D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* depth_multiplier: int (Default: 1) + +* n_filt: int + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +BatchNormalization +================== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* scale_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_filt: int (Default: -1) + +* use_gamma: bool (Default: True) + +* use_beta: bool (Default: True) + +Weight attributes +----------------- +* scale: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* scale_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Pooling1D +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_out: int + +* n_filt: int + +* pool_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +* count_pad: bool (Default: False) + +* pool_op: list [Max,Average] + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +Pooling2D +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_filt: int + +* pool_height: int + +* pool_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +* count_pad: bool (Default: False) + +* pool_op: list [Max,Average] + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +GlobalPooling1D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_filt: int + +* pool_op: list [Max,Average] + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +GlobalPooling2D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* n_filt: int + +* pool_op: list [Max,Average] + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +ZeroPadding1D +============= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_width: int + +* out_width: int + +* n_chan: int + +* pad_left: int + +* pad_right: int + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +ZeroPadding2D +============= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Merge +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +MatMul +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Dot +=== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, Vivado, VivadoAccelerator, VivadoAccelerator, Vitis, Vitis, Quartus, Quartus, Catapult, Catapult, SymbolicExpression, SymbolicExpression, oneAPI, oneAPI + +Concatenate +=========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +Resize +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* align_corners: bool (Default: False) + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* algorithm: list [nearest,bilinear] (Default: nearest) + +Transpose +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Embedding +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* embeddings_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_out: int + +* vocab_size: int + +Weight attributes +----------------- +* embeddings: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* embeddings_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +SimpleRNN +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_out: int + +* activation: str + +* return_sequences: bool (Default: False) + +* return_state: bool (Default: False) + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +* recurrent_weight: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* direction: list [forward,backward] (Default: forward) + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* recurrent_reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* static: bool (Default: True) + + * If set to True, will reuse the the same recurrent block for computation, resulting in lower resource usage at the expense of serialized computation and higher latency/II. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +LSTM +==== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +* recurrent_bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_out: int + +* activation: str + +* recurrent_activation: str + +* return_sequences: bool (Default: False) + +* return_state: bool (Default: False) + +* time_major: bool (Default: False) + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +* recurrent_weight: WeightVariable + +* recurrent_bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* direction: list [forward,backward] (Default: forward) + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +* recurrent_bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* recurrent_reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* static: bool (Default: True) + + * If set to True, will reuse the the same recurrent block for computation, resulting in lower resource usage at the expense of serialized computation and higher latency/II. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +GRU +=== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +* recurrent_bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_out: int + +* activation: str + +* recurrent_activation: str + +* return_sequences: bool (Default: False) + +* return_state: bool (Default: False) + +* time_major: bool (Default: False) + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +* recurrent_weight: WeightVariable + +* recurrent_bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* direction: list [forward,backward] (Default: forward) + +* apply_reset_gate: list [before,after] (Default: after) + +* weight_t: NamedType + +* bias_t: NamedType + +* recurrent_weight_t: NamedType + +* recurrent_bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* recurrent_reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* static: bool (Default: True) + + * If set to True, will reuse the the same recurrent block for computation, resulting in lower resource usage at the expense of serialized computation and higher latency/II. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +* table_size: int (Default: 1024) + + * The size of the lookup table used to approximate the function. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +* table_t: NamedType (Default: fixed<18,8,TRN,WRAP,0>) + + * The datatype (precision) used for the values of the lookup table. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, oneAPI + +GarNet +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, Vivado, VivadoAccelerator, VivadoAccelerator, Vitis, Vitis, Quartus, Quartus, Catapult, Catapult, SymbolicExpression, SymbolicExpression, oneAPI, oneAPI + +GarNetStack +=========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, Vivado, VivadoAccelerator, VivadoAccelerator, Vitis, Vitis, Quartus, Quartus, Catapult, Catapult, SymbolicExpression, SymbolicExpression, oneAPI, oneAPI + +Quant +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* narrow: bool + +* rounding_mode: str + +* signed: bool + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +ApplyAlpha +========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* scale_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_filt: int (Default: -1) + +* use_gamma: bool (Default: True) + +* use_beta: bool (Default: True) + +Weight attributes +----------------- +* scale: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* scale_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +BatchNormOnnx +============= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +LayerGroup +========== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* layer_list: list + +* input_layers: list + +* output_layers: list + +* data_reader: object + +* output_shape: list + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +SymbolicExpression +================== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* expression: list + +* n_symbols: int + +* lut_functions: list (Default: []) + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +BiasAdd +======= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Backend-specific attributes +--------------------------- +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +FixedPointQuantizer +=================== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +UnaryLUT +======== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Repack +====== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Clone +===== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +BatchNormalizationQuantizedTanh +=============================== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* accum_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* n_in: int + +* n_filt: int (Default: 0) + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* accum_t: NamedType + +* reuse_factor: int (Default: 1) + +PointwiseConv1D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_width: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_width: int + +* stride_width: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +PointwiseConv2D +=============== +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +* in_height: int + +* in_width: int + +* out_height: int + +* out_width: int + +* n_chan: int + +* n_filt: int + +* filt_height: int + +* filt_width: int + +* stride_height: int + +* stride_width: int + +* pad_top: int + +* pad_bottom: int + +* pad_left: int + +* pad_right: int + +Weight attributes +----------------- +* weight: WeightVariable + +* bias: WeightVariable + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +* weight_t: NamedType + +* bias_t: NamedType + +Backend-specific attributes +--------------------------- +* accum_t: NamedType + + * The datatype (precision) used to store intermediate results of the computation within the layer. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* reuse_factor: int (Default: 1) + + * The number of times each multiplier is used by controlling the amount of pipelining/unrolling. Lower number results in more parallelism and lower latency at the expense of the resources used.Reuse factor = 1 corresponds to all multiplications executed in parallel, and hence, the lowest possible latency. + + * Available in: Vivado, VivadoAccelerator, Vitis, Quartus, Catapult, SymbolicExpression, oneAPI + +* parallelization_factor: int (Default: 1) + + * The number of outputs computed in parallel. Essentially the number of multiplications of input window with the convolution kernel occuring in parallel. Higher number results in more parallelism (lower latency and II) at the expense of resources used.Currently only supported in io_parallel. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult, oneAPI + +* conv_implementation: list [LineBuffer,Encoded] (Default: LineBuffer) + + * "LineBuffer" implementation is preferred over "Encoded" for most use cases. This attribute only applies to io_stream. + + * Available in: Vivado, VivadoAccelerator, Vitis, Catapult + +Broadcast +========= +Base attributes +--------------- +* result_t: NamedType + + * The datatype (precision) of the output tensor. + +Type attributes +--------------- +* index: int + + * Internal node counter used for bookkeeping and variable/tensor naming. + +Configurable attributes +----------------------- +* trace: int (Default: False) + + * Enables saving of layer output (tracing) when using hls_model.predict(...) or hls_model.trace(...) + +* result_t: NamedType + + * The datatype (precision) of the output tensor. diff --git a/docs/flows.rst b/docs/ir/flows.rst similarity index 84% rename from docs/flows.rst rename to docs/ir/flows.rst index 37b8b44ff9..dbdef58896 100644 --- a/docs/flows.rst +++ b/docs/ir/flows.rst @@ -2,17 +2,6 @@ Optimizer Passes and Flows ========================== -Internal Structure ------------------- - -The ``hls4ml`` library will parse models from Keras, PyTorch or ONNX into an internal execution graph. This model graph is represented with the -:py:class:`~hls4ml.model.graph.ModelGraph` class. The nodes in this graph, corresponding to the layer and operations of the input model are represented -by classes derived from the :py:class:`~hls4ml.model.layers.Layer` base class. - -Layers are required to have defined inputs and outputs that define how they are connected in the graph and what is the shape of their output. All information -about the layer's state and configuration is stored in its attributes. All weights, variables and data types are attributes and there are mapping views to sort through them. -Layers can define expected attributes and can be verified for correctness, or to produce a list of configurable attributes that user can tweak. - Optimizer passes ---------------- diff --git a/docs/ir/ir.rst b/docs/ir/ir.rst new file mode 100644 index 0000000000..18b0a1c679 --- /dev/null +++ b/docs/ir/ir.rst @@ -0,0 +1,90 @@ +======================= +Internal representation +======================= + +The ``hls4ml`` library will parse models from Keras, PyTorch or ONNX into an internal execution graph. This model graph is represented with the +:py:class:`~hls4ml.model.graph.ModelGraph` class. The nodes in this graph, loosely corresponding to the layers and operations of the input model are represented +by classes derived from the :py:class:`~hls4ml.model.layers.Layer` base class. + +Layers are required to have defined inputs and outputs that define how they are connected in the graph and what is the shape of their output. All information +about the layer's state and configuration is stored in its attributes. All weights, variables and data types are attributes and there are mapping views to sort through them. +Layers can define expected attributes and can be verified for correctness, or to produce a list of configurable attributes that user can tweak. The complete list of attributes can be found in the :doc:`Attributes ` page. + + +Layers +====== + +The backends of ``hls4ml`` are independent from each other and free to implement features in any suitable way, most implementations share common concepts which we will mention here. + +Dense Layers +------------ + +One-dimensional Dense Layers +**************************** + +Dense layers over one-dimensional data perform a matrix-vector multiplication followed by elementwise addition of bias tensor. This routine is the underlying computation of many other layers as well and is reused as much as possible. It exists in several implementations across different backends, for different `io_type`'s and strategies. + +io_parallel +^^^^^^^^^^^ + +All the backends have a ``Resource`` implementation, which divides the computation into a loop of ``reuse_factor`` iterations, each iteration simultaneously accessing a different part of the array partitioned in BRAM. There are different implementations depending on whether the reuse factor is smaller or bigger than the input size. The two Xilinx backends and Catapult also implement a ``Latency`` implementation, which uses the reuse factor to control the amount of pipelining/unrolling of the whole function while the weight array is fully partitioned in registers. + +io_stream +^^^^^^^^^ + +The io_stream implementation only wraps the io_parallel implementation with streams or pipes for communication. Internally, data is still accessed in parallel as an array. + +Multi-dimensional Dense Layers +****************************** + +Multi-dimensional Dense layers are converted to pointwise convolutions, and do not directly use the above implementation. + + +Convolution Layers +------------------ + +Standard convolution +******************** + +By *standard* convolution we refer to the operation represented by the ``Conv1D/2D`` layer in Keras (``Conv1d/2d`` in PyTorch). Depending on the ``io_type`` option used, there are two classes of implementations in ``hls4ml``. + +io_parallel +^^^^^^^^^^^ + +Parallel IO is applicable to small models that require low latency implementation. Larger models face synthesizability limits very quickly. + +In Vivado/Vitis backends, parallel convolution relies on the *im2col* transformation of the input, which turns convolution into a matrix-multiplication task. This task is then implemented as a sequence of matrix-vector multiplications using the routine mentioned above. The ``Latency`` and ``Resource`` strategies refer to the function used for matrix-vector multiplication routine, with ``Resource`` allowing for a slightly larger models to be synthesized. Parallelism can be further controlled via the ``ParallelizationFactor``. Catapult backend in turn uses a direct implementation of convolution via nested loops. The ``Quartus``, ``oneAPI``, and ``Catapult`` backends also implement a ``Winograd`` algorithm choosable by setting the ``implementation`` to ``Winograd`` or ``combination``. Winograd implementation is available for only a handful of filter size configurations, and it is less concerned about bit accuracy and overflow. In certain conditions it can be faster. + +io_stream +^^^^^^^^^ + +There are two main classes of io_stream implementations, ``LineBuffer`` and ``Encoded``. ``LineBuffer`` is the default, and generally produces marginally better results, +while ``Catapult`` and ``Vivado`` also implement ``Encoded``, choosable with the ``ConvImplementation`` configuration option. In all cases, the data is processed serially, one pixel at a time, with a pixel containing an array of all the channel values for the pixel. + +Depthwise convolution +********************* + +Depthwise implementation substitutes the matrix-vector multiplication in the kernel to the elementwise multiplication. The only implementation available is based on ``Latency`` strategy, used by both ``io_parallel`` and ``io_stream``. + +Pointwise convolution +********************* + +Pointwise convolutions are a special case of convolution where the filter size is ``1`` for 1D or ``1x1`` for 2D. + +For the Vivado/Vitis backends, there is a dedicated ``io_parallel``/``Latency`` strategy implementation of 1D pointwise convolutional layers originally developed for `arXiv:2402.01876 `_. +The reuse factor (RF) is used to split the layer execution and reuse the existing module RF times. The RF also limits the number of multipliers in each module. +The initiation interval scales as the RF. One limitation is that it assumes ``in_width`` is divisible by the RF. + +Activations +----------- + +Most activations without extra parameters are represented with the ``Activation`` layer, and those with single parameters (leaky ReLU, thresholded ReLU, ELU) as ``ParametrizedActivation``. ``PReLU`` has its own class because it has a parameter matrix (stored as a weight). The hard (piecewise linear) sigmoid and tanh functions are implemented in a ``HardActivation`` layer, and ``Softmax`` has its own layer class. + +Backends have four softmax implementations that the user can choose from by setting the ``implementation`` parameter: + +* **latency**: Good latency, but somewhat high resource usage. It does not work well if there are many output classes. +* **stable**: Slower but with better accuracy, useful in scenarios where higher accuracy is needed. +* **legacy**: An older implementation with poor accuracy, but good performance. Usually the latency implementation is preferred. +* **argmax**: If you don't care about normalized outputs and only care about which one has the highest value, using argmax saves a lot of resources. This sets the highest value to 1, the others to 0. + +Vivado/Vitis backend additionally support completely skipping softmax activation and returning raw outputs. diff --git a/docs/api/hls-model.rst b/docs/ir/modelgraph.rst similarity index 58% rename from docs/api/hls-model.rst rename to docs/ir/modelgraph.rst index bf0d8ee3ce..048e67e101 100644 --- a/docs/api/hls-model.rst +++ b/docs/ir/modelgraph.rst @@ -1,8 +1,8 @@ ================ -HLS Model Class +ModelGraph Class ================ -This page documents our hls_model class usage. You can generate generate an hls model object from a keras model through ``hls4ml``'s API: +This page documents our ``ModelGraph`` class usage. You can generate generate an instance of this class through ``hls4ml``'s API, for example by converting a Keras model: .. code-block:: python @@ -11,10 +11,10 @@ This page documents our hls_model class usage. You can generate generate an hls # Generate a simple configuration from keras model config = hls4ml.utils.config_from_keras_model(keras_model, granularity='name') - # Convert to an hls model + # Convert to a ModelGraph instance (hls_model) hls_model = hls4ml.converters.convert_from_keras_model(keras_model, hls_config=config, output_dir='test_prj') -After that, you can use several methods in that object. Here is a list of all the methods: +This object can be used to perform common simulation and firmware-generation tasks. Here is a list of important user-facing methods: * :ref:`write ` @@ -23,8 +23,6 @@ After that, you can use several methods in that object. Here is a list of all th * :ref:`build ` * :ref:`trace ` -Similar functionalities are also supported through command line interface. If you prefer using them, please refer to Command Help section. - ---- .. _write-method: @@ -32,7 +30,7 @@ Similar functionalities are also supported through command line interface. If yo ``write`` method ==================== -Write your keras model as a hls project to ``hls_model``\ 's ``output_dir``\ : +Write the ``ModelGraph`` to the output directory specified in the config: .. code-block:: python @@ -45,7 +43,7 @@ Write your keras model as a hls project to ``hls_model``\ 's ``output_dir``\ : ``compile`` method ====================== -Compile your hls project. +Compiles the written C++/HLS code and links it into the Python runtime. Compiled model can be used to evaluate performance (accuracy) through ``predict()`` method. .. code-block:: python @@ -58,7 +56,7 @@ Compile your hls project. ``predict`` method ====================== -Similar to ``keras``\ 's predict API, you can get the predictions of ``hls_model`` just by supplying an input ``numpy`` array: +Similar to ``keras``\ 's predict API, you can get the predictions just by supplying an input ``numpy`` array: .. code-block:: python @@ -67,7 +65,7 @@ Similar to ``keras``\ 's predict API, you can get the predictions of ``hls_model y = hls_model.predict(X) -This is similar to doing ``csim`` simulation, but you can get your prediction results much faster. It's very helpful when you want to quickly prototype different configurations for your model. +This is similar to doing ``csim`` simulation, without creating the testbench and supplying data. It's very helpful when you want to quickly prototype different configurations for your model. ---- @@ -76,13 +74,17 @@ This is similar to doing ``csim`` simulation, but you can get your prediction re ``build`` method ==================== +This method "builds" the generated HLS project. The parameters of build are backend-specific and usually include simulation and synthesis. Refer to each backend for a complete list of supported parameters to ``build()``. + .. code-block:: python - hls_model.build() + report = hls_model.build() #You can also read the report of the build hls4ml.report.read_vivado_report('hls4ml_prj') +The returned ``report`` object will contain the result of build step, which may include C-simulation results, HLS synthesis estimates, co-simulation latency etc, depending on the backend used. + ---- .. _trace-method: diff --git a/docs/requirements.txt b/docs/requirements.txt index 66aa579ea6..fe3c4f2544 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -4,5 +4,4 @@ sphinx>=3.2.1 sphinx_contributors sphinx_github_changelog sphinx_rtd_theme -tensorflow<=2.15 toposort>=1.5.0 diff --git a/hls4ml/converters/__init__.py b/hls4ml/converters/__init__.py index 13e90df687..3d7ce1fe56 100644 --- a/hls4ml/converters/__init__.py +++ b/hls4ml/converters/__init__.py @@ -278,9 +278,10 @@ def convert_from_pytorch_model( Notes: Pytorch uses the "channels_first" data format for its tensors, while hls4ml expects the "channels_last" format used by keras. By default, hls4ml will automatically add layers to the model which transpose the inputs to the - "channels_last"format. Not that this is not supported for the "io_stream" io_type, for which the user will have - to transpose the input by hand before passing it to hls4ml. In that case the "inputs_channel_last" argument of - the "config_from_pytorch_model" function needs to be set to True. By default, the output of the model remains + "channels_last" format. Not that this is not supported for the "io_stream" io_type, for which the user will have + to transpose the input by hand before passing it to hls4ml. In that case the "channels_last_conversion" argument of + the "config_from_pytorch_model" function needs to be set to "internal". This argument can be used to completely + disable this internal conversion. By default, the output of the model remains in the "channels_last" data format. The "transpose_outputs" argument of the "config_from_pytorch_model" can be used to add a layer to the model that transposes back to "channels_first". As before, this will not work for io_stream. diff --git a/hls4ml/converters/utils.py b/hls4ml/converters/utils.py index d1c9e050d5..f365916b55 100644 --- a/hls4ml/converters/utils.py +++ b/hls4ml/converters/utils.py @@ -45,7 +45,7 @@ def compute_padding_1d(pad_type, in_size, stride, filt_size): is odd, it will add the extra column to the right. Args: - pad_type (str): Padding type, one of ``same``, `valid`` or ``causal`` (case insensitive). + pad_type (str): Padding type, one of ``same``, ``valid`` or ``causal`` (case insensitive). in_size (int): Input size. stride (int): Stride length. filt_size (int): Length of the kernel window. @@ -135,6 +135,23 @@ def compute_padding_2d(pad_type, in_height, in_width, stride_height, stride_widt def compute_padding_1d_pytorch(pad_type, in_size, stride, filt_size, dilation): + """Computes the amount of padding required on each side of the 1D input tensor following pytorch conventions. + + In case of ``same`` padding, this routine tries to pad evenly left and right, but if the amount of columns to be added + is odd, it will add the extra column to the right. + + Args: + pad_type (str or int): Padding type. If string, one of ``same``, ``valid`` or ``causal`` (case insensitive). + in_size (int): Input size. + stride (int): Stride length. + filt_size (int): Length of the kernel window. + + Raises: + Exception: Raised if the padding type is unknown. + + Returns: + tuple: Tuple containing the padded input size, left and right padding values. + """ if isinstance(pad_type, str): if pad_type.lower() == 'same': n_out = int( @@ -176,6 +193,26 @@ def compute_padding_1d_pytorch(pad_type, in_size, stride, filt_size, dilation): def compute_padding_2d_pytorch( pad_type, in_height, in_width, stride_height, stride_width, filt_height, filt_width, dilation_height, dilation_width ): + """Computes the amount of padding required on each side of the 2D input tensor following pytorch conventions. + + In case of ``same`` padding, this routine tries to pad evenly left and right (top and bottom), but if the amount of + columns to be added is odd, it will add the extra column to the right/bottom. + + Args: + pad_type (str or int): Padding type. If string, one of ``same`` or ``valid`` (case insensitive). + in_height (int): The height of the input tensor. + in_width (int): The width of the input tensor. + stride_height (int): Stride height. + stride_width (int): Stride width. + filt_height (int): Height of the kernel window. + filt_width (int): Width of the kernel window. + + Raises: + Exception: Raised if the padding type is unknown. + + Returns: + tuple: Tuple containing the padded input height, width, and top, bottom, left and right padding values. + """ if isinstance(pad_type, str): if pad_type.lower() == 'same': # Height diff --git a/hls4ml/model/layers.py b/hls4ml/model/layers.py index c276e2814b..edd0051c6e 100644 --- a/hls4ml/model/layers.py +++ b/hls4ml/model/layers.py @@ -935,7 +935,7 @@ def _get_act_function_name(self): class HardActivation(Activation): ''' - Implements the hard sigmoid and tan function in keras and qkeras + Implements the hard sigmoid and tanh function in keras and qkeras (Default parameters in qkeras are different, so should be configured) The hard sigmoid unction is clip(slope * x + shift, 0, 1), and the hard tanh function is 2 * hard_sigmoid - 1 diff --git a/hls4ml/utils/config.py b/hls4ml/utils/config.py index 1a25fb9c3f..e450084095 100644 --- a/hls4ml/utils/config.py +++ b/hls4ml/utils/config.py @@ -292,6 +292,15 @@ def config_from_pytorch_model( Users are advised to inspect the returned object to tweak the conversion configuration. The return object can be passed as `hls_config` parameter to `convert_from_pytorch_model`. + Note that hls4ml internally follows the keras convention for nested tensors known as + "channels last", wherease pytorch uses the "channels first" convention. + For exampe, for a tensor encoding an image with 3 channels, pytorch will expect the data + to be encoded as (Number_Of_Channels, Height , Width), whereas hls4ml expects + (Height , Width, Number_Of_Channels). By default, hls4ml will perform the necessary + conversions of the inputs and internal tensors automatically, but will return the output + in "channels last" However, this behavior can be controlled by the user using the + related arguments discussed below. + Args: model: PyTorch model input_shape (tuple or list of tuples): The shape of the input tensor, excluding the batch size. @@ -309,9 +318,10 @@ def config_from_pytorch_model( be an explicit precision: 'auto' is not allowed. default_reuse_factor (int, optional): Default reuse factor. Defaults to 1. channels_last_conversion (string, optional): Configures the conversion of pytorch layers to - 'channels_last' dataformate. Can be set to 'full', 'internal', or 'off'. If 'full', both the inputs - and internal layers will be converted. If 'internal', only internal layers will be converted; this - assumes the inputs are converted by the user. If 'off', no conversion is performed. + 'channels_last' data format used by hls4ml internally. Can be set to 'full' (default), 'internal', + or 'off'. If 'full', both the inputs and internal layers will be converted. If 'internal', + only internal layers will be converted; this assumes the inputs are converted by the user. + If 'off', no conversion is performed. transpose_outputs (bool, optional): Set to 'False' if the output should not be transposed from channels_last into channels_first data format. Defaults to 'False'. If False, outputs needs to be transposed manually. diff --git a/setup.cfg b/setup.cfg index cc15eec49f..dc1075d9f3 100644 --- a/setup.cfg +++ b/setup.cfg @@ -33,7 +33,7 @@ install_requires = tabulate tensorflow>=2.8.0,<=2.14.1 tensorflow-model-optimization<=0.7.5 -python_requires = >=3.10, <=3.12 +python_requires = >=3.10, <3.12 include_package_data = True scripts = scripts/hls4ml