Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EDS to DBC #488

Open
madshadsen opened this issue Jun 23, 2020 · 20 comments
Open

EDS to DBC #488

madshadsen opened this issue Jun 23, 2020 · 20 comments

Comments

@madshadsen
Copy link

Hi there,

I was wondering if you've considered adding an EDS to DBC converter for use with CANopen EDS files? I think it would be really helpful for many users!

@ebroecker
Copy link
Owner

Thanks for this hint.
But I have absolute no idea about canopen.
So some external help would be needed.

@mrfrenzy
Copy link

This would be a really useful feature to have as EDS-files are so widespread with a lot of hardware.
The file specification is well documented and free to download https://www.can-cia.org/can-knowledge/canopen/cia306/#

@mtslab
Copy link

mtslab commented May 20, 2022

If you guys figure out how to do this I will love to see it. I don't know why companies bother on writing EDS files if the standard is dbc.

@ebroecker
Copy link
Owner

I looked a bit into EDS but I'm not really able to understand how they work and how this could be translated to dbc.

Can someone provide some basic example?

@mrfrenzy
Copy link

mrfrenzy commented Jul 19, 2023

It is great if you want to look at the possibility of implementing this, I will help any way I can. Here is a basic example.
Bürkert makes a motor valve called 3280. You can send an opening percentage 0-100% via CAN and it will open this much. It will also respond back with how much it is actually open, the current operating status and if there are any problems.

To aid communication they supply an EDS file with a lot of information.
Most CANopen devices support NMT, SDO and PDO.
Network Management are used for device status.
Service Data Objects are used for programming.
Process Data Objects are used for communication and starts transmitting as soon as the device is powered up.

To make the example easy let's start with PDO only.
Opening the EDS file in any EDS viewer we can see what PDOs are supported automatically:
Transmit and receive PDO
(this can be changed with SDO configuration, but let's not worry about that to begin with).

Receive PDO 1 is a communication object the slave RECEIVES from our master. It has two parameters: CMDdigital and PVdigital.
Receive PDO 2 is a communication object the slave RECEIVES from our master. It has one parameter: SPDigital
Transmit PDO 1 is a communication object the slave SENDS to our master. It has two parameters: CMD and CMD_Disp
etc.

I have made a dbc file with the most important objects as an example, I attach this, the EDS file and a PDF file describing the contents. As you can see all the data in my dbc file can be derived from the EDS file.

BUER328X_1_14.eds.zip
MA3280-EDS-file-EU-ML.pdf
Burkert 3280 dbc.zip

@ebroecker
Copy link
Owner

Hi @mrfrenzy

Thank you for these details, this will help a lot!

Next I'll build some basic code example

@ebroecker
Copy link
Owner

ebroecker commented Jul 21, 2023

Hi @mrfrenzy,

I assembled some code.
As far as I understood, the arbitration-ids are dependent from the node_id.

So my code create this dbc with node_id 1:

MotorValve.dbc.zip

you can even test the code:
pip install git+https://github.com/ebroecker/canmatrix/tree/eds_support_488

ebroecker added a commit that referenced this issue Jul 22, 2023
ebroecker added a commit that referenced this issue Jul 24, 2023
ebroecker added a commit that referenced this issue Jul 24, 2023
@AIRicky
Copy link

AIRicky commented Aug 22, 2023

Hi there,

I was wondering if you've considered adding an EDS to DBC converter for use with CANopen EDS files? I think it would be really helpful for many users!

Hi there,

Do you know how to convert a DBC file to EDS?

Best,
Ricky

@FerrariX
Copy link

FerrariX commented Oct 10, 2023

https://github.com/ebroecker/canmatrix/tree/eds_support_488

Hi,

I'm getting a fatal error of repository not found.

output from running 'pip install git+https://github.com/ebroecker/canmatrix/tree/eds_support_488'

fatal: repository 'https://github.com/ebroecker/canmatrix/tree/eds_support_488/' not found
error: subprocess-exited-with-error

I've also tried running 'pip install git+https://github.com/ebroecker/canmatrix#egg=canmatrix[eds]' but still got error testing the posted 'BUER328X_1_14.eds' file.

Update:
I was able to get it to install using 'pip install -U git+https://github.com/ebroecker/canmatrix.git@eds_support_488'
Unfortunately, it gave me the following errors when I ran 'canconvert BUER328X_1_14.eds test.dbc':

ldf is not supported
xls is not supported
xlsx is not supported
yaml is not supported
eds is not supported
INFO - convert - Importing BUER328X_1_14.eds ...
ERROR - init - This file format is not supported for reading
INFO - convert - done

INFO - convert - Exporting test.dbc ...
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
....

Thanks

@ebroecker
Copy link
Owner

Hi @FerrariX ,

sorry for the maybe wrong link.
You managed to find a working link for checkout.

I forgot to tell:
you need the module canopen also.
so could you try a pip install canopen and try again?

@danjhunter
Copy link

danjhunter commented Nov 8, 2023

Hi there,
I was wondering if you've considered adding an EDS to DBC converter for use with CANopen EDS files? I think it would be really helpful for many users!

Hi there,

Do you know how to convert a DBC file to EDS?

Best, Ricky

Is it possible to convert a .DBC file to .EDS file with this branch @ebroecker? Specifically could you add ability to write .EDS format? Thanks

@ebroecker
Copy link
Owner

@danjhunter

Sorry - only Basic Support to decode EDS - encoding maybe not possible, i think dbc lacks needed information for this task

ebroecker added a commit that referenced this issue Dec 18, 2023
* basic EDS import support #488
* EDS add canopen requirement to setup.py  #488
@MatinF
Copy link

MatinF commented Jan 7, 2025

Hi Eduard,

I've been trying this conversion of EDS to DBC on a couple of EDS files (I've sent them via your email), but unfortunately they all fail. The most common error seems to be a missing PDO name - see below the full error trace:

Microsoft Windows [Version 10.0.19045.5247]
(c) Microsoft Corporation. All rights reserved.

C:\Users\marti\Downloads\eds-files>canconvert 1115.eds test.dbc
xls is not supported
xlsx is not supported
INFO - convert - Importing 1115.eds ...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Scripts\canconvert.exe\__main__.py", line 7, in <module>
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\cli\convert.py", line 158, in cli_convert
    canmatrix.convert.convert(infile, outfile, **options)
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\convert.py", line 68, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 71, in loadp
    return load(fileObject, import_type, key, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 90, in load
    dbs[key] = module_instance.load(file_object, **options)  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\eds.py", line 154, in load
    frame.arbitration_id = canmatrix.ArbitrationId(id=frame_id)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<attrs generated init canmatrix.canmatrix.ArbitrationId>", line 4, in __init__
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\canmatrix.py", line 645, in __attrs_post_init__
    raise ArbitrationIdOutOfRange('ID out of range')
canmatrix.canmatrix.ArbitrationIdOutOfRange: ID out of range

C:\Users\marti\Downloads\eds-files>canconvert HE-927325-0002.eds test.dbc
xls is not supported
xlsx is not supported
INFO - convert - Importing HE-927325-0002.eds ...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Scripts\canconvert.exe\__main__.py", line 7, in <module>
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\cli\convert.py", line 158, in cli_convert
    canmatrix.convert.convert(infile, outfile, **options)
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\convert.py", line 68, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 71, in loadp
    return load(fileObject, import_type, key, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 90, in load
    dbs[key] = module_instance.load(file_object, **options)  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\eds.py", line 150, in load
    frame = canmatrix.canmatrix.Frame(name=pdo_name, transmitters=[node_name])
                                           ^^^^^^^^
UnboundLocalError: cannot access local variable 'pdo_name' where it is not associated with a value

C:\Users\marti\Downloads\eds-files>canconvert QG_Ftype_3_axis_8g_v6.2.eds test.dbc
xls is not supported
xlsx is not supported
INFO - convert - Importing QG_Ftype_3_axis_8g_v6.2.eds ...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Scripts\canconvert.exe\__main__.py", line 7, in <module>
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\cli\convert.py", line 158, in cli_convert
    canmatrix.convert.convert(infile, outfile, **options)
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\convert.py", line 68, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 71, in loadp
    return load(fileObject, import_type, key, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 90, in load
    dbs[key] = module_instance.load(file_object, **options)  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\eds.py", line 150, in load
    frame = canmatrix.canmatrix.Frame(name=pdo_name, transmitters=[node_name])
                                           ^^^^^^^^
UnboundLocalError: cannot access local variable 'pdo_name' where it is not associated with a value

C:\Users\marti\Downloads\eds-files>canconvert sw001454.eds test.dbc
xls is not supported
xlsx is not supported
INFO - convert - Importing sw001454.eds ...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Scripts\canconvert.exe\__main__.py", line 7, in <module>
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\cli\convert.py", line 158, in cli_convert
    canmatrix.convert.convert(infile, outfile, **options)
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\convert.py", line 68, in convert
    dbs = canmatrix.formats.loadp(infile, **options)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 71, in loadp
    return load(fileObject, import_type, key, **options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\__init__.py", line 90, in load
    dbs[key] = module_instance.load(file_object, **options)  # type: ignore
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\marti\AppData\Local\Programs\Python\Python311\Lib\site-packages\canmatrix\formats\eds.py", line 150, in load
    frame = canmatrix.canmatrix.Frame(name=pdo_name, transmitters=[node_name])
                                           ^^^^^^^^
UnboundLocalError: cannot access local variable 'pdo_name' where it is not associated with a value

It would be awesome to get this fully tuned for EDS to DBC conversion and potentially integrated into the canmatrix master - this would help a ton of our users.

@MatinF
Copy link

MatinF commented Jan 7, 2025

@mrfrenzy I also took a look at your example EDS and DBC, which is a great overview. There are some things I am unsure about, however.

If we look at the Transmit PDO with ID 0x280, it contains the signal POS_Display. As evident from e.g. an EDS viewer, this signal is found in the first 32 bits of the payload:

image

Your DBC file indeed matches this, resulting in a raw value that may be 0x00 to 0xFFFFFFFF.

My challenge is this: In your DBC (and the current canmatrix decoding), the scale factor is 1 with an offset of 0. As a result, the physical value will not be within 0 to 100. It seems to me that the current logic is missing the extraction of the scale/offset values. If we want POS_Display restricted to the range of 0 to 100%, the scaling factor should be modified to 1/42949672.95 = 2.328306437080797e-8.

However, I may be missing some aspect of this. Further, I am unable to find the information about the physical value representation/limits (0 to 100%) outside of the PDF - it does not seem to me that this information is found within the EDS?

Presumably, this information could have been added to some extent through the use of e.g. two lines as below:

LowLimit=0
HighLimit=100

However, I did not find limit definitions for POS_Display specifically. Maybe this is due to an actual error in the EDS. But even if the limit information was there, I assume currently this is not used by canmatrix to attempt to identify a scale factor?

ebroecker added a commit that referenced this issue Jan 8, 2025
ebroecker added a commit that referenced this issue Jan 8, 2025
@ebroecker
Copy link
Owner

Hi,

"UnboundLocalError: cannot access local variable 'pdo_name' where it is not associated with a value" is fixed in branch :
https://github.com/ebroecker/canmatrix/tree/eds_bugfixing

I also tried to implement factor and offset.

@MatinF
Copy link

MatinF commented Jan 8, 2025

Thanks Eduard, now this converts the various EDS files I have in my test folder.

Some observations about the outputs:

  1. There seems to be a DBC syntax error in most DBCs created when I check via trying to open them via CANDB++, e.g. sw001454.dbc from the samples I sent does not open. It seems to be related to some missing entries perhaps in the top section related to the use of sender/receiver nodes later in the DBC

  2. There is generally a number of uses of " in the DBC due to direct extraction of this from the EDS. Maybe these can get replaced to e.g. ' or something in order to avoid this, as the character is considered invalid in DBC files

  3. There are several cases where there are issues with the sender/receiver nodes. E.g. some use spaces in the name etc, some are missing for some signals and some are missing for some messages - all of these cause the DBC to not open via CANDB++, but I think they are fairly quick to solve. A simple overall solution may be to skip these entirely and just use the Vector__XXX for all of these - this way I think a lot of robustness is gained with no loss in terms of data value.

  4. There are multiple cases where the names are too long in the EDS, not sure if there's a great solution to this other than adding a somewhat arbitrary replacement/capping rule, so this is more as an observation.

  5. Perhaps most importantly it seems like signal names are not getting extracted within the relevant PDOs correctly. Below is an example for the 1115.eds file where there are multiple PDOs with signals mapped, e.g. below:

image

However, the message in that example screenshot seems to contain some default dummy values:


BO_ 641 Transmit_PDO_2_Mapping: 8 
 SG_ PDO_Mapping_Entry0 : 0|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry1 : 32|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry2 : 64|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry3 : 96|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry4 : 128|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry5 : 160|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry6 : 192|32@1+ (1,0) [0|4294967295] "" PLC
 SG_ PDO_Mapping_Entry7 : 224|32@1+ (1,0) [0|4294967295] "" PLC

ebroecker added a commit that referenced this issue Jan 8, 2025
frame-names starting with digit
#488
@ebroecker
Copy link
Owner

Hi @MatinF ,

  1. and maybe 2. and 3. should be fixed

  2. long names are no problem, there are some warnings, but it should work out of the box

for 5. I have no solution for now, maybe I am not using the canopen-python

@MatinF
Copy link

MatinF commented Jan 9, 2025

Hi Eduard, thanks for the updates.

I have created a small script below that essentially identifies the various mapped parameters linked to the receive/transmit PDOs and which then correctly loads the start bit, bit length, name and data type (required to e.g. determined if signed/unsigned). I think this should be fairly simple to incorporate into your existing EDS code in order to correctly load the mapped signals and link them to the PDOs.

As for the scale/offset aspect, this (unfortunately) seems to be non-trivial. From what I can tell, EDS files do not directly incorporate this information in the same way that DBC files does. As such, some of the EDS files I have provide this detail in "comments" below the parameters, but in a completely non-standardized manner, making it impossible to extract this directly. But users will have a much better starting point for their DBC if we can make these final updates that will allow them to quickly get all the PDOs mapped correctly.

If you get to update this, I will try out the updated version subsequently.

import canopen

# Define the Node ID (default is 0)
node_id = 0

# Load the EDS file into the object dictionary
object_dictionary = canopen.import_od("1115.eds", node_id=node_id)

# CANopen data type mapping
DATA_TYPE_NAMES = {
    0x01: "BOOLEAN",
    0x02: "INTEGER8",
    0x03: "INTEGER16",
    0x04: "INTEGER32",
    0x05: "UNSIGNED8",
    0x06: "UNSIGNED16",
    0x07: "UNSIGNED32",
    0x08: "REAL32",
    0x09: "VISIBLE_STRING",
    0x0A: "OCTET_STRING",
    0x0B: "UNICODE_STRING",
    0x0C: "TIME_OF_DAY",
    0x0D: "TIME_DIFFERENCE",
    0x0F: "DOMAIN",
    0x10: "INTEGER24",
    0x11: "REAL64",
    0x12: "INTEGER40",
    0x13: "INTEGER48",
    0x14: "INTEGER56",
    0x15: "INTEGER64",
    0x16: "UNSIGNED24",
    0x18: "UNSIGNED40",
    0x19: "UNSIGNED48",
    0x1A: "UNSIGNED56",
    0x1B: "UNSIGNED64",
}

def get_data_type_name(data_type_code):
    return DATA_TYPE_NAMES.get(data_type_code, f"Unknown (0x{data_type_code:X})")

def extract_pdos(od, comm_range, map_range, pdo_type):
    print(f"\nExtracting {pdo_type} PDOs...")
    for comm_index, map_index in zip(range(comm_range[0], comm_range[1]), range(map_range[0], map_range[1])):
        if comm_index not in od or map_index not in od:
            continue

        # Retrieve the COB-ID
        comm_param = od[comm_index] #od.get(comm_index)
        cob_id_entry = comm_param.get(1) if comm_param else None
        if not cob_id_entry or cob_id_entry.default is None:
            print(f"  Warning: No valid COB-ID found for {pdo_type} PDO at index 0x{comm_index:04X}. Skipping.")
            continue

        cob_id = cob_id_entry.default & 0x7FF
        print(f"\n{pdo_type} PDO at Communication Index: 0x{comm_index:04X}, COB-ID: {cob_id}")

        # Process Mapping Parameters
        mapping_param = od.get(map_index)
        if not mapping_param:
            print(f"  Warning: No mapping parameter found for {pdo_type} PDO at index 0x{map_index:04X}.")
            continue

        num_entries = mapping_param[0].default if 0 in mapping_param else 0
        print(f"  Number of mapped objects: {num_entries}")

        current_bit_start = 0
        for subindex in range(1, num_entries + 1):
            mapping_entry = mapping_param.get(subindex)
            if not mapping_entry or mapping_entry.default is None:
                print(f"  Warning: Subindex {subindex} missing for mapping parameter at 0x{map_index:04X}.")
                continue

            # Decode the mapping entry
            mapping_value = mapping_entry.default
            obj_index = (mapping_value >> 16) & 0xFFFF
            obj_subindex = (mapping_value >> 8) & 0xFF
            bit_length = mapping_value & 0xFF

            # Fetch the mapped object
            mapped_obj = od.get_variable(obj_index, obj_subindex)
            if not mapped_obj:
                print(f"  Warning: Could not find object at Index: 0x{obj_index:04X}, Subindex: {obj_subindex}.")
                continue

            data_type_name = get_data_type_name(mapped_obj.data_type)
            print(f"  Mapped Object {subindex}:")
            print(f"    Name: {mapped_obj.name.replace(' ', '_')}")
            print(f"    Index: 0x{obj_index:04X}")
            print(f"    Subindex: {obj_subindex}")
            print(f"    Start Bit: {current_bit_start}")
            print(f"    Bit Length: {bit_length}")
            print(f"    Data Type: {data_type_name}")

            # Update the bit start position for the next entry
            current_bit_start += bit_length

# Extract RPDOs
extract_pdos(object_dictionary, (0x1400, 0x1408), (0x1600, 0x1608), "Receive")

# Extract TPDOs
extract_pdos(object_dictionary, (0x1800, 0x1808), (0x1A00, 0x1A08), "Transmit")

@MatinF
Copy link

MatinF commented Jan 9, 2025

As an additional note:

I think in practice it only makes sense for the EDS loading mechanism in canmatrix to support loading of receive/transmit PDOs. This is the part that will be relevant to load (and possibly convert to e.g. DBC) for all use cases I can envision.

The other messages that currently get loaded are essentially CANopen services like EMCY, NMT, SYNC etc. These are 'generic' messages that are always in every EDS file and have no unique information per CAN node that is relevant in the context of loading the database like this.

To some extent, the same goes for the SDOs (Service Data Objects). These have an identical role to OBD2/UDS request/response messages, which also implies that there is no real value to adding the SDO messages in the DBC unless one also completes them with very extensive multiplexing information.

For example, you can use an SDO request to request data on a specific parameter by adding the relevant 'index' of that parameter in the data bytes. Here, an SDO request can e.g. read data from a CANopen slave by adding the requested data parameter index in bytes 1 to 3 (counting from zero). The slave responds with the requested value and again stores the index in bytes 1 to 3 and the value in bytes 4 to 7. In principle, this could be encoded into the DBC file by using extended multiplexing and essentially adding all the parameters that can be requested and correctly adding their indices as multiplexors. But I think it may be a more complex addition.

In short, for the purpose of getting an initial 'working version' of EDS support, I think adding the PDOs only would be a good way to start. This would also ease a work flow where users might have an EDS for each CANopen node and they want to quickly create a DBC for each of them. Here it would be a problem if each DBC contains the same 'generic' messages, in particular if they contain no real value for the decoding. But open to inputs on this.

@MatinF
Copy link

MatinF commented Jan 9, 2025

Another update on this:

For the sake of good order, I tried digging a bit into whether it would be possible to include at least the SDO service related to 'reading data' from different nodes, as I think this will often come into play in practice, along with the PDO based data.

I have a small MF4 log file from our CANedge with some data on which I've tested the below mini DBC file:

VERSION ""


NS_ : 
	NS_DESC_
	CM_
	BA_DEF_
	BA_
	VAL_
	CAT_DEF_
	CAT_
	FILTER
	BA_DEF_DEF_
	EV_DATA_
	ENVVAR_DATA_
	SGTYPE_
	SGTYPE_VAL_
	BA_DEF_SGTYPE_
	BA_SGTYPE_
	SIG_TYPE_REF_
	VAL_TABLE_
	SIG_GROUP_
	SIG_VALTYPE_
	SIGTYPE_VALTYPE_
	BO_TX_BU_
	BA_DEF_REL_
	BA_REL_
	BA_DEF_DEF_REL_
	BU_SG_REL_
	BU_EV_REL_
	BU_BO_REL_
	SG_MUL_VAL_

BS_:

BU_:


BO_ 1409 SDO_TX: 8 Vector__XXX
 SG_ CMD_CCS M : 5|3@1+ (1,0) [0|0] "" Vector__XXX
 SG_ Index m2M : 8|24@1+ (1,0) [0|0] "" Vector__XXX
 SG_ Motor_temperature m8229 : 32|8@1+ (1,0) [0|0] "degC" Vector__XXX
 SG_ Controller_temperature m73766 : 32|8@1+ (1,0) [0|0] "" Vector__XXX
 SG_ APP_1_Max_Fw_Velocity m1060865 : 32|32@1- (1,0) [0|0] "" Vector__XXX
 SG_ Torque_regulator_actual m139291 : 32|16@1- (1,0) [0|0] "quants" Vector__XXX
 SG_ Throttle_voltage m73840 : 32|16@1- (1,0) [0|0] "mV" Vector__XXX


BA_DEF_  "BusType" STRING ;
BA_DEF_DEF_  "BusType" "CAN";

SG_MUL_VAL_ 1409 Motor_temperature Index 8229-8229;
SG_MUL_VAL_ 1409 Index CMD_CCS 2-2;
SG_MUL_VAL_ 1409 Controller_temperature Index 73766-73766;
SG_MUL_VAL_ 1409 APP_1_Max_Fw_Velocity Index 1060865-1060865;
SG_MUL_VAL_ 1409 Torque_regulator_actual Index 139291-139291;
SG_MUL_VAL_ 1409 Throttle_voltage Index 73840-73840;


You can also see the DBC visualized in an editor below:

image

The DBC uses the following logic:

  1. It contains an SDO response message named SDO_TX with ID 1409 (0x581) (which assumes the node in question has node ID 1)

  2. In this message, it starts by defining the command byte CMD_CCS signal (client command specifier). When the value of this field is 2, it corresponds to a response message with data (the length can be 1, 2, 3 or 4 bytes). We can use this CCS field as an initial multiplexor value to ensure we do not e.g. target other types of services (similar logic as we used back in the day for creating the OBD2 DBC file with using the service/mode as the multiplexor).

  3. Further, we create another multiplexor that comes into play when CMD_CCS = 2 which we call Index. This signal is 3 bytes long and combines the CANopen object dictionary 'index' and 'subindex' into a single value for convenience. Again, in the OBD2 terminology, this is the 'PID' equivalent. The Index signal becomes a new multiplexor for the various signals.

  4. With this, we can essentially include every object there is in the EDS file as multiplexed signals. Each object comes with an index and optional subindex. If there is no subindex in the OD, the value of the subindex is simply 0. We can now consider e.g. below trace example, where I've focused on the response message containing the signal Throttle_voltage:

image

As evident from the example, the 3-byte Index signal value is equal to 0x012070 (due to Intel byte order), i.e. 73840, which is what we use as the multiplexor value for Index when we want to trigger the Throttle_voltage signal. In a similar way, we can continue through the various signals.

If canmatrix supports extended multiplexing like this, I think it would be a very powerful supplement to be able to create the SDO message in an automated way for users. I have also created a short script below that will list all the various data objects along with the relevant details. I hope this helps in enabling the integration of this, but let me know your thoughts.

import canopen

# Define the Node ID (default is 0)
node_id = 0

# Load the EDS file into the object dictionary
print("Loading EDS file...")
object_dictionary = canopen.import_od("1115.eds", node_id=node_id)
print("EDS file loaded successfully.")

# CANopen data type mapping
DATA_TYPE_NAMES = {
    0x01: "BOOLEAN",
    0x02: "INTEGER8",
    0x03: "INTEGER16",
    0x04: "INTEGER32",
    0x05: "UNSIGNED8",
    0x06: "UNSIGNED16",
    0x07: "UNSIGNED32",
    0x08: "REAL32",
    0x09: "VISIBLE_STRING",
    0x0A: "OCTET_STRING",
    0x0B: "UNICODE_STRING",
    0x0C: "TIME_OF_DAY",
    0x0D: "TIME_DIFFERENCE",
    0x0F: "DOMAIN",
    0x10: "INTEGER24",
    0x11: "REAL64",
    0x12: "INTEGER40",
    0x13: "INTEGER48",
    0x14: "INTEGER56",
    0x15: "INTEGER64",
    0x16: "UNSIGNED24",
    0x18: "UNSIGNED40",
    0x19: "UNSIGNED48",
    0x1A: "UNSIGNED56",
    0x1B: "UNSIGNED64",
}

def get_data_type_name(data_type_code):
    return DATA_TYPE_NAMES.get(data_type_code, f"Unknown (0x{data_type_code:X})")

def format_index(index, subindex):
    return f"Index: 0x{index:04X}{subindex:02X}"

def get_bit_length(data_type_code):
    # Extract bit length based on the data type name
    data_type_name = get_data_type_name(data_type_code)
    if "INTEGER" in data_type_name or "UNSIGNED" in data_type_name or "REAL" in data_type_name:
        return int(data_type_name.split("INTEGER")[-1].split("UNSIGNED")[-1].split("REAL")[-1])
    elif data_type_name == "BOOLEAN":
        return 1
    else:
        return "N/A"

def list_all_objects(od):
    print("\nListing all objects in the Object Dictionary:")
    for obj in od.values():
        if isinstance(obj, canopen.objectdictionary.ODVariable):
            subindex = 0
            combined_value = int(f"{subindex:02X}{obj.index:04X}", 16)
            print(f"\n{format_index(obj.index, subindex)}")
            print(f"  M: {combined_value}")
            print(f"  Subindex (dec): {subindex}")
            print(f"  Name: {obj.name.replace(' ', '_')}")
            print(f"  Bit Length: {get_bit_length(obj.data_type)}")
            print(f"  Type: Variable")
            print(f"  Data Type: {get_data_type_name(obj.data_type)}")
            print(f"  Access Type: {obj.access_type}")
            print(f"  Default Value: {obj.default}")
        elif isinstance(obj, canopen.objectdictionary.ODRecord):
            for subobj in obj.values():
                combined_value = int(f"{subobj.subindex:02X}{obj.index:04X}", 16)
                print(f"\n{format_index(obj.index, subobj.subindex)}")
                print(f"  M: {combined_value}")
                print(f"  Subindex (dec): {subobj.subindex}")
                print(f"  Name: {subobj.name.replace(' ', '_')}")
                print(f"  Bit Length: {get_bit_length(subobj.data_type)}")
                print(f"  Type: Record")
                print(f"  Data Type: {get_data_type_name(subobj.data_type)}")
                print(f"  Access Type: {subobj.access_type}")
                print(f"  Default Value: {subobj.default}")
        elif isinstance(obj, canopen.objectdictionary.ODArray):
            for subobj in obj.values():
                combined_value = int(f"{subobj.subindex:02X}{obj.index:04X}", 16)
                print(f"\n{format_index(obj.index, subobj.subindex)}")
                print(f"  M: {combined_value}")
                print(f"  Subindex (dec): {subobj.subindex}")
                print(f"  Name: {subobj.name.replace(' ', '_')}")
                print(f"  Bit Length: {get_bit_length(subobj.data_type)}")
                print(f"  Type: Array")
                print(f"  Data Type: {get_data_type_name(subobj.data_type)}")
                print(f"  Access Type: {subobj.access_type}")
                print(f"  Default Value: {subobj.default}")

# List all objects in the Object Dictionary
list_all_objects(object_dictionary)


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants