Import simple NodeSet2 file is incomplete

[eoursel]

Describe the bug
When i try to import this NodeSet2 file which is just a useless type which contains some custom datatypes i observe several issues

• ObjectTypes are empty (HumanType, VehicleType, TricycleType). They should contains some variables in Children.
• The field wheel in TriccyelDataType should be an array.

To Reproduce
Import the attached NodeSet2 file.
Opc.Ua.NodeSet2.TriCycleType_V1.1.xml.zip

Additional context
Last source code from master branch

Best regards

[mpostol]

Blocker for mpostol/ASMD#158

[mpostol]

Transfer because the main functionality is implemented here.

[mpostol]

Further development will be conducted on the branch https://github.com/mpostol/OPC-UA-OOI/tree/SemanticData-6.1.3

[mpostol]

Testing scope

• possibility to import NodeSet file
• possibility to validate NodeSet file
• possibility to convert NodeSet to ModelDesign
• possibility to open in the ASMD
• possibility to compile ModelDesign and generate NodeSet file (round trip test)
• validation against the schema and compliance with the specification (Part 3 and Part 5) of the NodeSet file
• correct generation of the model documentation in MS Word format
• correct generation of website content with the model documentation

[mpostol]

@eoursel the result of ValidateAndExportModel is as follows:

Test Name:	eoursel510Test
Test Outcome:	Passed
Result StandardOutput:	
Debug Trace:
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering IAddressSpaceContext.ValidateAndExportModel - starting for the http://tricycletypev1/ namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ValidateAndExportModel - selected 8 nodes to be added to the model.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 8 nodes.

generic export implementation has been used for testing.

[eoursel]

hi @mpostol,
in the nodeset files there are more than 8 nodes. looks like the variables in the ObjectTypes are lost. Could you post a copy of the generated model design file?

Thanks again

[mpostol]

@eoursel thanks, I haven't yet generated the ModelDesign file. The task possibility to convert NodeSet to ModelDesign is scheduled next` (see Testing scope above). Thanks for the feedback, I will check it before forwarding it to the next step, because if any node is not selected in this step it will be omitted in the phase of ModelDesign generation for sure. We must fix the problem now. I will be back.
Thanks,

[mpostol]

@eoursel Except for simple errors in the code (let exclude this case for a while), the reason for the problem you have reported is conditions used to select nodes to validate and export. Now the method ValidateAndExportModel uses conditions defined in Part 5 8 Standard Objects and their Variables - it exports selected model from an internal Address Space. It means that

• if the node is not on the browse path available for an abstract client it is neglected.

Related issues:

[ ] reconsider conditions used for selection of Nodes
[ ] clarify what the the model contains 8 nodes actually means
[ ] add a warning that the AS contains nodes orphaned and inaccessible for browsing starting from the Root node

Comments are welcome, but for now, I will report the above-mentioned problems.

[eoursel]

hi @mpostol, i don't get your point. All the nodes are on the browse path. Do you have a concrete example of such a node? . I am using Siemens Siome which do not display any error in the validation of the NodeSet file.

Regards

[mpostol]

@eoursel It is just a hypothesis. I will check it. At least this scenario is now not subject of validation, but in my opinion should be, shouldn't it? After improving this validation method I will get the list of nodes if any. I will work on it using above mentioned independent issues. I will be back with the results soon.

[mpostol]

@eoursel do you have a copy of the document OPC UA Companion Specification Template. It is not available on the OPC website - the page is empty for my account.

I am collecting materials from the spec that can be used to resolve some questions related to inheritance, inherited member, and instance declaration in the context of your problem.

[eoursel]

Hi @mpostol should be in the harmonization working group. I am checking.

[mpostol]

Hi @eoursel, I have improved the error processing and logging infrastructure and the current result for the model in concern processing is as follows:

Test Name:	eoursel510Test
Test Outcome:	Passed
Result StandardOutput:	
Debug Trace:
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext creator - starting creation the OPC UA Address Space.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - the OPC UA defined has been uploaded.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import UAOOI.SemanticData.UANodeSetValidation.XML.UAModelContext.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - the context for the imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 3909 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Address Space - has bee created successfully.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContextService.ImportUANodeSet - importing form file
Trace: Information, Error Focus:Diagnostic, ErrorID: P0-0001010000 Info: The XML attribute or element is not supported and neglected. Extensions is omitted during the import
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering AddressSpaceContext.ImportNodeSet - starting import UAOOI.SemanticData.UANodeSetValidation.XML.UAModelContext.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information AddressSpaceContext.ImportNodeSet - the context for the imported model is created and starting import nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing AddressSpaceContext.ImportNodeSet - imported 54 nodes.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Entering IAddressSpaceContext.ValidateAndExportModel - starting for the http://tricycletypev1/ namespace.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Selected 8 types to be validated.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Selected 0 instances referenced by the ObjectsFolder to be validated.
Trace: Verbose, Error Focus:Diagnostic, ErrorID: P0-0003010000 Info: It is diagnostic information Finishing Validator.ValidateExportModel - the model contains 8 nodes, and nor errors reported
After removing inherited and instance declaration nodes the recovered information model contains 8

The result says that the model contains 54 nodes, but only 8 types and 0 instances referenced by the ObjectsFolder are selected at the beginning to be processed. I have not implemented reporting orphans - it is more complicated than expected but we can assume that there is a lot of nodes no selected for processing or by design removed as modeling rules, instance declaration, or inherited. To be processed a node must be selected at the very beginning or must be on any reference chain. Initial choice aggregates all types and instances referenced by ObjectFolder standard object. To properly implement an orphaned nodes search mechanism let me analyze a snipped from the code

    <UAObjectType NodeId="ns=1;i=12" BrowseName="1:VehicleType">
        <DisplayName>VehicleType</DisplayName>
        <References>
            <Reference ReferenceType="HasSubtype" IsForward="false">i=58</Reference>
        </References>
    </UAObjectType>
    <UAVariable DataType="DateTime" NodeId="ns=1;i=13" BrowseName="buildDate" ParentNodeId="ns=1;i=12">
        <DisplayName>buildDate</DisplayName>
        <References>
            <Reference ReferenceType="HasProperty" IsForward="false">ns=1;i=12</Reference>
            <Reference ReferenceType="HasTypeDefinition">i=63</Reference>
            <Reference ReferenceType="HasModellingRule">i=78</Reference>
        </References>
    </UAVariable>

The UAVariable NodeId="ns=1;i=13" is not referenced at all. It is not derived from the inheritance chain. Therefore it is recognized as orphaned and not considered for further processing. According to the spec Table F.6 – UANodeSet Instance Nodes ParentNodeId="ns=1;i=12"> cannot be used to expose a member in the Address Space. The Node is redundant, so we cannot say that the UANodeSet document is not valid. It is like a not referenced variable in software development.

Any comments are welcome. I will wait for your opinion.

[eoursel]

Hi @mpostol but there is the HasProperty reference to link i=13 to i=12 with the IsForward = false. As a result the i=13 node is defined as a property of the object type i=12.

And so this dammed variable is not orphaned but attached by a reverse reference.

Because it is not forbidden by the schema to make reverse references some nodeset authors use this construct which is completely brain damaged I must admit.

[mpostol]

@eoursel I see. You are right and it makes sense to add randomly new components (members). I was sure that IsForward is restricted for HasProperty and must be true. It looks like a bug in my references resolving mechanism - it is a really challenging algorithm. I will check it and let you know after tomorrow. Therefore I like to say ModelDesign recovery instead of ModelDesign export - it is reverse engineering, as the UANodeSet documents are always generated by a compiler.

[mpostol]

@eoursel working on the section Address Space Concept Executive Summary - in a document where I try to work out a conclusion of our findings - I try to figure out a scenario (sequence of services defined in Part 4)) for the OPC UA Client that it shall apply to get access to the NodeId="ns=1;i=13".

Can you help me in this respect?

[eoursel]

@mpostol I can't get your point. For sure if you browse the Node VehicleType you should find a reference HasProperty to the node BuildDate. I just checked with UaExpert or any other client.

I think that the way the NodeSet import is working in a server is that the two options are semantically identical

1. Node A -> HasReference Is Forward True -> Node B
2. Node B -> HasReference is Forward False -> Node A

The server will create in memory
Node A -> HasReference -> Node B

This is true for any any kind of reference. Not only HasProperty.

[mpostol]

@eoursel I am not saying that something is wrong but I ony try to recover a services call sequence a client shall apply to browse to NodeId="ns=1;i=13". I 100% agree that the two options are semantically identical.

As you know I am not checking the results against any selected product, but rather against the specification. Of course, any results from the existing products are more than welcome.

In the context of references, I am using three terms: SourceNode, TargetNode, and ParentNode. In the specification Part 3 4.3.4 References we can read:

The Node that contains the Reference is referred to as the `SourceNode` and the Node that is referenced is referred to as the `TargetNode`.

Of course, it is not the only place defining the reference term but it doesn't mention something about direction. From this text, we can learn that the reference shall be added only to one Node - I call it ParentNode. Additionally, it seems to me that the meaning of the ParentNode is different for the model (design-time) and OPC UA Client API exposed by a server. In the method GetMyReferences (see attachment) I get used ParentNode. After changing it to SourceNode I am getting a different result. My concern is if I should generate a warning in this respect - as you said "which is completely brain damaged".

[mpostol]

@eoursel, The BuildError.Identifier provides details allowing to select of the appropriate section in the specification that has been used to derive this validation rule. The BuildError entries are preliminary and subject to further modifications. In the last commit, I have added a formal description of the Identifier. In the document, you can find an example related to this particular case.

It is hard to say if a reported problem is a critical one or just a warning because the specification is inconsistent, incomplete and work is in progress. At the very beginning, as a corporate member, I had submitted a preliminary list of IM validation problems as a starting point for further work on IM compliance to promote the reusability of the models at design-time. Unfortunately, there was no follow-up, so the work was put on hold. The list is prepared in XML to guarantee portability between different environments.

[mpostol]

Hi @eoursel, I hope you are still following me. To move forward the discussion related to the errors severity level I have abstracted a bunch of snippets referring to the BrowseName from the specification.

I am trying to find a rule that can be applied to the model BrowseNameInheritedFrom0.xml to distinguish that EngineeringUnits is the BrowseName defined by OPCUA but tickness is defined by a custom model, but not OPCUA.
Any comments are welcome.

[mpostol]

The BaseType reference for node TrailerType is not valid: VehicleType.

Compilation fails if the base type (VehicleType) is located in the ModelDesign file later than the derived type (TrailerType). Because the ModelDesign is not standard this behavior could not be recognized as an error.

[mpostol]

@eoursel it looks like your model has circular references created using the HasChild. Can you confirm?

From the spec:

P03-0705 HasChild ReferenceType - the HasChild ReferenceType is an abstract ReferenceType; only subtypes of it can be used. It is a subtype of HierarchicalReferences.
The semantic is to indicate that References of this type span a non-looping hierarchy.
Starting from Node “A” and only following References of the subtypes of the HasChild ReferenceType it shall never be possible to return to “A”. But it is allowed that following the References there may be more than one path leading to another Node “B”.

My point is that it should be recognized as a critical error.
Waiting for your opinion.

[eoursel]

<!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} code {mso-style-priority:99; font-family:"Courier New";} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 70.85pt 70.85pt;} div.WordSection1 {page:WordSection1;} -->Hi @mpostol  I can’t see this refereence TrailerType -> HasOreredComponent -> TrailerType when i open the model with Siemens Siome.  But there is a reference to a variable LoadedTricycle with a DataType TriCycleDataType.  It looks good for me.  I don’t understand how you can construct this circular reference that Siome do not detect.   That’s said, i don’t want to attach a great value to this model, it was designed to  test the behavior of complex types with the C# code generator  and i am not the author of this model. Today, i would like to focus on VDMA models (Robotics Machine Vision) , AutomationML and Asset Administration Shells. I think we need support for C# code generation for these models which is not the case today.  Regards    De : MariuszEnvoyé le :jeudi 18 mars 2021 22:34À : mpostol/OPC-UA-OOICc : Eric OURSEL; MentionObjet :Re: [mpostol/OPC-UA-OOI] Import simple NodeSet2 file is incomplete ***@***.*** it looks like your model has circular references created using the HierhicalReferences. Can you confirm?From the spec:P03-0705 HasChild ReferenceType - the HasChild ReferenceType is an abstract ReferenceType; only subtypes of it can be used. It is a subtype of HierarchicalReferences.The semantic is to indicate that References of this type span a non-looping hierarchy.Starting from Node “A” and only following References of the subtypes of the HasChild ReferenceType it shall never be possible to return to “A”. But it is allowed that following the References there may be more than one path leading to another Node “B”.My point is that it should be recognized as a critical error.Waiting for your opinion.—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe. 

[mpostol]

@eoursel thanks for the feedback. Of course, my concern is not the model itself. I know that the model is just an example without any production value. My concern is how to prove that the methodology is correct. Recovering ModelDesign from UANodeSet is a complex process, therefore first I try to prove that the result is as expected.

From your comment, I can derive that the recovered Opc.Ua.NodeSet2.TriCycleType_V1.1.ModelDesign.xml
model is wrong. Therefore I must fix it first to avoid problems like that with production models. Of course, the models you mentioned are on my roadmap.

Refereing to the C# code genrator. My point is that it is much easier to implement UANodeSet => C# generator than UANodeSet =>ModelDesign=>C# multi-stage, multitool process. In all of the cases, we are discussing the domain-specific language (DSL). Following this idea consider starting from C#, and in this case, we need C#=>ModelDesign converter. The question is which one DSL should be the source. There is no universal answer for sure. The answer must be a result of research but not just a decision.

Anyway, I am open to any suggestions. This work is community-driven so your opinion matters.

[mpostol]

@eoursel let me inform you that I have just generated the recovered ModelDesign. This folder also contains other associated files. The diagnostic log is attached to this comment.

Hopefully, the content is as expected. Let me know how it works for you.

RecoveryTrace.log

[mpostol]

@eoursel it will be continued in mpostol/UA-Nodeset#4.