Build failed

[metanav]

I am compiling using the command:

mbed compile -t GCC_ARM -m K64F --profile=./uTensor/build_profile/release.json

Which gives these errors:

Compile [ 99.5%]: simple_mnist.cpp
[Warning] uTensor_util.hpp@53,30: format '%u' expects argument of type 'unsigned int', but argument 2 has type 'long unsigned int' [-Wformat=]
[Error] simple_mnist.cpp@274,18: 'QuantizedAddOp' does not name a type; did you mean 'QuantizeV2Op'?
[Error] simple_mnist.cpp@274,40: expected primary-expression before ',' token
[Error] simple_mnist.cpp@274,49: expected primary-expression before ',' token
[Error] simple_mnist.cpp@274,51: expected primary-expression before 'int'
[Error] simple_mnist.cpp@335,18: 'QuantizedAddOp' does not name a type; did you mean 'QuantizeV2Op'?
[Error] simple_mnist.cpp@335,40: expected primary-expression before ',' token
[Error] simple_mnist.cpp@335,49: expected primary-expression before ',' token
[Error] simple_mnist.cpp@335,51: expected primary-expression before 'int'
[Error] simple_mnist.cpp@445,18: 'QuantizedAddOp' does not name a type; did you mean 'QuantizeV2Op'?
[Error] simple_mnist.cpp@445,40: expected primary-expression before ',' token
[Error] simple_mnist.cpp@445,49: expected primary-expression before ',' token
[Error] simple_mnist.cpp@445,51: expected primary-expression before 'int'
[Warning] ArrayOps.hpp@28,23: comparison between signed and unsigned integer expressions [-Wsign-compare]
[Warning] ArrayOps.hpp@137,23: comparison between signed and unsigned integer expressions [-Wsign-compare]
[ERROR] In file included from ./uTensor/uTensor/core/tensor.hpp:10:0,
                 from ./uTensor/uTensor/util/quantization_utils.hpp:6,
                 from ./uTensor/uTensor/ops/MatrixOps.hpp:7,
                 from ./models/simple_mnist.cpp:3:

[metanav]

Ok I checked out the develop branch inside the uTensor and started build again:
The previous error has gone but new errors are:

Compile [ 98.9%]: uTensor_util.cpp
[Error] dirent.h@10,2: #error "<dirent.h> not supported"
[Error] stat.h@27,8: redefinition of 'struct stat'
[ERROR] In file included from /Users/naveen/gcc-arm-none-eabi-7-2017-q4-major/arm-none-eabi/include/dirent.h:7:0,
                 from ./uTensor/uTensor/util/uTensor_util.cpp:3:
/Users/naveen/gcc-arm-none-eabi-7-2017-q4-major/arm-none-eabi/include/sys/dirent.h:10:2: error: #error "<dirent.h> not supported"
 #error "<dirent.h> not supported"
  ^~~~~
In file included from ./uTensor/uTensor/util/uTensor_util.cpp:2:0:
/Users/naveen/gcc-arm-none-eabi-7-2017-q4-major/arm-none-eabi/include/sys/stat.h:27:8: error: redefinition of 'struct stat'
 struct stat
        ^~~~
In file included from ./mbed-os/platform/mbed_error.h:25:0,
                 from ./mbed-os/rtos/Queue.h:30,
                 from ./mbed-os/rtos/Mail.h:28,
                 from ./mbed-os/rtos/rtos.h:34,
                 from ./mbed-os/mbed.h:37,
                 from ./uTensor/uTensor/util/uTensor_util.hpp:104,
                 from ./uTensor/uTensor/util/uTensor_util.cpp:1:
./mbed-os/platform/mbed_retarget.h:453:8: note: previous definition of 'struct stat'

[dboyliao]

Check the include order of simple_mnist.cpp and simple_mnist.hpp
Make sure context.hpp is included before other headers.

It's a known issue, but we just don't have enough time to go through it.

[metanav]

Here are headers:

models/simple_mnist.hpp

#ifndef __MODELS_SIMPLE_MNIST_H
#define __MODELS_SIMPLE_MNIST_H
#include "uTensor/core/context.hpp"
void get_simple_mnist_ctx(Context& ctx, Tensor* input_0);
#endif // __MODELS_SIMPLE_MNIST_H

models/simple_mnist.cpp:

#include "simple_mnist.hpp"
#include "uTensor/ops/MatrixOps.hpp"
#include "uTensor/ops/NnOps.hpp"
#include "uTensor/ops/ArrayOps.hpp"
#include "uTensor/core/tensor.hpp"
#include "uTensor/loaders/tensorIdxImporter.hpp"
#include "uTensor/ops/MathOps.hpp"

main.cpp

#include "simple_mnist.hpp"
#include "tensorIdxImporter.hpp"
#include "tensor.hpp"
#include "FATFileSystem.h"
#include "SDBlockDevice.h"
#include "mbed.h"

Still getting the compilation time errors:

Compile [ 98.9%]: uTensor_util.cpp
[Error] dirent.h@10,2: #error "<dirent.h> not supported"
[Error] stat.h@27,8: redefinition of 'struct stat'

[neil-tan]

@metanav
Thanks for letting us know. Is your uTensor runtime on the develop branch? We are in the process of releasing changes. Sorry for the errors.

@mbartling I will try this out. Anything else you can think of?

Neil

[metanav]

@neil-tan What do you mean by "uTensor runtime"?

Project directory listings:

main.cpp		mbed-os.lib		mbed_settings.py	uTensor/
mbed-os/		mbed_app.json		models/			uTensor.lib

If you meant the "uTensor" directory, that is on the develop branch in my project.

[neil-tan]

@metanav

Could you cd into the uTensor directory and run git branch -a. Try checking out the develop branch if it’s not already on there. If this still doesn’t work, I can try replicate this tonight.

[metanav]

I get this:

* develop
  master

So it is on the develop branch.

[neil-tan]

Ok. I will have a look later

[neil-tan]

Looks like the last known good commit is shown in this file:

https://github.com/uTensor/utensor-mnist-demo/blob/master/uTensor.lib

I will try to see what’s broken. But do please give me a shout if this fixes your problem.

[metanav]

Thanks @neil-tan the build process completed successfully after adding this commit.

The final step is to run on the device (using same compile params with -f) keeps waiting on this line:

Total Static RAM memory (data + bss): 23116 bytes
Total Flash memory (text + data): 338919 bytes

Image: ./BUILD/HEXIWEAR/GCC_ARM/helloworld.bin

It supposed to flash the bin to the device, right? But for some reason mbed is not detecting my device.
I manually copied the bin to the DAPLINK directory and reset the device. But still the Coolterm does not show any messages.
Also, there is red led blinking in some pattern, seems like some error. By the way I am using hexiwear development board with Cortex M4 and 256 SRAM and SD card. I tested the SD card through some other program and it is working. Any clues to debug it?

[neil-tan]

@metanav Hey, a Hexiwear, this is pretty sweet.
Unfortunately, we don't have one here. I can try giving you the best advice though.

Are you using windows, or Mac/Linux? For windows, you might need to install the windows serial driver. The baudrate is usually either 9600 or 115200. I recommend using the "helloworld" example.

Would you be able to post the source code of your SD test program? I can probably a quick look of the pin assignments.

[mbartling]

@metanav you can get around the dirent issue by commenting out the following line.
https://github.com/uTensor/uTensor/blob/develop/uTensor/util/uTensor_util.cpp#L3

[metanav]

@mbartling Thanks I will try.

[metanav]

@neil-tan I am using Mac (High Sierra). I can use screen or Coolterm. What should I expect to see in the case of error?

I used following program to test SD successfully.

#include "mbed.h"
#include "SDFileSystem.h"
 
SDFileSystem sd(PTE3, PTE1, PTE2, PTE4, "sd");

int main() { 
    mkdir("/sd/mydir", 0777);
    FILE *fp = fopen("/sd/mydir/sdtest.txt", "w");
    if(fp == NULL) {
        error("Could not open file for write\n");
    }
    fprintf(fp, "Hello world!");
    fclose(fp);  
}

[neil-tan]

hey @metanav

CoolTerm should be fine. The flashing led probably means a runtime error. Try the most basic printf example first:

#include "mbed.h"              
 
Serial pc(USBTX, USBRX);
 
int main() {
    pc.printf("Echoes back to the screen anything you type\n");
    while(1) {
        pc.putc(pc.getc());
    }
}

My guess is the pin assignment has gone wrong. Check this file, you may add the right SD SPI pin assignments for your target here. This doc explains how mbed_app.json works.

Then, I'd recommend printing something at the beginning of your main(). Once we get that working, the rest of the error messages are easy to spot.

Have you been able to get the SD example working using Mbed-Cli? Were you able to use mbed compile -m auto -f to flash the board? You can use mbedls to list the current connected target's name. In your case, it's probably HEXIWEAR.

[neil-tan]

@metanav Also, try changing the mbed-os version to 5.6. You can do this by cd into the mbed-os directory and checkout the 5.6 branch.

[metanav]

@neil-tan I think the SD pin assignment is fine. I have already set the pin assignment in the mbed_app.json.

"target_overrides": {
        "HEXIWEAR": {
            "sd-mosi": "PTE3",
            "sd-miso": "PTE1",
            "sd-clk": "PTE2",
            "sd-cs": "PTE4"
        }
}

Now after little playing with CoolTerm I was able to see the message:

Simple MNIST end-to-end uTensor cli example (device)
[Error] ./uTensor/uTensor/core/context.cpp:96 @push 
  Tensor "OuputLayer/logits_eightbit/OuputLayer/Variable_1__port__0/reduction_dims:0"
  not found

[metanav]

If I misspelled the tmp.idx as tmp.id it gives me error on the CoolTerm:

[Error] ./uTensor/uTensor/loaders/tensorIdxImporter.cpp:61 @parseMeta Error opening file: /fs/tmp.id

Otherwise it is fine with the correct file path, so I can confirm the SD is working fine.
The problematic area is the function call:

get_simple_mnist_ctx(ctx, input_x);

[dboyliao]

@metanav You just find a bug in my code generator reference counting optimizer.
If you are using the simple_mnist.pb to generate the code, I think I'm looking at the same cpp code as yours.

The "OuputLayer/logits_eightbit/OuputLayer/Variable_1__port__0/reduction_dims:0" tensor has incorrect reference count so it's garbage collected by the ctx object.

I'll fix it this night at Taiwan local time.
Thanks for your patience.

[dboyliao]

@metanav
I just fix the reference counting issue for the code generator.
The "tensor does not exist" issue should be gone now.
You can try this at the develop branch of utensor_cgen.

Let me know if there is any other issue.

[neil-tan]

@metanav If there's any specific things you've done to get uTensor to work on Hexiwear (like mbed_app.json, etc), I'd be interested in a PR : )

[metanav]

@dboyliao
I checked out the develop branch of the utensor_cgen. But still getting the same error:

Simple MNIST end-to-end uTensor cli example (device)
[Error] ./uTensor/uTensor/core/context.cpp:96 @push Tensor 
"OuputLayer/logits_eightbit/OuputLayer/Variable_1__port__0/reduction_dims:0" not found

Also, I tried to do a fresh installation of everything with no success.

[dboyliao]

Ok, I'll check the execution order later.
Thanks!

[neil-tan]

@metanav were you able to share the .pb file you are working with?

[dboyliao]

@neil-tan
I think the pb file is the simple_mnist.pb in our demo repo.
Should be available to everyone.

[metanav]

@neil-tan You can find the pb file in the utensor_cgen repo (develop branch):
tests/deep_mlp/simple_mnist.pb

[dboyliao]

@metanav I look into the generated code and uTensor source code.
I can't see why the tensor is garbage collected.
I have to compile and actually run it to see why.
I don't have free time to do it until next week.

[metanav]

@dboyliao Have you got chance to check it?

[neil-tan]

hey @metanav Not sure if this helps; did you have a chance to take a look at the project creation guide?

[dboyliao]

@metanav
I just get it compiled on ubuntu (with docker) last week.
But I'm kind of busy this week (projects on due for works).
Hopefully I can get it done and tested this weekend.