Alternative Coral Edge TPU object detction models (e.g. Spaghettinet)

[moandcompany]

Hi,

I've been working on setting up and training alternative object detection models for my Frigate and Coral Edge TPU applications.

In the process of building a tailored evaluation set and Frigate-specific test/measurement harness, I compiled the new-ish Spaghettinet object detector for Coral applications. Against the COCO evaluation set, Spaghettinet performs sliightly better than the reference and default model used by Frigate, MobileDet-Edge TPU.

Compared with MobileDets, the new family of SpaghettiNet-EdgeTPU detection models achieves +2.2% mAP (absolute) on COCO at the same latency and consumes less than 70% of the energy used by MobileDet-EdgeTPU to achieve similar accuracy.

Spaghettinet may be of interest to Frigate users as a drop-in model replacement, as it is shares the same COCO labelspace as other commonly used object detectors for Coral.

• Basic latency/throughput tests of Spaghettinet show that it's highly comparable to MobileDet-EdgeTPU in terms of latency/throughput. Frigate users that feel throughput constrained on the detection side may want to try the Spaghettinet small (s) or medium (m) models, but the Spaghettinet large (l) model has similar latency/throughput characteristics to the default Frigate model; using a smaller Spaghettinet model should give you higher system throughput if needed.
• The statistics below were generated by running each of respective models for 101 iterations, discarding the first inference in each trial where model-loading time skews measured inference time.

If you're interested in trying the Coral-compiled Spaghettinet model, I'm making a copy available via Google Drive

[Davy-H]

It looks quite interesting. I'll have to find some time to try it. Improved accuracy would be nice (especially if it reduces the number of false positives) and lower latency would always be nice.

Have you noticed any improved accuracy compared to MobileDet-Edge when used in Frigate?

[divemasterjm]

Hi,

I've been working on setting up and training alternative object detection models for my Frigate and Coral Edge TPU applications.

In the process of building a tailored evaluation set and Frigate-specific test/measurement harness, I compiled the new-ish Spaghettinet object detector for Coral applications. Against the COCO evaluation set, Spaghettinet performs sliightly better than the reference and default model used by Frigate, MobileDet-Edge TPU.

Compared with MobileDets, the new family of SpaghettiNet-EdgeTPU detection models achieves +2.2% mAP (absolute) on COCO at the same latency and consumes less than 70% of the energy used by MobileDet-EdgeTPU to achieve similar accuracy.

Spaghettinet may be of interest to Frigate users as a drop-in model replacement, as it is shares the same COCO labelspace as other commonly used object detectors for Coral.

• Basic latency/throughput tests of Spaghettinet show that it's highly comparable to MobileDet-EdgeTPU in terms of latency/throughput. Frigate users that feel throughput constrained on the detection side may want to try the Spaghettinet small (s) or medium (m) models, but the Spaghettinet large (l) model has similar latency/throughput characteristics to the default Frigate model; using a smaller Spaghettinet model should give you higher system throughput if needed.
• The statistics below were generated by running each of respective models for 101 iterations, discarding the first inference in each trial where model-loading time skews measured inference time.

If you're interested in trying the Coral-compiled Spaghettinet model, I'm making a copy available via Google Drive

Very interesting, can you please tell me ho to configure it in frigate to change it to point to the new model?

Thanks

[Davy-H]

Something like this:

detectors:
model:
height: 320
width: 320
path: /config/spaghettinet_edgetpu.tflite

I found it was about 0.4ms slower on my coral TPU compared to the standard model. I didn't notice any particular improvement in accuracy but I didn't do any particular tests.

[LaurenceGough]

Thanks for this, it sounds great! I see it appears to be originally made for the new Pixel phone object detection.

Edit: The below was my fault Frigate couldn't see it. After setting up another bind mount of the tflite model file it now is working. I'll report back with how it goes! :)

I am getting Frigate failing to boot and the following error when trying to use it, using the name "spaghettinet_edgetpu_l_compiled" or any name for that matter. I have tried different locations too, including the same area of the Frigate config. Permissions appear to be fine (read permissions granted).

 "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 135, in run_detector
    object_detector = LocalObjectDetector(
  File "/opt/frigate/frigate/edgetpu.py", line 45, in __init__
    self.interpreter = tflite.Interpreter(
  File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 348, in __init__
    _interpreter_wrapper.CreateWrapperFromFile(
ValueError: Could not open '/spaghettinet_edgetpu_l_compiled.tflite'.

detectors:
  coral:
    type: edgetpu
    device: usb

model:
  width: 320
  height: 320
  path: /spaghettinet_edgetpu_l_compiled.tflite

Frigate release V11.1

[LaurenceGough]

Hi Nick. I am running it in a docker instance managed via Portainer - running off DietPi OS.

[NickM-27]

Just making sure, you passed the file into portainer and then into frigate docker container right? Frigate won't be able to see the file on the host.

[LaurenceGough]

I might have done things a bit awkwardly compared to that, I used SFTP via WinSCP to copy the file across into the docker-data Frigate volume (same place as the config.yml file). I thought perhaps permissions so I checked them on the file and it is rw-r--r--. I think read access is enough for this file, perhaps I am wrong. I'll test now with full permissions for all.

Thank you

Edit: Same problem sadly with rwxrwxrwx I'll try mapping it in Portainer directly. Thanks.

[NickM-27]

But does frigate pull that in? If you get a shell in the frigate container can you see the model when you run ls / ? If you put it near the frigate config then it definitely won't located in /

[LaurenceGough]

Thanks Nick, yes it was just me being an idiot as usual :D!!! I mapped it as a bind and now it's working, doh!

I'll report back how it goes as I have been fighting false positives vs missed detections with the default for quite a while now.

[LaurenceGough]

Are there any newer versions of this model available?

I've been trying the CCTV optimised version found here https://codeberg.org/Curid/TF-CCTV but it performs far worse than this and the one that comes with Frigate (the standard coral model I believe) unfortunately.

Thanks.

[kjames2001]

Hi,

I've been working on setting up and training alternative object detection models for my Frigate and Coral Edge TPU applications.

In the process of building a tailored evaluation set and Frigate-specific test/measurement harness, I compiled the new-ish Spaghettinet object detector for Coral applications. Against the COCO evaluation set, Spaghettinet performs sliightly better than the reference and default model used by Frigate, MobileDet-Edge TPU.

Compared with MobileDets, the new family of SpaghettiNet-EdgeTPU detection models achieves +2.2% mAP (absolute) on COCO at the same latency and consumes less than 70% of the energy used by MobileDet-EdgeTPU to achieve similar accuracy.

Spaghettinet may be of interest to Frigate users as a drop-in model replacement, as it is shares the same COCO labelspace as other commonly used object detectors for Coral.

* Basic latency/throughput tests of Spaghettinet show that it's highly comparable to MobileDet-EdgeTPU in terms of latency/throughput. Frigate users that feel throughput constrained on the detection side may want to try the Spaghettinet small (s) or medium (m) models, but the Spaghettinet large (l) model has similar latency/throughput characteristics to the default Frigate model; using a smaller Spaghettinet model should give you higher system throughput if needed.

* The statistics below were generated by running each of respective models for 101 iterations, discarding the first inference in each trial where model-loading time skews measured inference time.

If you're interested in trying the Coral-compiled Spaghettinet model, I'm making a copy available via Google Drive

Thank you so much for this model file! Just tested for an afternoon, much more accurate than the default model. Very accurate and fast, and no false positive for me yet.