Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Memory leak with TBB 2020.3 #24483

Open
3 tasks done
shipp108 opened this issue May 13, 2024 · 0 comments
Open
3 tasks done

[Bug]: Memory leak with TBB 2020.3 #24483

shipp108 opened this issue May 13, 2024 · 0 comments
Assignees
Labels
bug Something isn't working support_request

Comments

@shipp108
Copy link

OpenVINO Version

tag 2023.3.0

Operating System

Ubuntu 18.04 (LTS)

Device used for inference

CPU

Framework

None

Model used

No response

Issue description

When multiple inferences are performed on openvino compiled based on TBB 2020.3, VmHWM and VmRSS will continue to grow.

But the memory scan tool does not explicitly reveal information:

  • Asan has no memory leak report
  • Valgrind has only "possibly lost" and "still reachable" leak reports, and there are stack memory requests associated with pthread_create that are not released. I have also adjusted and tested task_scheduler_init::.blocking_terminate(std::nothrow_t{}) of ExecutorManagerImpl, but it does not work.

And for the (TBB_VERSION_MAJOR > 2020) tbb API, there are no memory issues.

After debugging, and ref [destroy](tbb::task::destroy)'s annotation "Usually, calling this method is unnecessary, because a task is implicitly deleted after its execute() method runs. However, sometimes a task needs to be explicitly deallocated, such as when a root task is used as the parent in spawn_and_wait_for_all."

the problem should be tbb::task::allocate_root() is not released.

After the following adjustments, the VmHWM and VmRSS are in stable state

Add the tbb::task::destroy(root); after root.spawn_and_wait_for_all(a);.

class UpdateNodes : public UpdateNodesBase {
public:
    using UpdateNodesBase::UpdateNodesBase;
    void run(size_t stopIndx) override {
        m_completion.store(false);
        auto startCounter = m_prepareCounter.load();
        tbb::task& root = *new(tbb::task::allocate_root()) tbb::empty_task;
        root.set_ref_count(3); // two for children and one preserved

        auto task1 = [this](size_t start, size_t stop) {
            this->updateShapes(start, stop);
        };
        AsyncTask<decltype(task1)>& a = *new (root.allocate_child()) AsyncTask<decltype(task1)>(task1, startCounter, stopIndx);

        auto task2 = [this](size_t start, size_t stop) {
            this->updateDynParams(start, stop);
        };
        AsyncTask<decltype(task2)>& b = *new (root.allocate_child()) AsyncTask<decltype(task2)>(task2, startCounter, stopIndx);

        b.set_affinity(2); // slot 1 plus 1
        tbb::task::spawn(b);
        root.spawn_and_wait_for_all(a);

        tbb::task::destroy(root);
    }
};

Please help confirm this problem.

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

  • I'm reporting an issue. It's not a question.
  • I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
  • There is reproducer code and related data files such as images, videos, models, etc.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working support_request
Projects
None yet
Development

No branches or pull requests

2 participants