Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-40][APIService] Add LogClient to fetch log #15966

Open
2 tasks done
Tracked by #14102
ruanwenjun opened this issue May 9, 2024 · 0 comments
Open
2 tasks done
Tracked by #14102

[DSIP-40][APIService] Add LogClient to fetch log #15966

ruanwenjun opened this issue May 9, 2024 · 0 comments

Comments

@ruanwenjun
Copy link
Member

ruanwenjun commented May 9, 2024

Search before asking

  • I had searched in the DSIP and found no similar DSIP.

Motivation

Right now, the log is stored at worker/master and remote storage.
The query logic look like below

image

The LogService is duplicate at worker/master, we need to maintain two service implementation which have the same code.

And once the master/worker is not active, then we can not fetch the log, this problem may occur frequently in k8s.

So it's better to create a LogClient at API side to manage the log fetch logic, and create a LogServer to response for the client request, the LogServer will be imported at master/worker.

Design Detail

image
  1. We will fetch the log from master/worker firstly.
  2. If the master/worker not exist or we cannot find the log file from master/worker then we will try to fetch from remote log storage.

Compatibility, Deprecation, and Migration Plan

Compatibility with previous version.

Test Plan

Test by manual.

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant