Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

API request to contents/ is slow #2674

Open
ziggystar opened this issue Feb 11, 2021 · 6 comments · May be fixed by #2873
Open

API request to contents/ is slow #2674

ziggystar opened this issue Feb 11, 2021 · 6 comments · May be fixed by #2873
Labels
APIv3 GitHub compatible API improvement

Comments

@ziggystar
Copy link
Contributor

Before submitting an issue to GitBucket I have first:

Issue

Impacted version: 4.35.3

Deployment mode: Tomcat

Problem description:

  • accessing the content API call /api/v3/repos/<group>/<repo>/contents with or without ?ref is slow for a large repository

I have a large repository (17k commits, 700MB on disk). The API call to contents/ to the root, which contains 49 objects (dirs and files) takes 10 seconds.

  • a call to a directory with 104 entries takes 32s
  • 30 entries takes 5s
  • 10 entries 2.5s

This also causes a problem with Jenkins, since within the log I see all those calls running into a time out.

I think this is a performance problem in gitbucket, because to me it appears that this call should perform much faster.
@takezoe takezoe added APIv3 GitHub compatible API improvement labels Feb 11, 2021
@ziggystar
Copy link
Contributor Author

ziggystar commented Feb 11, 2021 •
edited

I've found the MaxFiles and lowering it (to 10) solves the problem. Apparently the problem seems to be caused by the used git library, which appears to be rather slow.

As I understand, to compute the result for the contents API call, it is not necessary to use the complete information FileInfo provides. e.g. the last commit and the corresponding message are not needed.

In the resulting API response, only the file SHAs are missing, and these are not computed by the (presumably slow) call to JGitUtil.getFileList: List[JGitUtil.FileInfo].

@takezoe takezoe added this to the 4.36.0 milestone Feb 14, 2021
@takezoe
Copy link
Member

takezoe commented Feb 14, 2021

Thanks for investigation. Sounds like we can optimize it. I think we can deal with it in 4.36.0.

@takezoe takezoe mentioned this issue Feb 21, 2021
Closed
6 tasks
@takezoe
Copy link
Member

takezoe commented Feb 21, 2021

@ziggystar I created #2686 to optimize the performance of Contents API, but I'm not sure how much effective it is. Could you try that branch if possible to compare the performance?

@ziggystar
Copy link
Contributor Author

@takezoe I can clone and compile the project. How can I run it?

I tried sbt run and sbt package for creating the war. But the JettyLauncher classes are not bundled with the project. I could not find any instructions about this. Best would be to run it from IDE, but I could not figure out how to do this.

@takezoe
Copy link
Member

takezoe commented Feb 23, 2021

You can create an executable war file by sbt executable:
https://github.com/gitbucket/gitbucket/blob/master/doc/build.md

@takezoe takezoe modified the milestones: 4.36.0, 4.37.0 Jul 17, 2021
@shinsenter
Copy link

Same problem.

This was referenced Sep 15, 2021
Open
Open
@takezoe takezoe modified the milestones: 4.37.0, 4.38.0 Dec 11, 2021
@takezoe takezoe removed this from the 4.38.0 milestone Sep 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
APIv3 GitHub compatible API improvement
Milestone
No milestone
3 participants
@ziggystar @takezoe @shinsenter