FAQ: How to track latest nightly builds

[mergen3107]

What is a nightly KOReader release?

KOReader development cycle includes two major milestones: stable monthly-ish release suitable for an average user, and intermediate nightly releases which represent cutting edge features intended mostly for developers and testers.

1. Stable monthly-ish release is named latest release in GitHub. They have the nomenclature of vYYYY.MM, where YYYY is the year, and MM is the month of the release. For example, current one is v2023.10.

(Occasionally there are critical bugs found right after the stable release, then the latest release receives an update like vYYYY.MM.F, where F is the iteration of fixes. E.g., v2023.06.1.)

All stable KOReader releases can be found here:
https://github.com/koreader/koreader/releases

2. In the same vein as Rome wasn't built in a day, KOReader stable releases aren't compiled in a single GitHub commit. Throughout the month after the latest release, many devs put their hard work and passion to improve features, fix bugs, optimize memory, and much more - all under careful inspection of the code and user needs.

Nightly build compilation mechanism

If there is a newly merged commit, or a Pull Request which contains multiple commits, KOReader development's backend server compiles the so-called nightly release (or just nightly), around 6-7 AM CEST the closest morning after the merge. They have the nomenclature of vYYYY.MM-ii-buildnumber_date, where ii is the number of merged commits. After each stable release, the ii counter resets.

Fantastic commits and where to find them

The order and contents of the merged commits that made it to nightlies can be found at:
v2023.10...master

After each new stable release the link should be changed accordingly if you would like to track the changes in terms of merged commits/nightlies.

Here, for the latest compiled nightly, the index ii will be the number of commits on this page, e.g., v2023.10-2 for two merged commits.

Where to download nightlies

All those nightlies can be found at:
http://build.koreader.rocks/download/nightly/

This is the FTP-based approach, where each directory represents a nightly release, and the directories contain all the compiled packages for platforms that KOReader supports.

That means, if you are waiting for a particular fix, you need to wait about a day until the nightly with ii ≥ number of commits you need is compiled.

Nightly releases are intended for a narrow group of testers and developers, so there might some nightlies that are less stable than other nightlies.

Example

GitHub compare shows that by the time of this screenshot there have been 7 commits merged with current master branch, relative to the v2023.10 release:

The list of nightly builds, however, shows that the last nightly build contains only 5 commits:

That means that the last 2 commits out of these 7 did not make it to a nightly build yet.
Good news is that you will only need to wait a couple of hours until 6-7 AM CET to be able to download the nightly and test these 2 commits.

Advanced Troubleshooting

Nightlies History

Please keep in mind that due to the server-side space limitations there are only a certain number of nightlies that can be concurrently stored on the koreader.rocks server. If you intend to test come PRs extensively, makes sure to backup necessary nightlies.

You can always revert to a closest (or any other) stable release, if you need to.

(System-specific downgrade limitations still apply. For example, for downgrading on Android, you need to delete the newer nightly before you can installer an earlier stable/nightly release).

Larger archive on GitLab (4 weeks)

There's a larger archive on https://gitlab.com/koreader/nightly-builds/-/pipelines

(Again, some system-specific limitations apply. For example, for the Android builds specifically there's the minor niggle that you'd have to sign them yourself. That means, currently installed KOReader must be deleted for either downgrades or upgrades w.r.t. the Larger archive nightly builds because the apk signature is now different.)

[mergen3107]

@NiLuJe @poire-z @Frenzie @pazos @hius07 @offset-torque (and everyone interested in this kind of documentation)

Could you please review this piece I came up with? Throughout some months I realized that I need to explain such things about nightlies from scratch to people on forums (4PDA, MobileRead, reddit) and here on GitHub.

So, I propose to edit this documentation piece and pin it to the of the issues page.

Any edits are welcome (including complete shutdown of such an idea if you deem it unnecessary/useless :D )

We might even consider putting this to the User Guide?

This is my small contribution for making KOReader friendlier to people. I just realized that KOReader is not great because it is just a final product, but because it is also a process! Specifically, a learning process, to which people tend to stick like moths at night stick around the burning lights.

So, this is just another way to support this learning experience.

Thank you!

[benoit-pierre]

If there is a newly merged Pull Request (PR), KOReader development's backend server compiles the so-called nightly release (or just the nightly), around 6-7 AM CEST the closest morning after the merge. They have the nomenclature of vYYYY.MM-II, where II is the number of merged PRs.

Number of additional commits since the last release (a PR can contain more than one commit).

[mergen3107]

Thank you! @benoit-pierre
I was hesitant that I missed something like a group PR thingy, but wasn't sure how to fix it exactly 😅

[Frenzie]

Please keep in mind that due to the server-side space limitations there are only a certain number of nightlies that can be concurrently stored on the server. If you intend to test come PRs extensively, makes sure to backup necessary nightlies.

While I don't know to what extent we should advertise it (not because it's secret, but just to prevent confusion), there's a much larger archive on https://gitlab.com/koreader/nightly-builds/-/pipelines

For the Android builds specifically there's the minor niggle that you'd have to sign them yourself.

Edit: though "much larger" may be a bit of an overstatement. It's limited to four weeks these days while it used to go back at least a year. But it's still a lot more than just one week! ;-)

[mergen3107]

Thanks @Frenzie !

Interesting, I learned something new again!

I was thinking all along, that even though this FAQ piece is advanced by itself, there probably might also be the Advanced troubleshooting as well.

I just didn't know where to draw the line, but your suggestion definitely goes to Advanced, I think.

[mergen3107]

I also got a question:
If a commit merging happens to occur right during the compilation time around 6-7 AM CEST, does that mean that first part of the packages will rollout without this new commit? Or does the compilation takes only N number of commits to work with until it's done?

[Frenzie]

does that mean that first part of the packages will rollout without this new commit?

That's theoretically possible, but all builds normally start within seconds of each other.

[mergen3107]

I separated the OP into a few more sections. Please have a look and let me know if I need to change one particular heading ;)

[Frenzie]

Concretely, we have a script running on the koreader.rocks server that triggers a pipeline at 6 in the morning.

https://github.com/koreader/koreader-misc/blob/c82da186ea90fcd8b76512de4ea683c2503eb72d/nightswatcher/nightswatcher.py#L64-L67

During the European day and especially a few hours later when it overlaps with the American day there's a decent chance that individual jobs might have to wait for others to finish first, but at 6 AM all 15 jobs can start all at once.

Basically the scenario you sketch might be doable right this moment since the American working day is still in full swing, but at 6 the window of opportunity is just so tiny.

But even if it had a decent chance of happening I'd probably hardly consider it an issue.

[Frenzie]

Incidentally, out of curiosity and to get #11069 out in the wild sooner I started a manual pipeline 15 minutes ago and that illustrated what it looks like when they don't start all at once.
https://gitlab.com/koreader/nightly-builds/-/pipelines/1059472074

[mergen3107]

Thanks!

I didn't realize all target platform packages start in parallel!

I was under the impression they are built one by one, don't know why.

[poire-z]

(Nothing much to add or correct - it's nice that this is written down and available.)

[mergen3107]

Added an example section to clearly show how to find commits/nightlies :D

[offset-torque]

We might even consider putting this to the User Guide?

User guide is essentially aimed at teaching average users how to use KOReader's features. Currently our most complicated procedure in the guide is "downloading ready made user patches to your device".

Quoting your description: "Nightly releases are intended for a narrow group of testers and developers".
So I think this information belongs to the Developer documentation instead of the user guide. Because once a user downloads a nightly version, there is nothing else in the user guide that would help them with the next steps. They already have to continue from the Developer documentation.

Of course if our devs think this would be useful in the user guide, I can add it.

[Frenzie]

No, I agree that's out of scope. It could potentially make sense to mention in the issue template (i.e., maybe it's already been fixed) but that should also refrain from being too overwhelming.

[mergen3107]

FYI, nightly server is down (again).

[Frenzie]

Well, there goes any release. ;-)

[mergen3107]

I just missed the first PR in the nightly that fixes docsettings/hashsettings, lol :D
I'll try to patch it up in VM.

[mergen3107]

I'll try to patch it up in VM.

So I tried to diff-patch the kindlepw2 stable 2023.10 package and it complained about spec/ folder missing, which is included in the dev source.

So, I eneded up... compiling a fresh kindlepw2 nightly myself using my existing dev env :D (plus I got to learn what kox and x-tools are ;) I just did ./kodev release kindlepw2.

However, in the end in spat out an archive with .targz in the end. I renamed it to .tar.gz and was able to extract it.
Is it just my Ubuntu acting up, or I just found a bug?

[Frenzie]

Neither, it has something to do with Chinese caching proxies but it's probably outdated.

…

On Tue, Nov 21, 2023, 23:08 mergen3107 ***@***.***> wrote: I'll try to patch it up in VM. So I tried to diff-patch the kindlepw2 stable 2023.10 package and it complained about spec/ folder missing, which is included in the dev source. So, I eneded up... compiling a fresh kindlepw2 nightly myself using my existing dev env :D (plus I got to learn what kox and x-tools are ;) I just did ./kodev release kindlepw2. However, in the end in spat out an archive with targz in the end. I renamed it to tar.gz and was able to extract it. Is it just my Ubuntu acting up, or I just found a bug? — Reply to this email directly, view it on GitHub <#11068 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABRQBIFSTEDIIC3AR35LSTYFUQ57AVCNFSM6AAAAAA63NP6S6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRRG43TSNBQGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>