[Bug] Erroneous allocation calculation with btrfs #134
Comments
As the time I may allot for addressing this issue is quite scarce and this is rather a corner case (a freshly factory reset Jolla 1 passes the original test), hence of low priority from my perspective, any help WRT points 2 to 4 above is much appreciated. |
On my Jolla1@SFOS2.2.1: Consequently both, Any ideas how to resolve this? P.S.: Just a quick counter-check that my Jolla1@SFOS2.2.1 is still working well. |
|
Oh, while almost giving up, I accidentally found a potential way:
Please provide a little time for points 1 and 2, if possible, now that I have researched a apparently feasible route. |
Notes / ideas
As a label can be omitted, UUIDs are the only safe identifier. |
Label: 'root' uuid: 7d24fbb1-73c3-42e2-bae1-c02321491dd5
Total devices 1 FS bytes used 3.25GiB
devid 1 size 7.75GiB used 4.98GiB path /dev/mmcblk2p2
Label: 'Ulysse31' uuid: f8c2502a-b267-438a-9591-9e84e28afb9a
Total devices 1 FS bytes used 76.23GiB
devid 1 size 111.00GiB used 84.03GiB path /dev/mmcblk1p1
Label: 'home' uuid: 185d1488-a386-4c29-8276-ad290ffdca7e
Total devices 1 FS bytes used 2.12GiB
devid 2 size 99.98GiB used 5.03GiB path /dev/mapper/home_encrypted
Btrfs v3.16Note: the Support for btrfs on SD card, on the other hand, is pretty standard (except for one of the earlier versions of SailfishX on Xperia X, when Jolla forgot to enable the driver in the kernel).
I ran this snippet in both GNU bash and busybox bash: btrfs_path="$(mount -lt btrfs | fgrep ' / ' | cut -d ' ' -f1)"
btrfs_allocation="$(btrfs filesystem show)"
btrfs_total=0
btrfs_used=0
while read ld i ls t lu u lp p; do
# look for lines like:
# devid 1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
if [[ "$ld" != 'devid' || "$ls" != 'size' || "$lp" != 'path' ]]; then
continue
fi
# is it the device mounted as / ?
if [[ "$p" != "$btrfs_path" ]]; then
continue
fi
# parse numbers with regex and tally them
let btrfs_total+="$( echo "${t%GiB}" | tr -d '.' )"
let btrfs_used+="$( echo "${u%GiB}" | tr -d '.' )"
done <<EOF
$btrfs_allocation
EOF
btrfs_unallocated="$((btrfs_total-btrfs_used))"
echo $btrfs_unallocatedThis can still break in some corner case. One way to recover from "enosp" (*) is to temporarily add another block device to the btrfs partition, do the space freeing, rebalancing, etc. and the remove the additional device. (*): when the CoW has painted itself in a corcer and cannot delete to free space, because it first need to allocate a metadata chunk to CoW metadata, but it ran out of allocatable space in that case, one would need to exact the UUID thus: eval $(blkid $(mount -lt btrfs | fgrep ' / ' | cut -d ' ' -f1) | grep -oE '[^T]UUID=[^ ]+')
echo $UUIDor btrfs_uuid="$(lsblk -nlo UUID,MOUNTPOINT | grep '/$' | cut -d ' ' -f1)"and have a mini-state system that turns parsing on only after a line with By abusing the already existing read line: btrfs_path="$(mount -lt btrfs | fgrep ' / ' | cut -d ' ' -f1)"
eval $(blkid "$btrfs_path" | grep -oE '[^T]UUID=[^ ]+')
btrfs_uuid="${UUID}"
btrfs_allocation="$(btrfs filesystem show -d)"
btrfs_total=0
btrfs_used=0
parsing=0
while read ld i ls t lu u lp p; do
# HACK look for lines like:
# Label: none uuid: baa40ad9-01ca-4a8b-a0d0-7abeb6231b27
if [[ "$ld" == 'Label:' && "$ls" == 'uuid:' ]]; then
if [[ "$t" == "${btrfs_uuid}" ]]; then
parsing=1
else
parsing=0
fi
continue
fi
if [[ "$parsing" != "1" ]]; then
continue
fi
# look for lines like:
# devid 1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
if [[ "$ld" != 'devid' || "$ls" != 'size' || "$lp" != 'path' ]]; then
continue
fi
# parse numbers with regex and tally them
let btrfs_total+="$( echo "${t%GiB}" | tr -d '.' )"
let btrfs_used+="$( echo "${u%GiB}" | tr -d '.' )"
done <<EOF
$btrfs_allocation
EOF
btrfs_unallocated="$((btrfs_total-btrfs_used))"
echo $btrfs_unallocatedBut at this point, one would consider gawk instead (is it pre-installed on Jolla 1?)
Fundamentally, what are you trying to achieve?
Freespace would indeed be: free_space = <free_dataspace_reported_by_ But this doesn't guarantee that you can actually write "free_space" bytes, because updating the file system could require allocating a metadata chunk which would consume x 2 (because by default, Jolla has accidentally set the metadata to So that would be "floor( ( <free_chunkspace_space_reported_by_ |
|
Thank you for the swift and informative reply! I concur with most of your statements, but not the final one:
As indicated before, then my Jolla 1 would be at -1GiB free chunk-space (actually it is just 0), which means what? A denoted in the referenced, old wiki, this is a usual situation, because an arbitrary amount of these chunks may be unused (though allocated). Only a
As I am aware of all this (guess why my Jolla 1 uses
Well, chunk size is a 2^n value which is at least 256 MiB and at most 1 GiB or 10% of the volume size, whichever is less. IIRC it was 512 MiB on the Jolla 1. Nevertheless, it will often be 1 GiB, hence this has to be an expected value. But ultimately this is of one the primary reasons, why both, free chunk space and free space in allocated chunks have to be taken into account. P.S.: I noticed this concise, practical guidance by Nvidia. It made me aware, that the answer to your «"free space" or "free chunk space"» question is: "both"
|
Which means that it could be not possible to allocate additional chunks if needed. At that point:
|
SailfishOS VERSION (Settings → About product → Build): 4.5.0.21
HARDWARE (Settings → About product → Manufacturer & Product name): Pine64 PinePhone Pro
sfos-upgrade VERSION (
rpm -qi sfos-upgrade): 3.11.1BUG DESCRIPTION
When relying on
btrfs(as opposed tobtrfs-balancer), the way allocation is computed is mistaken.this code segment gets as an inputs the Data chunks allocated, and how much file data is written into them. The ratio computed is the occupancy of the chunks themselves, this doesn't give any information about the unallocated space on the device.
Instead
btrsf filesystem showshould be used (orbtrfs filesystem usageon more recent versions).STEPS TO REPRODUCE
balance start -v -dusage=80 -musage=80 /)sfos-upgrade 4.5.0.24)ADDITIONAL INFORMATION
Here are my numbers (after balancing):
Currently, sfos-upgrade computes using this line:
and this gets the information that .8 GiB are unwritten on the allocated chunks.
(This is expected: after balancing the chunks will probably be very compact).
What it should do is use all per fdevice line(s):
Total number of space on device available is 7.75, 4.20 has been allocated to chunks, 3.55 is unallocated and can still further be allocated to Data or Metadata chunks.
A possible piece of bash code that does this:
I've tested it in GNU/bash (I don't know if it works in busybox bash. Does it support regex? Otherwise the tr+sed+grep+etc. route would be needed) .
The text was updated successfully, but these errors were encountered: