Add-on database wiped? After update to 12.3

[patienttruth]

Describe the issue you are experiencing

Hi, I updated to 12.3 tonight and later realized that my traccar add-on was not functioning. After troubleshooting and hitting dead ends I tried to open the browser to the web UI. It asked to create a new user which is what it does with a brand new install.

I wrote wiped with a question mark because I haven't actually looked into the database to see what it looks like. That's not something I'm familiar with doing, but happy to muddle may way through if it's helpful.

I checked history, and sure enough the device tracker provided by the integration went down right when I updated.

Thankfully this is not a big deal as the tracking history isn't critical, but still it would be good to know what happened.

What operating system image do you use?

ova (for Virtual Machines)

What version of Home Assistant Operating System is installed?

12.3

Did you upgrade the Operating System.

Yes

Steps to reproduce the issue

1. Had traccar add-on & integration fully functional w/maria DB addon
2. Upgrade to 12.3

Anything in the Supervisor logs that might be useful for us?

Logs didn't have anything going back that far. (I'd tried rebooting before realizing the database was reset).

Anything in the Host logs that might be useful for us?

Nothing regarding traccar or mariadb.

System information

System Information

version	core-2024.5.2
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.12.2
os_name	Linux
os_version	6.6.29-haos
arch	x86_64
timezone	America/Los_Angeles
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
GitHub API Calls Remaining	4546
Installed Version	1.34.0
Stage	running
Available Repositories	1393
Downloaded Repositories	40

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 12.3
update_channel	stable
supervisor_version	supervisor-2024.05.1
agent_version	1.6.0
docker_version	25.0.5
disk_total	30.8 GB
disk_used	21.5 GB
healthy	true
supported	true
board	ova
supervisor_api	ok
version_api	ok
installed_addons	Bookstack (2.0.4), Let's Encrypt (5.0.15), File editor (5.8.0), Advanced SSH & Web Terminal (17.2.0), motionEye (0.20.0), Piper (1.5.0), Grocy (0.21.0), Terminal & SSH (9.13.0), Whisper (2.0.0), openWakeWord (1.10.0), Music Assistant BETA (2.0.0b148), MariaDB (2.7.1), Traccar (0.25.0), go2rtc (1.9.1), Studio Code Server (5.15.0), Frigate Proxy (1.5), ESPHome (2024.4.2)

Dashboards

dashboards	9
resources	18
views	33
mode	storage

Recorder

oldest_recorder_run	April 30, 2024 at 04:08
current_recorder_run	May 8, 2024 at 22:17
estimated_db_size	273.18 MiB
database_engine	sqlite
database_version	3.44.2

Sonoff

version	3.7.3 (e240aaf)
cloud_online	0 / 3
local_online	2 / 2

Additional information

No response

[sairon]

I don't think this is an OS issue but you might be wiser after checking (most probably) MariaDB logs. Using the OS host shell or after updating to the latest SSH add-on released today (9.14.0), you can use the -n flag for the ha addon logs ADDON_NAME and get more than the default 100 lines - check the logs around the time when the sensor changed the state.

[patienttruth]

I don't think this is an OS issue but you might be wiser after checking (most probably) MariaDB logs. Using the OS host shell or after updating to the latest SSH add-on released today (9.14.0), you can use the -n flag for the ha addon logs ADDON_NAME and get more than the default 100 lines - check the logs around the time when the sensor changed the state.

Thanks, that's a good lead.

I'm getting an error though with that flag.

Edit - still working on this, going to dive in on a PC later for full interface. Thanks for the lead.

[patienttruth]

Alright, my error before was because I was using Advanced SSH instead of Terminal & SSH as you suggested. I've walked myself in circles a few times trying to figure out the HA cli quirks, but this came down to my attention to detail.

I'm not seeing any logs from before the HAOS update. I tried setting -n to 700, but it's not going past around 450 when the HAOS update took place. This is the log from the first startup of Maria DB after the update. Does anything stand out to you?

log

s6-rc: info: service s6rc-oneshot-runner: starting                                                                                                                                                                                       
s6-rc: info: service s6rc-oneshot-runner successfully started                                                                                                                                                                            
s6-rc: info: service fix-attrs: starting                                                                                                                                                                                                 
s6-rc: info: service fix-attrs successfully started                                                                                                                                                                                      
s6-rc: info: service legacy-cont-init: starting                                                                                                                                                                                          
s6-rc: info: service legacy-cont-init successfully started                                                                                                                                                                               
s6-rc: info: service mariadb-pre: starting                                                                                                                                                                                               
[18:15:42] INFO: Using existing mariadb                                                                                                                                                                                                  
s6-rc: info: service mariadb-pre successfully started                                                                                                                                                                                    
s6-rc: info: service mariadb-core: starting                                                                                                                                                                                              
s6-rc: info: service mariadb-core successfully started                                                                                                                                                                                   
s6-rc: info: service mariadb-post: starting                                                                                                                                                                                              
[18:15:42] INFO: Starting MariaDB                                                                                                                                                                                                        
2024-05-08 18:15:43 0 [Note] Starting MariaDB 10.11.6-MariaDB source revision fecd78b83785d5ae96f2c6ff340375be803cd299 as process 67                                                                                                     
2024-05-08 18:15:43 0 [Note] InnoDB: Compressed tables use zlib 1.3.1                                                                                                                                                                    
2024-05-08 18:15:43 0 [Note] InnoDB: Number of transaction pools: 1                                                                                                                                                                      
2024-05-08 18:15:43 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions                                                                                                                                                                
2024-05-08 18:15:43 0 [Note] mysqld: O_TMPFILE is not supported on /var/tmp (disabling future attempts)                                                                                                                                  
2024-05-08 18:15:43 0 [Note] InnoDB: Using Linux native AIO                                                                                                                                                                              
2024-05-08 18:15:43 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB                                                                                                                            
2024-05-08 18:15:43 0 [Note] InnoDB: Completed initialization of buffer pool                                                                                                                                                             
2024-05-08 18:15:43 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)                                                                                                                                         
2024-05-08 18:15:43 0 [Note] InnoDB: End of log at LSN=46980                                                                                                                                                                             
2024-05-08 18:15:43 0 [Note] InnoDB: 128 rollback segments are active.                                                                                                                                                                   
2024-05-08 18:15:43 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...                                                                                                        
2024-05-08 18:15:43 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.                                                                                                                                                              
2024-05-08 18:15:43 0 [Note] InnoDB: log sequence number 46980; transaction id 14                                                                                                                                                        
2024-05-08 18:15:43 0 [Note] Plugin 'FEEDBACK' is disabled.                                                                                                                                                                              
2024-05-08 18:15:43 0 [Note] InnoDB: Loading buffer pool(s) from /data/databases/ib_buffer_pool                                                                                                                                          
2024-05-08 18:15:43 0 [Note] InnoDB: Buffer pool(s) load completed at 240508 18:15:43                                                                                                                                                    
2024-05-08 18:15:43 0 [Note] Server socket created on IP: '0.0.0.0'.                                                                                                                                                                     
2024-05-08 18:15:43 0 [Note] Server socket created on IP: '::'.                                                                                                                                                                          
2024-05-08 18:15:43 0 [Note] mysqld: ready for connections.                                                                                                                                                                              
Version: '10.11.6-MariaDB'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Alpine Linux                                                                                                                                                  
[18:15:44] INFO: MariaDB started                                                                                                                                                                                                         
[18:15:44] INFO: Check data integrity and fix corruptions                                                                                                                                                                                
mysql.column_stats                                 OK                                                                                                                                                                                    
mysql.columns_priv                                 OK                                                                                                                                                                                    
mysql.db                                           OK                                                                                                                                                                                    
mysql.event                                        OK                                                                                                                                                                                    
mysql.func                                         OK                                                                                                                                                                                    
mysql.global_priv                                  OK                                                                                                                                                                                    
mysql.gtid_slave_pos                               OK                                                                                                                                                                                    
mysql.help_category                                OK                                                                                                                                                                                    
mysql.help_keyword                                 OK                                                                                                                                                                                    
mysql.help_relation                                OK                                                                                                                                                                                    
mysql.help_topic                                   OK                                                                                                                                                                                    
mysql.index_stats                                  OK                                                                                                                                                                                    
mysql.innodb_index_stats                           OK                                                                                                                                                                                    
mysql.innodb_table_stats                           OK                                                                                                                                                                                    
mysql.plugin                                       OK                                                                                                                                                                                    
mysql.proc                                         OK
mysql.procs_priv                                   OK
mysql.proxies_priv                                 OK
mysql.roles_mapping                                OK
mysql.servers                                      OK
mysql.table_stats                                  OK
mysql.tables_priv                                  OK
mysql.time_zone                                    OK
mysql.time_zone_leap_second                        OK
mysql.time_zone_name                               OK
mysql.time_zone_transition                         OK
mysql.time_zone_transition_type                    OK
mysql.transaction_registry                         OK
sys.sys_config                                     OK
[18:15:44] INFO: Ensuring internal database upgrades are performed
[18:15:44] INFO: Ensure databases exists
[18:15:44] INFO: Create database homeassistant
[18:15:44] INFO: Ensure users exists and are updated
[18:15:44] INFO: Update user homeassistant
[18:15:44] INFO: Init/Update rights
[18:15:44] INFO: Granting all privileges to homeassistant on homeassistant
[18:15:44] INFO: Sending service information to Home Assistant

I did find this entry a few hours later when I realized traccar wasn't working:

log 2

s6-rc: info: service mariadb-post successfully started                                                                                                                                                                                   
s6-rc: info: service legacy-services: starting                                                                                                                                                                                           
s6-rc: info: service legacy-services successfully started                                                                                                                                                                                
s6-rc: info: service mariadb-lock-core: starting                                                                                                                                                                                         
[21:00:07] INFO: Start MariaDB client (to lock tables for backups)                                                                                                                                                                       
s6-rc: info: service mariadb-lock-core successfully started                                                                                                                                                                              
s6-rc: info: service mariadb-lock-post: starting                                                                                                                                                                                         
[21:00:07] INFO: MariaDB tables locked                                                                                                                                                                                                   
s6-rc: info: service mariadb-lock-post successfully started                                                                                                                                                                              
s6-rc: info: service mariadb-lock-post: stopping                                                                                                                                                                                         
[21:00:08] INFO: MariaDB tables unlocked                                                                                                                                                                                                 
s6-rc: info: service mariadb-lock-post successfully stopped                                                                                                                                                                              
s6-rc: info: service mariadb-lock-core: stopping                                                                                                                                                                                         
[21:00:08] INFO: MariaDB client (to lock tables for backups) stopped                                                                                                                                                                     
s6-rc: info: service mariadb-lock-core successfully stopped                                                                                                                                                                              
2024-05-08 21:58:00 92 [Warning] Aborted connection 92 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 95 [Warning] Aborted connection 95 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 87 [Warning] Aborted connection 87 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 89 [Warning] Aborted connection 89 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 94 [Warning] Aborted connection 94 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 91 [Warning] Aborted connection 91 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 93 [Warning] Aborted connection 93 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 88 [Warning] Aborted connection 88 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 90 [Warning] Aborted connection 90 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 21:58:00 19 [Warning] Aborted connection 19 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 22:16:06 108 [Warning] Aborted connection 108 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 105 [Warning] Aborted connection 105 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 103 [Warning] Aborted connection 103 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 101 [Warning] Aborted connection 101 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 98 [Warning] Aborted connection 98 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 22:16:06 106 [Warning] Aborted connection 106 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 104 [Warning] Aborted connection 104 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 99 [Warning] Aborted connection 99 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                                 
2024-05-08 22:16:06 102 [Warning] Aborted connection 102 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
2024-05-08 22:16:06 107 [Warning] Aborted connection 107 to db: 'traccar' user: 'service' host: '172.30.32.1' (Got an error reading communication packets)                                                                               
s6-rc: info: service legacy-services: stopping                                                                                                                                                                                           
s6-rc: info: service legacy-services successfully stopped                                                                                                                                                                                
s6-rc: info: service mariadb-post: stopping
[22:16:18] INFO: Deleting service information from Home Assistant
[22:16:18] ERROR: Got unexpected response from the API: System is not ready with state: shutdown
s6-rc: info: service mariadb-post successfully stopped
s6-rc: info: service mariadb-core: stopping
2024-05-08 22:16:18 0 [Note] mysqld (initiated by: unknown): Normal shutdown
2024-05-08 22:16:18 0 [Note] InnoDB: FTS optimize thread exiting.
2024-05-08 22:16:18 0 [Note] InnoDB: Starting shutdown...
2024-05-08 22:16:18 0 [Note] InnoDB: Dumping buffer pool(s) to /data/databases/ib_buffer_pool
2024-05-08 22:16:18 0 [Note] InnoDB: Buffer pool(s) dump completed at 240508 22:16:18
2024-05-08 22:16:19 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
2024-05-08 22:16:19 0 [Note] InnoDB: Shutdown completed; log sequence number 3160993; transaction id 3656
2024-05-08 22:16:19 0 [Note] mysqld: Shutdown complete

*Another HA cli quirk that I've spent a long time struggling with is copying out of the terminal add-ons. and finally I found this...
The secret handshake with HA Terminal to copy text

[patienttruth]

I'm happy to move this issue to a more appropriate repo if I can figure out where. I've got traccar back up and running, but no leads yet on why the database got lost/unassociated/?

[RodWillaims]

I had a similar issue after upgrading to 12.3 under a Proxmox VM. All my HACS frontend items stopped working. It appeared to be a permissions issue, so I went to each of the HACS Frontend elements and pressed the "redownload" option and this fixed the issue without requiring any configuration. I then repeated for other elements.

[patienttruth]

I had a similar issue after upgrading to 12.3 under a Proxmox VM. All my HACS frontend items stopped working. It appeared to be a permissions issue, so I went to each of the HACS Frontend elements and pressed the "redownload" option and this fixed the issue without requiring any configuration. I then repeated for other elements.

Interesting. I wonder what happened on the backend for both of these symptoms.

I use a few front end elements via HACS and they remained functional. 🤷🏼‍♂️