1. **HAL24K Knowledge Condensed**
- Introduction
- HAL24K Labs is pivotal in the development of the Dimension Platform including Dimension MLOps DataLab and more. It utilizes AWS and Azure for crafting solutions that integrate AI and ML for rapid application development.
- Based in Amsterdam, The Netherlands
- Website: https://hal24k.com/
- Dimension Platform
- At the heart of HAL24K Labs the Dimension Platform is a Kubernetes-based infrastructure supporting a suite of applications leveraging AWS and Azure for deployment.
- Production: https://dimension.hal24k.com/
- Dev: https://dev.dimension.hal24k.com/
- Acc: https://acc.dimension.hal24k.com/
- Dimension ML Ops
- Dimension MLOps is a suite of micro-services including a Model Manager and Label Manager designed to manage and run ML models. It integrates Argo Workflows and MLFlow supported by CVWB library.
- Dimension Ops
- DimensionOps a CLI tool and micro-service streamlines project creation deployment and workflow management in the Dimension Platform facilitating data scientists' interaction with platform components.
- DataLab (Jupyterhub)
- DataLab provides a robust environment for data scientists through Jupyterhub hosted on Kubernetes integrating AWS services for data storage access control and monitoring.
- Maince
- Maince, a HAL24K Labs spinoff, focuses on transforming data into actionable insights for infrastructure management leveraging AI solutions for asset management and maintenance optimization.
- Based in Amsterdam, The Netherlands
- Website: https://www.maince.ai/
- Asset Viewer
- AssetViewer developed by Maince offers comprehensive asset inspection and annotation tools. It uses AWS CloudFront, S3, Cognito, and Amplify for frontend and authentication with backend support from AKS and EKS.
- Prod: https://apps.maince.ai/asset-viewer
- Dev: https://apps.dev.maince.ai/asset-viewer
- Acc: https://apps.acc.maince.ai/asset-viewer
- Maintenance Planner Optimisation (MPO)
- MPO, similar to AssetViewer, is a web application for road maintenance planning using CloudFront, S3, Cognito, and Amplify for frontend and authentication with backend on AKS or EKS.
- Condition Monitor
- Condition Monitor processes road condition data for maintenance planning integrating with MPO and utilizing data management and processing tools for robust road network analysis.
- HAL24K Agri
- HAL24K Agri focuses on soil health and sustainable agriculture offering products like the Life and Seed applications which use AWS ECS Fargate, CloudFront, and S3 for hosting and integrating with Dimension MLOps for advanced analytics.
- Based in Giesbeek, The Netherlands
- Website: https://www.hal24k-agri.com/
- Seed Demo Tenant (Dev): https://demo.dev.hal24kagri.com/seed
- Life Demo Tenant (Dev): https://demo.dev.hal24kagri.com/life
- Seed and Life Applications
- Seed: Processes drone-captured images for crop health analysis providing actionable advice for farmers.
- Life: Analyzes microscopic soil organisms leveraging HAL24K Agri's imaging technology and Dimension MLOps for soil health monitoring.
- Collaborate Water
- Collaborate Water is a spin-off company of HAL24K Labs. It was formerly called HAL24K Water and specializes in public water management solutions using AI and machine learning. They also sell a white-labeled version of the Dimension Platform and call it Collaborate.
- Based in the United Kingdom
- Website: http://collaboratewater.com/
- Collaborate Platform (aka Dimension Platform): https://collaboratewater.ai/
2. **HAL24K Labs Agile & Scrum Process**
- Sprint Naming
- Sprints are named YYYY/NN. So the beginning of the year we start with Sprint 2024/01 and each sprint is named 2024/02 and so on.
- Sprint Ceremonies
- We have a standup every morning at 10:00 CET.
- We have a Sprint Planning meeting every Thursday in the second week of the sprint (two days before the end of the sprint).
- We have a Sprint Prioritization meeting every Thursday starting at 13:00 of the first week of the sprint to organize the tickets for the following sprint. The meetings are 30 minutes meetings for each spin-off: Maince, HAL24K Agri, Collaborate Water, HAL24K Labs.
- We have a Sprint Demo and Retrospective on the last Friday of the sprint after the stand-up at 10:30.
- We have every two weeks a Backlog Grooming meeting to go through tickets which are older or ones which have not been properly defined.
- Sprint Duration
- Our sprints run in 2-week cycles starting at the end of business (EOB) on Monday at 18:00 and end two weeks on the following Monday at 18:00 (EOB). Because of this, there is an overlap of dates where in Jira the start dates and end dates are on the same day.
- For example:
- Sprint 2023/23 - 6 November 2023 - 20 November 2023
- Sprint 2023/24 - 20 November 2023 - 4 December 2023
- Sprint 2023/25 - 4 December 2023 - 18 December 2023
- We usually allow the sprint at the end of the year to flow over to the next year and then the next follow sprint will start with the new year.
- For Example:
- Sprint 2023/26 - 18 December 2023 - 1 January 2024
- Sprint 2024/01 - 1 January 2024 - 15 January 2023
- Sprint Releases
- Monday at 14:00 CET: There is the Sprint release for the Dimension Platform
- HAL24K Labs Dimension Platform version is at v3.XX.0
- The release includes:
- the “Dimension Platform” repository of micro-services
- the Dimension Flux repository for new application configurations and deployments within Kubernetes.
- the Terraform repository for any new infrastructure configurations
- There can be additional patch versions when regressions are found and need to be fixed and deployed.
- Tuesday at 14:00 CET: AssetViewer and Maintenance Planner (aka AOM) releases
- Maintenance Planner (aka AOM) version is at v1.XX.0
- AssetViewer (aka Inspections Module) version is at v1.XX.0
- HAL24K Agri Seed and Life Products version is at v1.XX.0
- There can be additional patch versions when regressions are found and need to be fixed and deployed.
- Tuesday at 19:00 CET: Collaborate Water Dimension Platform releases
- Collaborate Water Dimension is the same version as the HAL24K Labs Dimension Platform version since the Collaborate Water Dimension is a “white-labeled” version of Dimension.
- Sprint Releases are Triggered by the Jira Releases. These then trigger a webhook which then a version tag (Example: v3.85.0) to the main branch of the various backend Dimension-Platform repositories and then the tagged Docker containers get pushed to Harbor Docker Registry where Flux 2 CD picks up the new tags and deployed them to the Kubernetes environments. For the frontend the tagged main branches are pushed to the CloudFront +S3 distributions.
- Dimension Platform Versions are currently following in lockstep with the sprints:
- Sprint 2023/23 - v3.85.0
- Sprint 2023/24 - v3.86.0
- Sprint 2023/25 - v3.87.0
- Sprint 2023/26 - v3.88.0
- Sprint 2024/01 - v3.89.0
- As mentioned before other products such as Maince’s AssetViewer and Maintenance Planner; as well as HAL24K Agri’s Life and Seed products follow a different release cycle and versioning.
- Sprint Administration
- Sprint & Retrospective Reports
- There is a Sprint Report written for each sprint and also a retrospective report in Confluence. The team uses an online site called Easy Retro (https://easyretro.io/publicboard/rEbNH5suzdZasoBRAeQRqSkFOzF3/06d9fa89-b643-466d-9d38-3d07dd614313) to fill in the What Went Well, What To Improve, and Action Items for each sprint and it is discussed openly.
- Release Notes
- Release Notes are generated by the Jira Release process and saved in Confluence.
- Release notes are written for each Sprint based on the Jira Ticket descriptions. These are saved in Confluence in our internal release notes for each product Dimension Platform, AssetViewer, and Maintenance Planner. But we also have a public Confluence support space where client-facing release notes are stored. These release notes are a shorter more concise set of new features and improvements written for a different audience.
- Release Management Ticket
- Every Sprint we create a Release Management (RM) ticket in the Jira Project https://hal24k.atlassian.net/jira/software/c/projects/RM/issues
- We then add this RM-ticket number to the Jira Release page, the Engineering Google calendar sprint entry, the Sprint Report in Confluence, and the Retrospective report in Confluence.
- Below is an example of the contents of the RM ticket:
- – Start Ticket Template –
- Title: Release v3.85.0 | Sprint 2023/23
- Description:
- This ticket is created for releasing the work of the linked sprint. It covers the related documentation, testing, deployment, and further steps necessary for the release.
- For more information on managing these tickets see https://hal24k.atlassian.net/wiki/spaces/EN/pages/3115909123/Creating+New+Jira+Releases+and+Managing+Release+Tickets
- Related Documents
- https://hal24k.atlassian.net/wiki/spaces/EN/pages/3116335108/Dimension+Platform+Releases
- https://hal24k.atlassian.net/wiki/spaces/CD/pages/2080407553/Integration+Testing
- – End Ticket Template –
3. **External Release Notes Example**
- Below is an example of release notes for external clients to be used as a general guide for formatting, structure, and writing tone.
- – Begin External Release Notes Template –
- New Features
- Weather Manager Service: The platform now includes the Weather Manager, a comprehensive web service for accessing historical, current, and future weather data through Visual Crossing's robust weather API.
- Enhancements
- Dimension Ops Extensions: Added Github Actions adapter and Github project adapter expanding our DevOps capabilities within the platform's CI/CD workflow.
- Security Update: The CSV upload feature in Dimension Dashboard (Superset) has been disabled.
- Fixes
- Dimension Components Library: In the AnnotationViewer component, compatibility issues with AssetViewer were resolved, clearing the path for its upcoming release.
- – End External Release Notes Template –
4. **Internal Release Notes Example**
- Below is an example of the internal Release Notes and the writing style and tone.
- – Begin Release Notes Template –
- New Features
- Weather Manager Service: The platform now includes the Weather Manager, a comprehensive web service for accessing historical, current, and future weather data through Visual Crossing's robust weather API (CDX-5272).
- Enhancements
- Dimension Dashboard Updates: The CSV upload feature in Dimension Dashboard (Superset) has been disabled to streamline data management processes (CDX-5257).
- Release Automation: Implementation of a new trigger that automatically generates a GitHub release corresponding with Jira version releases, enhancing our CI/CD pipeline (CDX-5255).
- Dimension Ops Extensions: Added Github Actions adapter and Github project adapter expanding our DevOps capabilities within the platform's CI/CD workflow (CDX-5028, CDX-5027).
- System Upgrades
- Kubernetes and Istio: The platform's infrastructure has been updated to Kubernetes 1.27 and Istio 1.19 ensuring the latest features and security standards are met (CDX-5230).
- Fixes
- AnnotationViewer Integration: Resolved compatibility issues with Asset, clearing the path for its upcoming release (CDX-5280).
- GitHub Actions: Updated MongoDB to version 7 within our GitHub Actions to leverage the latest database features and optimizations (CDX-5237).
- Development
- Weather Manager Progression: Continued development and testing of the weather-manager micro-service to ensure reliability and accuracy of weather data retrieval (CDX-5275, CDX-5273).
- – End Release Notes Template –
5. **Retrospective Template**
- Example 2023/22 Retrospective
- Below is an example Retrospective Report which can be used as a general guideline for formatting and writing style
- – Begin Retrospective Template –
- Overview
- Sprint Period: 23 October 2023 - 6 November 2023
- Sprint Goals: Enhance modularity and robustness across modules, improve UI/UX, and streamline automation processes.
- Release Management Ticket: RM-231: Release v3.84.0 | Sprint 2023/22 DONE
- Sprint Demo
- In this sprint, the team saw remarkable demonstrations that showcased the progress made in various projects. Samuel presented the GitHub adapter for Dimension Ops, a significant development that aligns with the existing features of the Bitbucket adapter, such as viewing GitHub Actions' build steps and logs.
- Moreover, Bobby unveiled Compass, a strategic initiative aimed at cataloging and inventorying all services across our diverse systems and applications. He also took the team through the Atlas project management application, shedding light on its functionality and potential to enhance our project management processes.
- Went Well
- The sprint was highlighted by successful demos that enhanced the team's visibility into various projects. Samuel's demo of the GitHub adapter
