Map generation process for tests

[Arthur-GYT]

Hi there ! I follow this guide to regenerate the map (for testing of PR's like #8052) but when I get to the basic usage stage, I don't understand what I need to do to test the regenerated map with an android emulator. Is there something I've missed?

[Arthur-GYT]

Okay, I found how, but when I try the command python3.11 -m maps_generator --countries="France_*" --skip="Coastline" I have an error (see screenshot and logs below). @RedAuburn Have you an idea why ?

[2024-05-05 15:26:24,422] INFO stages Stage Features: start ...
[2024-05-05 15:26:24,423] INFO stages Test test_booking_data was skipped.
[2024-05-05 15:26:24,445] INFO gen_tool Run generator tool [generator_tool version 2024.05.05-2-610535fe-Linux]: /home/a-gayot/Github/Organic-Maps/../omim-build-release/generator_tool --threads_count=16 --data_path=/home/a-gayot/Github/Organic-Maps/../maps_build/2024_05_05__15_19_34/intermediate_data --intermediate_data_path=/home/a-gayot/Github/Organic-Maps/../maps_build/2024_05_05__15_19_34/intermediate_data --cache_path=/home/a-gayot/Github/Organic-Maps/../maps_build/2024_05_05__15_19_34/intermediate_data --osm_file_type=o5m --osm_file_name=/home/a-gayot/Github/Organic-Maps/../maps_build/2024_05_05__15_19_34/planet.o5m --node_storage=map --user_resource_path=/home/a-gayot/Github/Organic-Maps/data --dump_cities_boundaries=true --cities_boundaries_data=/home/a-gayot/Github/Organic-Maps/../maps_build/2024_05_05__15_19_34/intermediate_data/cities_boundaries.bin --generate_features=true --isolines_path= --generate_packed_borders=true 
I(1) 0.00000 generator/intermediate_data.cpp:188 MapFilePointStorageReader(): Nodes reading is started
[2024-05-05 15:28:01,172] INFO stages Stage Features: finished in 0:01:36.748013

[Developer]
# Path to the `organicmaps` source code repository:
OMIM_PATH: ~/Github/Organic-Maps
# A path with the generator_tool binary:
BUILD_PATH: ${Developer:OMIM_PATH}/../omim-build-release


[Main]
# A special small planet file will be downloaded if DEBUG is set to 1.
DEBUG: 0
# A main working directory. There is a subdirectory created for each generator run
# which contains the planet and other downloads, temporary build files, logs and completed MWMs.
MAIN_OUT_PATH: ${Developer:OMIM_PATH}/../maps_build
# Path for storing caches for nodes, ways, relations.
# If it's not set then caches are stored inside the directory of the current build.
# CACHE_PATH: ${Main:MAIN_OUT_PATH}/cache


[Generator tool]
# Path to the data/ folder in the repository:
USER_RESOURCE_PATH: ${Developer:OMIM_PATH}/data
# Features stage only parallelism level. Set to 0 for auto detection.
THREADS_COUNT_FEATURES_STAGE: 0


[Osm tools]
# Path to osmctools binaries:
OSM_TOOLS_PATH: ${Developer:OMIM_PATH}/../osmctools
# If the binaries are not found neither in the configured path nor system-wide,
# then the tools are built from the sources:
OSM_TOOLS_SRC_PATH: ${Developer:OMIM_PATH}/tools/osmctools


[Logging]
# maps_generator's general (python output only) log file path and name.
# More detailed logs that include output of the `generator_tool` binary
# are located in the `logs/` subdir of a particular build directory,
# e.g. `maps_build/2023_06_04__20_05_07/logs/`.
LOG_FILE_PATH: ${Main:MAIN_OUT_PATH}/generation.log


[External]
# Planet file location. It should be a dump of OSM data in osm.pbf format.
# By default its an entire planet from "planet.openstreetmap.org".
# Or set it to a particular country/region extract from e.g. [Geofabrik](http://download.geofabrik.de/index.html).
# Note that an entire planet generation takes 40+ hours on a 256GB RAM server (and 1TB+ disk space).
# Stick to smaller extracts unless you have a machine this large.
# Here and further, its possible to specify either an URL (to be downloaded automatically)
# or a local file path like file:///path/to/file.
# A sample URL to download a latest OSM dump for North Macedonia:
PLANET_URL: https://download.geofabrik.de/europe/france-latest.osm.pbf
# Location of the md5 checksum of the planet file:
PLANET_MD5_URL: ${External:PLANET_URL}.md5
# A base url to the latest_coasts.geom and latest_coasts.rawgeom files.
# For example, if PLANET_COASTS_URL = https://somesite.com/download/
# then the https://somesite.com/download/latest_coasts.geom url will be used to download latest_coasts.geom and
# the https://somesite.com/download/latest_coasts.rawgeom url will be used to download latest_coasts.rawgeom.
# Comment to skip getting the coastlines files.
# PLANET_COASTS_URL:
# Should be 'true' for an entire planet build to make a special routing section in World.mwm
# for alerting about absent regions without which the route can't be built.
NEED_BUILD_WORLD_ROADS: false
# Subway file location, see docs/SUBWAY_GENERATION.md if you want to generate your own file.
# Comment to disable subway layer generation.
#SUBWAY_URL: https://cdn.organicmaps.app/subway.json

# Location of the EXPERIMENTAL GTFS-extracted public transport transit files:
# TRANSIT_URL:

# Urls for production maps generation.
# UGC_URL:
# HOTELS_URL:
# PROMO_CATALOG_CITIES:
# POPULARITY_URL:
# FOOD_URL:
# FOOD_TRANSLATIONS_URL:
# SRTM_PATH:
# ISOLINES_PATH:
# UK_POSTCODES_URL:
# US_POSTCODES_URL:


[Stages]
# Set to 1 to update the entire OSM planet file (as taken from "planet.openstreetmap.org")
# via an osmupdate tool before the generation. Not for use with partial planet extracts.
NEED_PLANET_UPDATE: 0
# If you want to calculate diffs you need to specify where the old maps are.
DATA_ARCHIVE_DIR: ${Generator tool:USER_RESOURCE_PATH}
# How many versions in the archive to use for diff calculation:
DIFF_VERSION_DEPTH: 2


[Common]
# Default parallelism level for the most of jobs. Set to 0 for auto detection.
THREADS_COUNT: 0


[Stats]
# Path to rules for calculating statistics by type:
STATS_TYPES_CONFIG: ${Developer:OMIM_PATH}/tools/python/maps_generator/var/etc/stats_types_config.txt

[pastk]

(index regeneration is actually not needed as it had been done as a part of mwm generation, but it doesn't hurt either)

put the resulting mwm file (countries.txt is not needed) into the existing maps folder e.g. 240326 and restart OM in your emulator

[pastk]

ah and you might have to disable mwm checksum checking somewhere in storage.cpp (look for "Inconsistent MWM and version for") in your debug OM build for the emulator (or use a beta build, it will ignore this check)

[Arthur-GYT]

I don't understand which maps folder you're talking about ^^.

[pastk]

I don't understand which maps folder you're talking about ^^.

Where map files reside on your emulator.
If you use Android then its like /data/data/app.organicmaps/files/...

If you want to test via the desktop app then put the mwm in e.g. data/240326

[Arthur-GYT]

Thanks a lot ^^.

[pastk]

Where map files reside on your emulator. If you use Android then its like /data/data/app.organicmaps/files/...

Sorry this one is for bookmarks, map files are in Android/data/app.organicmaps/files/... :)

But I see you've found them already, great!