abmplot fails when 1) number of agents changes over time and 2) custom agent_size function is used

[markusgumbel]

Describe the bug

I have a simulation where I remove agents by chance (by remove_agent!). I also define a custom function (p_size) for an agent's size provided to abmplot. When I run the simulation (see MWE below), an exception is thrown:

Error in callback:
DimensionMismatch: arrays could not be broadcast to a common size; got a dimension with lengths 100 and 52

See also full stack trace at the end.

What am I doing wrong?

PS: Thank you very much for the Agents.jl package! I am really happy to use it.

Minimal Working Example

Here's the example. Please note that the example works when the custom agent_size function p_size is not used -- see comments at the end of the example.

using Agents, GLMakie

@agent struct Particle(ContinuousAgent{2,Float64}) end

function build_model()
    space = ContinuousSpace((1, 1))
    model = StandardABM(Particle, space; agent_step! = particle_step!)
    for i = 1:100
        pos = Tuple(rand(2))
        vel = Tuple(rand(2))
        add_agent!(Particle(i, pos, vel), model)
    end
    return model
end

function particle_step!(p, model)
    randomwalk!(p, model, 1.0)
    if rand() < 0.5
        remove_agent!(p, model) # Number of agents changes here.
    end
end

p_size(p::Particle) = 10 # Function for an agent's size.

model = build_model()

GLMakie.activate!(visible = true)
figure, ax, abmobs = abmplot(model; add_controls = true, agent_size = p_size) # does not work
# figure, ax, abmobs = abmplot(model; add_controls = true, agent_size = 10) # works
resize!(figure, 1200, 800)
wait(display(figure))

Agents.jl version

Agents v6.0.12 with Julia 1.10.2 and Linux Ubuntu 22.04.

Full stack trace

Error in callback:
DimensionMismatch: arrays could not be broadcast to a common size; got a dimension with lengths 100 and 52
Stacktrace:
  [1] _bcs1
    @ ./broadcast.jl:555 [inlined]
  [2] _bcs
    @ ./broadcast.jl:549 [inlined]
  [3] broadcast_shape
    @ ./broadcast.jl:543 [inlined]
  [4] combine_axes
    @ ./broadcast.jl:524 [inlined]
  [5] combine_axes (repeats 2 times)
    @ ./broadcast.jl:523 [inlined]
  [6] instantiate
    @ ./broadcast.jl:306 [inlined]
  [7] materialize
    @ ./broadcast.jl:903 [inlined]
  [8] offset_bezierpath(atlas::Makie.TextureAtlas, bp::BezierPath, scale::Vector{Vec{2, Float32}}, offset::Vector{Vec{2, Float32}})
    @ Makie ~/.julia/packages/Makie/ND0gA/src/utilities/texture_atlas.jl:537
  [9] offset_marker(atlas::Makie.TextureAtlas, marker::BezierPath, font::FreeTypeAbstraction.FTFont, markersize::Vector{Vec{2, Float32}}, markeroffset::Vector{Vec{2, Float32}})
    @ Makie ~/.julia/packages/Makie/ND0gA/src/utilities/texture_atlas.jl:541
 [10] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::@Kwargs{})
    @ Base ./essentials.jl:892
 [11] invokelatest(::Any, ::Any, ::Vararg{Any})
    @ Base ./essentials.jl:889
 [12] (::Observables.MapCallback)(value::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:436
 [13] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [14] invokelatest
    @ ./essentials.jl:889 [inlined]
 [15] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [16] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:123
--- the last 5 lines are repeated 1 more time ---
 [22] (::Observables.SetindexCallback)(x::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:148
--- the last 10 lines are repeated 1 more time ---
 [33] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [34] invokelatest
    @ ./essentials.jl:889 [inlined]
 [35] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [36] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:123
 [37] (::Observables.MapCallback)(value::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:436
 [38] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [39] invokelatest
    @ ./essentials.jl:889 [inlined]
 [40] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [41] step!(abmobs::ABMObservable{Observable{StandardABM{ContinuousSpace{2, true, Float64, typeof(Agents.no_vel_update)}, Particle, Dict{Int64, Particle}, Tuple{DataType}, typeof(particle_step!), typeof(dummystep), typeof(Agents.Schedulers.fastest), Nothing, Random.TaskLocalRNG}}, Nothing, Nothing, Nothing, Nothing, Bool, Observable{Int64}, Observable{Tuple{Base.RefValue{Int64}, Vector{Int64}}}}, t::Int64)
    @ AgentsVisualizations ~/.julia/packages/Agents/0NJQH/ext/AgentsVisualizations/src/model_observable.jl:38
 [42] (::AgentsVisualizations.var"#48#55"{ABMObservable{Observable{StandardABM{ContinuousSpace{2, true, Float64, typeof(Agents.no_vel_update)}, Particle, Dict{Int64, Particle}, Tuple{DataType}, typeof(particle_step!), typeof(dummystep), typeof(Agents.Schedulers.fastest), Nothing, Random.TaskLocalRNG}}, Nothing, Nothing, Nothing, Nothing, Bool, Observable{Int64}, Observable{Tuple{Base.RefValue{Int64}, Vector{Int64}}}}, Observable{Any}})(c::Int64)
    @ AgentsVisualizations ~/.julia/packages/Agents/0NJQH/ext/AgentsVisualizations/src/interaction.jl:47
 [43] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [44] invokelatest
    @ ./essentials.jl:889 [inlined]
 [45] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [46] setindex!(observable::Observable, val::Any)
    @ Observables ~/.julia/packages/Observables/YdEbO/src/Observables.jl:123
 [47] (::Makie.var"#1973#1983"{Button, Observable{Symbol}})(::MouseEvent)
    @ Makie ~/.julia/packages/Makie/ND0gA/src/makielayout/blocks/button.jl:68
 [48] (::Makie.var"#1371#1372"{Makie.var"#1973#1983"{Button, Observable{Symbol}}})(event::MouseEvent)
    @ Makie ~/.julia/packages/Makie/ND0gA/src/makielayout/mousestatemachine.jl:94
 [49] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [50] invokelatest
    @ ./essentials.jl:889 [inlined]
 [51] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [52] setindex!
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:123 [inlined]
 [53] (::Makie.var"#1441#1443"{Scene, Base.RefValue{Bool}, Base.RefValue{Union{Nothing, Makie.Mouse.Button}}, Base.RefValue{Float64}, Base.RefValue{Float64}, Base.RefValue{Bool}, Base.RefValue{Bool}, Base.RefValue{Union{Nothing, Makie.Mouse.Button}}, Base.RefValue{Bool}, Base.RefValue{Point{2, Float32}}, Base.RefValue{Point{2, Float32}}, Base.RefValue{Makie.Mouse.Action}, Observable{MouseEvent}, Float64, Module})(event::Makie.MouseButtonEvent)
    @ Makie ~/.julia/packages/Makie/ND0gA/src/makielayout/mousestatemachine.jl:299
 [54] #invokelatest#2
    @ ./essentials.jl:892 [inlined]
 [55] invokelatest
    @ ./essentials.jl:889 [inlined]
 [56] notify
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:206 [inlined]
 [57] setindex!
    @ ~/.julia/packages/Observables/YdEbO/src/Observables.jl:123 [inlined]
 [58] (::GLMakie.var"#mousebuttons#168"{Observable{Makie.MouseButtonEvent}})(window::GLFW.Window, button::GLFW.MouseButton, action::GLFW.Action, mods::Int32)
    @ GLMakie ~/.julia/packages/GLMakie/qLxLP/src/events.jl:102
 [59] _MouseButtonCallbackWrapper(window::GLFW.Window, button::GLFW.MouseButton, action::GLFW.Action, mods::Int32)
    @ GLFW ~/.julia/packages/GLFW/BWxfF/src/callback.jl:43
 [60] PollEvents
    @ ~/.julia/packages/GLFW/BWxfF/src/glfw3.jl:620 [inlined]
 [61] pollevents(screen::GLMakie.Screen{GLFW.Window})
    @ GLMakie ~/.julia/packages/GLMakie/qLxLP/src/screen.jl:449
 [62] on_demand_renderloop(screen::GLMakie.Screen{GLFW.Window})
    @ GLMakie ~/.julia/packages/GLMakie/qLxLP/src/screen.jl:937
 [63] renderloop(screen::GLMakie.Screen{GLFW.Window})
    @ GLMakie ~/.julia/packages/GLMakie/qLxLP/src/screen.jl:963

[Tortar]

mmmh, I can't actually reproduce, I get

maybe GLMakie version you installed is not compatible with Agents? I have GLMakie v0.9.11 and Agents 6.0.12 installed

[Tortar]

since you are using Agents 6.0 syntax, you have a old GLMakie version installed somehow probably

[markusgumbel]

Thank you for your quick response. I also use GLMakie v0.9.11 (and Makie v0.20.10 if this is relevant).
Just to be sure: Did you play the "step model" or "run model" button? Starting the program works for me too, but not stepping through or running the model.

[Tortar]

ah no sorry, didn't see that in your reproducer, yes the error is there also for me :-)

[Tortar]

I investigated a bit the issue and I'm unsure of what could be the cause because this works instead:

using Agents, GLMakie

@agent struct Particle(ContinuousAgent{2,Float64}) end

function build_model()
    space = ContinuousSpace((1, 1))
    model = StandardABM(Particle, space; agent_step! = particle_step!)
    for i = 1:100
        pos = Tuple(rand(2))
        vel = Tuple(rand(2))
        add_agent!(Particle(i, pos, vel), model)
    end
    return model
end

function particle_step!(p, model)
    randomwalk!(p, model, 1.0)
    if rand() < 0.5
        remove_agent!(p, model) # Number of agents changes here.
    end
end

p_color(p::Particle) = :red

model = build_model()

figure, ax, abmobs = abmplot(model; add_controls = true, agent_color = p_color)
resize!(figure, 1200, 800)
wait(display(figure))

which is strange because internally agent_color and agent_size are treated identically. Maybe it could be an issue with Makie itself? Do you have any idea @Datseris?

[Datseris]

I am trying to find where in the source code the UPDATE of plotted agents is done. I cannot, unfortunately. If you can find where the update of the scatterplot is done in the source code please paste a link here and I ll have a look.

[markusgumbel]

Thanks again for your analysis. I am not sure if I can help but I looked in the sources and found in
abmplots.jl some code where the changes are "lifted" according to Makie's lift concept.

function lift_attributes(model, ac, as, am, offset)
    pos = @lift(abmplot_pos($model, $offset))
    color = @lift(abmplot_colors($model, $ac))
    marker = @lift(abmplot_markers($model, $am, $pos))
    markersize = @lift(abmplot_markersizes($model, $as))

    return pos, color, marker, markersize
end

and before

function Makie.plot!(p::_ABMPlot)
    model = p.abmobs[].model[]
...
    p.pos, p.color, p.marker, p.markersize =
        lift_attributes(p.abmobs[].model, p.agent_color, p.agent_size, p.agent_marker, p.offset)
...
    return p
end

[Datseris]

Hm, I do not know the solution. I thought it was a concurrency issue with makie. Where one vector (eg size) is updated and a plot update is triggered before the other vector (e.g. color) is updated, leading to missmatch of dimensions. However, normally the plotting code works fine with agents being deleted, we have many examples and animations where this occur and agents have different colors. What is fundamentally different in this MWE that makes it not work...?

Is it really that the size is different? If we run e.g., the rock paper scissors example where each type also has different size, would we get the same error...?

[Tortar]

yes, tried, we get the same error.

I think I found a github issue and PR which describe the culprit of this: MakieOrg/Makie.jl#3658 and JuliaGizmos/Observables.jl#109

[Datseris]

great, thanks!

[Datseris]

We can also solve this on our side by the way. Instead of using @lift, we first create the observables, and then we create an on function that does:

on(model_observable) do model
pos_observable[] = ...
size_observable[] = ...
# and at the end:
notify(pos_observable) # this is enough to trigger plot update

it is actually super sipmle to change the source code to do that.