You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the configuration allows for a fixed list of plot types
goal
have an option to log custom plots
would be something like a callback or similar
use case example: derivations of existing plots for advanced analysis or to enhance the existing workflow
use case example: add plots currently not existing
why is this needed
at the point of time I could manually plot and log plots to e.g. MLFlow (or other tracking tools) by using its tracking API directly
the downside here is that PyCaret does not allow (as far as I know and understand) to do intermediate plots when using compare_models for example
Workflow example:
now, when using a script that sequentially uses compare, tune, ensemble / blend etc.
here, plots are logged to tracking but the only available plots are the fixed ones (in form of the IDs presented in the plot_model-function)
now adding plots in the intermediate steps would not allow to use a script with the above flow
instead one would have to go through a list of models to train, make predictions (namely e.g. y_true & y_pred) and make the figure, then log to tracking, then make the next model, and repeat
this would be the procedure for each step, so that in the end for all steps the custom plot is available
Workflow with custom plots
define the custom evaluation plot
add it via the setup
run the training script with multiple pycaret AutoML stages (compare, tune, enemble etc.)
for each step the custom plot is logged automatically, like it would with the current plot_model
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Summary
I am looking for the option to add custom logs.
Current
setup
(see code here, see documentation onsetup
here, see documentation onplot_model
here)goal
why is this needed
compare_models
for exampleWorkflow example:
plot_model
-function)y_true
&y_pred
) and make the figure, then log to tracking, then make the next model, and repeatWorkflow with custom plots
plot_model
Beta Was this translation helpful? Give feedback.
All reactions