-
Notifications
You must be signed in to change notification settings - Fork 295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
batch of suggestions #2437
Comments
Hi, Is a batch trial what you are looking for? It allows you to attach multiple arms to one trial and deploy and evaluate them together. (See https://ax.dev/api/core.html#batchtrial for more details) In terms of trials running together, the GenerationStrategy objects allows you to specify "max_parallelism" which controls the number of trials which can be running at once. See the generation strategy wiki here for more details on that https://ax.dev/tutorials/generation_strategy.html#1.-Quick-start-examples Let me know if this helps or if you have any other questions. |
Thanks ffor your answer. yes attach mutiple arms to one trial. I have tried to use it with the follwoing example def create_experiment(): def evaluate_batch_trial(ax_client, batch_trial):
ax_client = create_experiment() batch_trial = ax_client.create_batch_trial() evaluate_batch_trial(ax_client, batch_trial) but it does not work correct? is there any tutorials which explain how mutiple arms can be used with example |
For a tutorial, you can follow the generation strategy tutorial https://ax.dev/tutorials/generation_strategy.html For your example, could you please provide error logs and more details about why it is failing? |
AttributeError Traceback (most recent call last) AttributeError: 'AxClient' object has no attribute 'create_batch_trial' , I have tried in the above code to use batch trail to get batch ot trials and then evaluate them but it seems that I did it wrong |
See https://ax.dev/api/service.html#module-ax.service.ax_client AxClient as of yet does not support batch trial, you'll need to follow the generation strategy tutorial to test out batch trials. |
https://ax.dev/tutorials/generation_strategy.html but this toutorial I think for just one objective function ? can I use it for batch trials in case of Multi-objective functions? |
Yes, try the "use_batch_trials" flag in the "choose_generation_strategy" method call. Ax/ax/modelbridge/dispatch_utils.py Line 290 in 9a624a2
|
gs = choose_generation_strategy( Main optimization loopfor _ in range(2): |
You can just call |
@Balandat so in this case I do not need to use choose_generation_strateg and no need to set the use_batch_trials=True if I understand you correct? |
Yes.
In a nutshell, yes. There are some subtleties about how exactly they are generated, but for all intents and purposes you can see them as equivalent in terms of generation. The main discerning factor of a |
for the of MOO (in my case differnt than the tutorial since I do not have the refernce point). should I follow the same steps since there are more detaials and also sobol sampling inclused etc. or should I follow
which is easy to understand it but in this case I will be not able to evaluate the results using one of the two algoritms explained in the tutorial(https://ax.dev/versions/0.1.18/tutorials/multiobjective_optimization.html) and If I folllow this tutorial how can I define the primary and secondary objective functions |
You can follow the same steps for MOO, no need to use
You mean b/c the tutorial uses the developer API to evaluate the different algorithms? It is possible to also use different algorithms than the defaults in You also want to look at the current version of the tutorial if you're on a newer version of Ax (0.1.18 as in the tutorial is ancient): https://ax.dev/tutorials/multiobjective_optimization.html |
@Balandat Thanks for your answer: my problem like the following I have two objective functions that I want to find the solution for them with non linear constrained and I need to get the values of the sugested parameters then use them to calaculate the objective functions but since get one sugesstion by one will requried long time I need to use the batch trial to get batch of sugesstions. I found the follwoing toutorial which looks not the new version : and the one which you sent me: my second question : in case that I will follow the implementation in the tutorial you share it (second link) how can I get batch of trials ? my third question : objectives = ax_client.experiment.optimization_config.objective.objectives |
Hi, `for i in range(5):
This will create a group of 5 trials to be executed together. "trials_dict" is the batch of trials that you want.
|
thank you very much for your answer . Does this means that I can also direct use just Pareto front after the iterations loop without to set which algorithm qNEHVI or qNParEGO ? , The Deep dive make me confused becuase I thought I should set /choose one from this two algorithm. for complex problem do you think that someone will need 200 iterations or more? . |
Yes, that is right- no need to set up qNEHVI or qNParEGO in order to use the pareto frontier call. I don't expect you'd need >200 iterations, I'd start with a low number like 25 to get a feel for the results. |
but how can decide we that 25 iteration is ok or we need more ? and in case that we need more should we start the experiment form first trial or can we continuou after the first 25 iteration and how can we do this in case that I found that 25 iterations need to be increase to e.g.50 without to start from the first trial ? |
is it possible to get multiple trials at once and then evaluate them together.
The text was updated successfully, but these errors were encountered: