Does Autogluon have automated feature selection

[apavlo89]

Does it?

[mglowacki100]

Yes, but it is experimental feature disabled by default. To activate it in method .fit(feature_prune_kwargs={'force_prune': True})

[jmren168]

@shchur @Innixma
Thank you for the reply. It's a good idea to split the whole process into two parts.

[jmren168]

@shchur

Any ideas of how to get the pruned features?

I tried to use "predictor._learner.feature_generator.transform(train_data)", but still got the original feature set.
Or how to load a LightGBM\T1_Prune\model.pkl to get the used features?

Mank thanks.

[jmren168]

It seems like predictor does not save the pruned features. fit_summary is showed as bellow.

*** Summary of fit() ***
Estimated performance of each model:
                 model  score_val eval_metric  pred_time_val   fit_time  pred_time_val_marginal  fit_time_marginal  stack_level  can_infer  fit_order
0    LightGBM\T5_Prune   0.998457    accuracy       0.011999   5.186092                0.011999           5.186092            1       True          5
1    LightGBM\T1_Prune   0.998457    accuracy       0.013999   8.284000                0.013999           8.284000            1       True          1
2  WeightedEnsemble_L2   0.998457    accuracy       0.016000   8.967139                0.002001           0.683138            2       True          6
3    LightGBM\T3_Prune   0.996914    accuracy       0.017001   9.069832                0.017001           9.069832            1       True          3
4    LightGBM\T4_Prune   0.996914    accuracy       0.084004  60.405211                0.084004          60.405211            1       True          4
5    LightGBM\T2_Prune   0.992284    accuracy       0.012002   9.139832                0.012002           9.139832            1       True          2
Number of models trained: 6
Types of models trained:
{'WeightedEnsembleModel', 'LGBModel'}
Bagging used: False 
Multi-layer stack-ensembling used: False 
Feature Metadata (Processed):
(raw dtype, special dtypes):
('float', []) : 220 | ['a','b','c', ...]
*** End of fit() summary ***

I'm sure the feature selection is successfully pruning 220 features to 141 features, see log as below

Successfully ended prune loop after 15 feature pruning rounds (229.08s).
Feature count before/after feature pruning: 220 -> 141.

[jia-zhuang]

Just load the model.pkl, you can get the features:

import pickle
with open('AutogluonModels/ag-20240510_135549/models/LightGBM_Prune/model.pkl', 'rb') as f:
    model = pickle.load(f)

print(model.features)

[jmren168]

@jia-zhuang

Thanks. It works.