Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix cases where losing epoch get a huge negative loss value #10216

Closed
wants to merge 3 commits into from

Conversation

stash86
Copy link
Contributor

@stash86 stash86 commented May 16, 2024

Summary

On some cases, you can have losing epoch (total profit< 0), but it can have a big negative loss value, which make the bot think it's the best epoch. This should fix such bug

Solve the issue: #10192

Quick changelog

  • <change log 1>
  • <change log 1>

What's new?

@coveralls
Copy link

Coverage Status

coverage: 94.666% (-0.01%) from 94.679%
when pulling 6306fcf on stash86:bt-metrics
into 19d6ce5 on freqtrade:develop.

@xmatthias
Copy link
Member

xmatthias commented May 17, 2024

I'm not sure it does if i'm honest.

i'm using the following test cases to look at this calculation - some results are from the issue - some are artificial to see which value has which influence ...

DRAWDOWN_MULT = 0.075

results = [
   (30.785, 0.35 / 100),
    (-221.333, 15.98 / 100),
    (-315.876, 40.04 / 100),
    (-459.923, 50.84 / 100),
    (-581.544, 68.98 / 100),
    (64.167, 5.31 / 100),
    (65.167, 5.31 / 100),
    (66.167, 5.31 / 100),
    (-628.566, 74.53 / 100),
    (602.105, 16.04 / 100),
    (602.105, 26.04 / 100),
    (602.105, 66.04 / 100),
    (602.105, 120.04 / 100),  # Drawdown after profit
    (-494.568, 70.59 / 100),
    (-251.262, 35.02 / 100),
    (-429.174, 46.65 / 100),
    (-759.062, 82.04 / 100),
]
for (total_profit, rel_drawdown) in results:

    res = -1 * (total_profit * (1 - rel_drawdown * DRAWDOWN_MULT))

    loss_value = total_profit * (1 - rel_drawdown * DRAWDOWN_MULT)

    if (total_profit < 0) and (loss_value > 0):
        pass
    else:
        loss_value = -1 * loss_value
    print(f"{res=:>10.7g}\t\t{loss_value=:>10.7g}\t\t{total_profit=:.3f},\t {rel_drawdown=:.2%} {res == loss_value}")

For explanation:
res is the current calculation
loss_value is the calculation in this branch

Output:

res= -30.77692		loss_value= -30.77692		total_profit=30.785,	 rel_drawdown=0.35% True
res=  218.6803		loss_value=  218.6803		total_profit=-221.333,	 rel_drawdown=15.98% True
res=  306.3902		loss_value=  306.3902		total_profit=-315.876,	 rel_drawdown=40.04% True
res=  442.3861		loss_value=  442.3861		total_profit=-459.923,	 rel_drawdown=50.84% True
res=  551.4578		loss_value=  551.4578		total_profit=-581.544,	 rel_drawdown=68.98% True
res= -63.91145		loss_value= -63.91145		total_profit=64.167,	 rel_drawdown=5.31% True
res= -64.90747		loss_value= -64.90747		total_profit=65.167,	 rel_drawdown=5.31% True
res= -65.90349		loss_value= -65.90349		total_profit=66.167,	 rel_drawdown=5.31% True
res=  593.4307		loss_value=  593.4307		total_profit=-628.566,	 rel_drawdown=74.53% True
res= -594.8617		loss_value= -594.8617		total_profit=602.105,	 rel_drawdown=16.04% True
res= -590.3459		loss_value= -590.3459		total_profit=602.105,	 rel_drawdown=26.04% True
res= -572.2827		loss_value= -572.2827		total_profit=602.105,	 rel_drawdown=66.04% True
res= -547.8975		loss_value= -547.8975		total_profit=602.105,	 rel_drawdown=120.04% True
res=  468.3843		loss_value=  468.3843		total_profit=-494.568,	 rel_drawdown=70.59% True
res=  244.6626		loss_value=  244.6626		total_profit=-251.262,	 rel_drawdown=35.02% True
res=  414.1583		loss_value=  414.1583		total_profit=-429.174,	 rel_drawdown=46.65% True
res=  712.3569		loss_value=  712.3569		total_profit=-759.062,	 rel_drawdown=82.04% True

The 2 don't deviate - not in ANY of the provided cases (not from the issue, nor in the artificial ones.


To be perfectly honest, i don't think the base calculation of this works the way it is now (the sign is just a small problem here).
drawdown is a value that's in the range of 0-1+ (120% can happen if you're in profit and then lose it all and more).
Profit however can be 0.1 (trading BTC?) - or 1000 (trading USDT).
The way it's combined leads to potentially big numbers (profit) which are either positive or negative - with VERY small numbers (we talk about numbers from 0-~1.5) - which are multiplied by 0.075 ... making a small number even smaller - reducing the impact of drawdown to almost nothing (visible in the test-cases with 602$ profit).

@xmatthias xmatthias linked an issue May 19, 2024 that may be closed by this pull request
@xmatthias
Copy link
Member

Closing this in favor of #10221 - which revises the calculation completely.

@xmatthias xmatthias closed this May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Hyperopt loss going wrong way ?
3 participants