Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Warning after updating to 2.1.0 #2340

Closed
marekott opened this issue Nov 27, 2023 · 9 comments
Closed

Warning after updating to 2.1.0 #2340

marekott opened this issue Nov 27, 2023 · 9 comments
Assignees
Labels
Type: Bug Something isn't working

Comments

@marekott
Copy link
Contributor

Describe the bug

After updating to 2.1.0 of Yarp.ReverseProxy we have suddenly started observing a lot of warnings logged for traffic forwarded to microservice using SignalR. For example:
UpgradeRequestClient: The client reported an error when copying the upgraded request body.
UpgradeResponseCanceled: Copying the upgraded response body was canceled.
RequestCanceled: The request was canceled before receiving a response.

Is it expected behavior after mentioned update? I've never observed such logs on previous version and there were no other changes that could explain it. If it is desired behavior (I can see there were some changes regarding logging: #2325) how should I understand above logs? Is there something to worry about?

@marekott marekott added the Type: Bug Something isn't working label Nov 27, 2023
@JohnCampionJr
Copy link

I am experiencing this too; is there any way to disable the warning?

@MihaZupan
Copy link
Member

Was this specifically a change when going from 2.0 to 2.1, or did you see similar behavior in other versions as well?
Can you also please share the exceptions (including stack traces) associated with these errors? That will help us narrow down where failures are occurring.

There were some changes around how different error conditions are classified, but I don't believe it should have affected these categories.

Errors like RequestCanceled indicate that the client talking to the proxy went away before we were able to get a response from the backend destination. My expectation is that you should be seeing it at a similar rate in 2.0 or 2.1.

@JohnCampionJr
Copy link

I returned to v2.0.1 and the warning went away. There is definitely some sort of regression.

I'm using it in this project:
https://github.com/JohnCampionJr/AspNetCore.SpaYarp/tree/multispayarp

I'll see if I can provide the exceptions.

@JohnCampionJr
Copy link

JohnCampionJr commented Dec 7, 2023

Here's the exception, only seen in 2.1 and not in 2.0.1.

.net8.0, macOS if that helps.

warn: Yarp.ReverseProxy.Forwarder.HttpForwarder[48]
      UpgradeRequestCanceled: Copying the upgraded request body was canceled.
      System.IO.IOException: The client reset the request stream.
         at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
         at System.IO.Pipelines.Pipe.GetReadAsyncResult()
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http2.Http2MessageBody.ReadAsync(CancellationToken cancellationToken)
         at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal(Memory`1 destination, CancellationToken cancellationToken)
         at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
         at Yarp.ReverseProxy.Forwarder.StreamCopier.CopyAsync(Stream input, Stream output, Int64 promisedContentLength, StreamCopierTelemetry telemetry, ActivityCancellationTokenSource activityToken, Boolean autoFlush, CancellationToken cancellation)

@MihaZupan
Copy link
Member

Between 2.0.1 and 2.1.0, the log level of "ForwardingError" (event 48) was raised from Information to Warning (#2025).
It's possible the only difference after updating to 2.1.0 is that you're not silently ignoring these logs anymore.

If you lower the log level you observe for YARP to Information while running 2.0.1, do you see a comparable rate of errors?

@MihaZupan
Copy link
Member

Have you had a chance to confirm whether this is the case for you?

@scottt13
Copy link

Same issue here, many warnings being logged for cancelled requests after upgrading to 2.1.0. We try to keep a fairly clean house in terms of warnings and errors and this is now flooding our warning logs with messages that we are not going to be able to do anything with because it is expected that some requests will be cancelled from time to time.

@MihaZupan
Copy link
Member

MihaZupan commented Jan 24, 2024

Agreed on the cancelled requests being too noisy here.

Given that there is no indication that there is a regression here resulting in more errors, and only that the existing errors were raised from Info to Warn, I'm closing this as a duplicate of #2195 which deals with reducing the amount of noise from cancelled requests.

@plachor
Copy link

plachor commented Jan 26, 2024

Hi @MihaZupan just to be sure I understand, you state that those 3 warnings mentioned by @marekott are logged due to request aborted by client and Yarp is simply logging them with warning where in fact are more like debug logs. So we can simply ignore those till you fix severity level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants