This includes changing the title, adjusting the language to be more generic, and adding a section with guidance for HTTP CONNECT.
Most of the considerations in optimistic-upgrade, which originally emerged from consideration of "connect-tcp", also apply to HTTP CONNECT. We should consider expanding the scope of the draft to cover that topic as well. cc NAME


URL made the use of media type mandatory for appending to an upload. This warrants a new interop version.


As discussed in URL, this requires the upload offset to not decrease. The server is not allowed to "forget" data appended to an upload. The benefit is that it allows a client to reduce the amount of data buffered while the upload is ongoing as it receives acknowledgements from the server that data has been stored permanently and will not have to be retransmitted. Closes URL
URL added the requirement that the offset reported in 104 informational responses must not decrease over time. For example, it is not allowed that one 104 response includes and the next includes . For now, this requirement is not mentioned for responses to offset retrieval responses. As NAME mentioned in URL, we should add this rule there as well. Maybe it also makes sense to move this into a more central section of the document, so that we don't have to repeat this rule trice (upload creation, upload appending, and offset retrieval).
Thinking about this a bit more, progress reporting and offset retrieving are different enough that adding this requirement to offset retrieving might not actually make sense. In a single append, it's impossible for progress to go back and it's fine to be extremely strict. However, offset retrieving can happen much later. A server could have already forgotten about the previous uploaded body, but still want the upload restarted from 0. In the case that the client cannot supply the bytes, it can always decide to not resume, or perform an explicit cancel.
I am not sure if such a case would actually occur where the server has forgotten parts of the uploaded data but has not forgotten the existence of the upload resource itself, allowing the client to still resume the upload. Do you have an example application? I would have imagined that the upload should be considered as unresumable and failed then. The client could create a new upload if its still has the file data and the application logic allows that. That was my motivation behind enforcing a strict requirement that the upload offset never decreases. The Upload-Offset values should be a guarantee by the server that this amount of data has been saved and won't be lost again. If we don't include such a requirement, it could encourage servers to handle the received data without too much care because they are allowed to decrease the offset again.


This addresses the reference issues, comments and nits from the AD review (see )
NAME would you mind taking a quick look before I publish a new draft?
(tried using ietf-comments but ran into issues so just tracking manually in one combined issue)... cc NAME Thank you for the work on this document. Almost all my comments are about references. I think a new version is necessary before starting IETF Last Call, to avoid process issues along the way. Francesca The boilerplate is duplicate, please remove the second occurrence. Can you please update the reference to 8941 to draft-ietf-httpbis-sfbis ? That doc is with the RFC Editor so should not be holding this document up. Also, I believe the reference to draft-ietf-httpbis-sfbis should be normative, not informative, since terminology from that doc is used. Alternatively, if you want to keep the ref informative, you can import the part of the terminology that is necessary for this doc. I think that's a uglier solution, so I highly prefer sfbis to be made normative, but won't block on it. [URLPattern] "URL Pattern Standard", March 2024, URL needs to be indicated as Living standard (see RFC 9110 or 9421 for eample of whatwg specs references). Please fix this so that the Fetch spec is properly referenced (normatively is needed, I believe). RFC 8792 should be (informatively) referenced. I agree with Mark's write up, 5861 should really be informative. There is several occurrences of {Origin}, please fix. Please add a reference. It would be good to have an informative reference to 6265 (or even 6265bis). This review is in the ["IETF Comments" Markdown format][ICMF], You can use the [ tool][ICT] to automatically convert this review into individual GitHub issues. [ICMF]: URL [ICT]: URL


I do not agree with this suggestion. Upgrade is intended to support a not-yet-defined future. Limits placed upon it must be evaluated in terms of what will be excluded from future use, not what is currently being used in practice. If you want to limit what can be sent in a body, do that instead. The entire premise of this document is that a client will send something incredibly stupid and uncontrolled in a request body while asking for an upgrade. Just forbid that and be done. In any case, Upgrade with GET, HEAD, and OPTIONS has no such issues. Upgrade should also be possible to use with QUERY with any compliant server. If you can find a server that fails to process a valid request body and then proceeds to treat it as a pipelined request, that is a vulnerability in their code.
This PR originally assumed the deprecation of "TLS" in . I've removed that deprecation there, so other HTTP methods are necessarily allowed. Due to that change, I've rewritten the recommendation here, removed the PR stacking, and removed the "draft PR" flag.
Formally, Upgrade can be attached to any HTTP request, in which case the request is processed in HTTP/1.1, and the server has the option to Upgrade before returning the response. This includes requests with and without nonempty bodies. Handling of Upgrade with a nonempty body is well-defined, but the Upgrade paths that are actually implemented today (WebSockets and MASQUE) only use GET requests with empty bodies. NAME raised concerns that this is something of a "sharp edge" for security. For example, a server implementation might assume that there is no body in an Upgrade request, leading to a mismatch between the client and server when a body is present. We could make this problem go away by mandating that Upgrade only be used with "Content-Length: 0", and perhaps only with "GET". However, this requirement would appear to contradict RFC 2817, which says "A client MAY offer to switch to secured operation during any clear HTTP request", so that document would have to be Updated or marked Historic.
FWIW HTTP/2 upgrade has some statements that might conflict with the proposal. From URL before the client can send HTTP/2 frames. This means that a large request can block the use of the connection until it is completely sent. important, an OPTIONS request can be used to perform the upgrade to HTTP/2, at the cost of an additional round trip. On the other hand, RFC 9113 , so the topic of this issue is compatible with that notion with a bit of wordsmithing.
Well phrased. More concretely, for servers which read (and buffer) the full request -- including request header and request body -- before processing the request, this is a non-issue. However, many servers make routing and other decisions after the request headers are received. To handle upgrade after the request has been handled, all requests would need to check for Upgrade at the end of handling requests, not at the beginning, which is more common in code that I have seen and in code that I have written. I think it would be ideal if Upgrade requests for new Upgrade tokens were sent to endpoints for which an alternate resource is not provided if the Upgrade does not succeed. If a request with Upgrade is rejected, the HTTP status could reflect that. At the moment, a client can request "GET /altresource.mp4 HTTP/1.1" with "Upgrade: websocket" and the RFCs specify that /altresource.mp4 should be delivered in its entirety before the server switches to WebSockets, and the HTTP response headers included Upgrade: websocket, along with Content-Length or Transfer-Encoding: chunked for /altresource.mp4. The combination of request/response bodies for resources tangential to the Upgrade request is what I would like to see avoided in future Upgrade tokens. As NAME worded so well, I think there are sharp security edges here, and complications in implementations. HTTP/2 extended CONNECT is a better approach as it is only the :protocol change request, and does not include a request body or a request for an alternate resource, at least as I understand RFC 8441 to describe HTTP/2 extended CONNECT for . Another example of where request bodies can potentially be mishandled is if an origin server passes the Upgrade request to a backend via CGI, FastCGI, SCGI, reverse proxy, or something else. Even if the origin server handles the request body properly with the Upgrade request, I certainly do not trust an amateur PHP program to handle the request body properly before switching protocols. (Yes, I am sure that somewhere, someone handles this properly, but I am willing to wager there are also places where an empty request body is incorrectly assumed by naive scripts.) To be proactive, lighttpd 1.4.74 and later will by default ignore Upgrade in all requests with a request body unless lighttpd 1.4.74 or later is explicitly configured to enable request body along with Upgrade via URL (gw == gateway in lighttpd, for CGI, reverse proxy, and other backends) tl;dr: I am not asking to restrict HTTP/1.1 Upgrade only to the GET method. I would like to request that optimistic-upgrade add guidance suggesting that future Upgrade tokens should suggest/recommend use in requests without a request body, and note that some proxies or origin servers may ignore HTTP/1.1 Upgrade when a non-empty request body is present. Thank you for your consideration.
Since we are discussing this publicly, I hope these details are appropriate. lighttpd 1.4.56 and later supports the HTTP/1.1 (now deprecated by the RFCs), but lighttpd ignores unconditionally if a request body is present. Among the reasons for doing so -- in addition to simpler implementation -- is that if an attacker controlled the request body content and wrote it in such a way to start with HTTP/2 Connection Preface and an HTTP/2 request, and if the server failed to handle the HTTP/1.1 request body before switching to HTTP/2 protocol, then an attacker could inject HTTP/2 requests via that mishandled (on the server) HTTP/1.1 request body.
As I understand, this problem is specific to HTTP/1.1. I'm not sure if we could add new requirements, unless we change the core HTTP documents and can modify all existing HTTP clients that use upgrade to adopt the new behavior. At the same time, IMO it would be perfectly reasonable to point out the "sharp edges" and provide guidelines to new extensions that would leverage the upgrade mechanism. In that respect, I like the idea of recommending use of GET but not any other HTTP method for new extensions, as suggested in the original post from NAME
NAME Yes, the request header is specific to HTTP/1.1. I can imagine a future Upgrade token which requires POST with some sort of (short) request body containing authentication and/or payment information. Even so, I still like the idea of recommending that any new HTTP/1.1 Upgrade tokens prefer to use GET without a request body, or to use other methods if a request body is required. Aside: NAME you might be interested in adding some Upgrade tests to URL, specifically for request smuggling where servers support and might not properly handle an unexpected request body. (lighttpd always ignores if request body is non-zero length.)


A number of minor misc changes and corrections to improve readability, usability, and consistency throughout the spec.
6265bis is nearly ready, but after years of changes there are probably some minor editorial issues that should be corrected. Someone (me) should proofread it and fix any problems found.
The RFC Editor will help with that.
Ah, good to know. I'll still give it a first pass to clean it up a bit.
LGTM.


This PR clarifies a few things around the Upload-Limit field from : The last upload append request may now be smaller than the minimum append size specified in . For example, if a 510 MB upload is split into 100 MB requests, the last request can just be 10 MB. It clarified how a client can indicate the entire upload length when creating a new upload via the Content-Length header. The PR also mentions that no resumable upload resource may be created if the minimum size is not met, but the server might still accept and process the data (although without offering resumability) The Upload-Limit header is added to informational response in an example


This avoids having to determine whether is an actual existing standard or not. This is an alternative to
Merging for draft cutoff.
The lists an entry for HTTP/.\, and specifically identifies "2.0" as a possible version value, as instructed by . However, this flow seems to have been superseded by "Upgrade: h2c", which was introduced in RFC 7540 and deprecated in RFC 9113. I am not aware of any implementations of "Upgrade: HTTP/2.0" in clients or servers. I think we have two main options: Status Quo: Regard "HTTP/.\" as a mechanism that is formally defined but not implemented anywhere. Provide formal analysis of its security for completeness. Deprecation: Update RFC 9110 in this draft to remove the definition of this upgrade token. Mark "HTTP" as "obsolete" in the Upgrade Tokens Registry.
Moving discussion over to the issue, perhaps. Upgrading to TLS is a defined thing that gets used; I'm not clear that we need to deprecate that. Upgrading to future HTTP versions is a little more interesting. Certainly it's illegal to upgrade to or because upgrade isn't defined by those specifications. RFC 7540 defines a distinct token, RFC 9113 deprecates it, and RFC 9114 does not define an Upgrade path for obvious reasons. It certainly looks like we've decided not to use this mechanism for changing HTTP versions. Does that mean we should eliminate it? I'm not opposed, but I'm also not certain we need to go there.
Moving to TLS with upgrade (i.e., RFC 2817) is now so much of a niche thing that I think that deprecation would be wise. We've heard that it is still in use, but it's certainly not inoperable in quite the same way as other parts of the protocol. Either way, I think it would be better to write a separate document deprecating Upgrade (either completely or more narrowly) than to sneak it in here. This document can point out that it is unsafe in the ways in which it is unsafe, but it doesn't need to make the deeper cut to achieve that.
It seems clear that we don't want to deprecate in this document, so I've removed that text from . I've kept open to discuss the fate of . The WG seems to believe that this mechanism is already deprecated, if it was ever defined at all, and the IANA registry entry is just misleading or incorrect.
The trouble is that exists in a sort of quantum superposition of standardized and deprecated, but that will collapse if we mention it in this draft in any way. The options I see are: Don't claim (as the current text does) to present an exhaustive analysis of the defined Upgrade Tokens, and silently ignore the registration of . Treat it as a live standard. Declare that it was already deprecated. Deprecate it.
I'd say deprecating isn't particularly useful or interesting, since that doesn't impact implementations. If we can't agree on which quantum state we are in and want to be in at the same time, then I'd leave the cat in the box. The more interesting question is: should implementations do something to protect against a dangerous ? For h2, the preface should protect against smuggling. Where do we stand for other versions, is there a risk we should warn implementers about?
My expectation is that implementations of do not exist, and never will, so I don't think it's very useful to provide guidance to them. There's a question of whether one can even speak about a "downward Upgrade" from HTTP/1.1 to 0.9-1.1. Assuming that is allowed, I think 1.0 and 1.1 are safe, if we assume that the HTTP-Version line is sent/repeated at this point. (Does Upgrade: HTTP create a new HTTP connection or re-version the existing one???) They both start with "HTTP/1", which is not a valid method name due to the "/". HTTP/0.9 is possibly more interesting since it does not send an HTTP-Version line. I don't see a way for the server to figure out whether the client is speaking HTTP/1.1 (after the rejected upgrade) or HTTP/0.9 (optimistically). However, I don't think this poses a security threat, so long as the attacker-controlled data is the same in HTTP/1.1 as it is in HTTP/0.9. There could possibly be a problem if the client, believing it is speaking HTTP/0.9, allows untrusted code to manipulate headers that were not special in HTTP/0.9 but are special in HTTP/1.1 (e.g. Content-Length, Transfer-Encoding). But as I said, this strikes me as counting angels on the head of a pin. "Upgrade: HTTP/0.9" is just not a thing, and we should all agree on that and move on.
I totally agree with you. I was more thinking about guidance for intermediaries that support other Upgrades (e.g., WebSocket) for what they should do if they encounter unknown/unexpected upgrade tokens like these. If there is no such guidance, than we can just go back to pretending that doesn't exist
HTTP intermediaries can't really do anything with unrecognized Upgrade tokens in HTTP/1.1. (Not even the useful thing, which would be to convert them to/from for Extended CONNECT.)


Relax DNS requirements when ORIGIN is received Change the initial origin set to just SNI
wait - no. You wouldn't send the alternative in the ORIGIN frame, you would send the origin that you are the alternative for, right? that would make sense - if you want to use an alternative you can't rely on the SNI default.
this is indeed looking goodminor nitsThis is looking good.


Fixes URL
NAME could you take a look?
Comment by NAME This is rather unimportant, but I just wanted to mention it in case the authors find it useful. Feel free to ignore. The document states that there are no new security considerations, but that's perhaps not quite true. I think it might be useful to call out that an implementation cannot rely on its peer behaving correctly, so implementers will have to take into account they may still receive oversized frames from misbehaving clients. This is arguably no different from the situation today, so it can be argued that the current considerations are accurate. I just thought it might be useful to call it out so some engineer doesn't remove validation checks since the other side is supposed to behave now. Just because we have standards, doesn't mean that everyone complies.
RFC8878 says in its security considerations: Perhaps we could add something like: "Decoders still need to take into account that they can receive oversized frames that do not follow the window size limit specified in this document and fail decoding when such invalid frames are received." NAME Thoughts?
We could do something explicit like that, yeah. But I'm partial to just broadening the existing statement a little: This seems sufficient to me because this issue is called out explicitly in para 4 there (URL). What do you think, NAME NAME
Don't have strong opinions, but I think maybe adding a sentence explicitly about the window size limit is good, given the document is specifically about enforcing the window size limit. This will ensure nobody misses it even if they don't click through the reference to RFC8878.


This is a partial revert of 5469d51 from PR
Talked with NAME a moment ago. I think there are some additional changes that will be necessary, but I don't object to the idea of this revert. Seems like a mistake in the original PR.
Happy to amend the PR if you can give me an idea of what additional changes are required.
I wrote up a change that should cover all the points we want to make. Namely, we still want UAs to accept that larger character set while encouraging server to use the more limited character set.
This looks reasonable to me, thanks to you both.


Editorial stuff, mostly line wrapping and adjusting terminology. Happy to chat through it if you like.
thanks for this


This makes it clearer what standard is being depended on, while still deep-linking to the right section of that standard. This generates citations like "see of ]". The "RequestDestination" part is free text in the attribute. I don't see a way to change the "Part" text. We can use the to rearrange it to "see ], )" if y'all prefer.
Great, thanks for the PR. It's a lot cleaner than a bunch of independent references.
NAME just filed, thanks. URL URL
NAME have you asked the WHATWG folks for stable anchors to those sections? See URL for details of how to do that.


adds more used definitions from HTTP.
representation metadata, content, ... in Conventions and Definitions.


splits C-E and T-E sections clarifications quoted from HTTP and MESSAGING link to DIGEST-FIELDS appendix This PR contains some quotes from HTTP and MESSAGING. While they can be redundant, the previous wording made me think that they were not.
Transfer-Encoding does not affect the selected representation data, see URL Since it only apply to MESSAGING, it should be probably decoupled from a section addressing representation metadata (e.g., content codings): URL
Thank you very much!


Mark moved this note in his edits, but I think that we don't need it any more at all. The security considerations will suffice.


Mostly editorial, but touches normative text.
I'm looking for an additional qualification on the statement, something like: Originally posted by NAME in URL
NAME also suggested a reframing to "SHOULD": Originally posted by NAME in URL (Also, should we rename the "safe-method-w-body" label to "query-method", now that that naming thing is settled. It took me a while to remember the right label.)
I'm not confident that implementers will always know what might be sensitive and probably should not expose any of the payload in the URL. I think the qualifier doesn't reduce the suggestion's practical scope at all, so if it gets us to consensus, let's add it.
I'm less concerned about this ending up in logs than you are, it seems :) For me, the risk that a server log might have sensitive information is already at 100%, which means that log processing needs to include considerable care as far as privacy and security risks go.
Both variants work for me, and IMHO are better than what we have now. Mike, pick one :-)
Query media types could evolve whose syntax allows for specification of elements that the user/user-agent considers "sensitive", and also the server could know from the schema of the underlying data. But really I think a big hammer is needed here: the default should be that servers encode none of the query contents into the resulting user-agents should be able to request that behavior with a request header (e.g., ) But user-agents should also be able to request that the resulting encode as much as possible of the request body so that the user can end up having a URI that can form part of the UI. This should be completely as far as the server is concerned, and will not always be possible due to URI length constraints and/or the difficulty of encoding complex queries in URI local-parts and q-params.
I believe that should be completely up to the server.
I'm inclined to agree with Julian here. From the client's perspective, the server is a monolith -- in the course of handling the request, the server necessarily sees the contents and the data (multi-party compute schemes notwithstanding) and the client trusts it to do so according to whatever privacy policy exists. It's the server operator's issue whether they wish to keep certain information out of their logs in order to reduce how tightly access to their logs must be managed. Plus, it's implementation guidance at this point, not protocol conformance. We point out the concern and let people in the real world figure out how they navigate it.
An optional header by which the client can request a URI that encodes the query (which may not be feasible, but often will be) vs. not, seems like a good idea to me -- the server doesn't have to grant the client's wish, but having this request header specified will result in better usability. (To go with this an optional response header by which to indicate which option the server went with would be nice.)
This is all so conditional that I wonder whether it even needs a SHOULD (NOT). Would it be enough to say instead. The other options are fine.


The definition of upload creation requests allows the client to include the entire file, some file data or no data at all. While we provide this flexibility, we don't give guidance on when which approach is best suited. This PR adds this missing piece and explains how an optimistic client can attempt to upload the file in one request and how a careful client uses a separate request for creating the upload. In addition, it also mentions how to transparently upgrade to resumable uploads, which wasn't explained in detail before. Let me know if you have any feedback on this.
Are "Optimistic Upload Creation" and "Upgrading To Resumable Uploads" the exact same way? Maybe the 3rd way could be checking first before uploading?
The way they are described right now they are similar, yes. In theory, an optimistic upload creation can also be used where the client doesn't upload the entire file, but just a large part of it (think 500 MB of a 10 GB file). When upgrading a non-resumable request to a resumable upload, this act of division is not allowed. But I am not sure if we should go into that much detail here. If not, then we can merge these two sections. Upload limits are a good point! With the careful upload, the client also knows the limits before sending the first PATCH request. The optimistic client must be prepared to receive a 104 response with limits that it has to adapt to. We can mention that a careful client may also send an OPTIONS request before hand if it deems that necessary.
I think upload-limit can just be enforced on the server side. If it sends a 104 and a 5xx response, the client would automatically retry. We have deployed a client-side implementation that is not aware of the upload-limit header field.
Thanks for all of your comments! I updated the PR to put the section about upgrading to resumable uploads under the optimistic upload creation since they are similar. I also added a brief mention of Upload-Limit and shortened the text a bit. Let me know if you think the text needs more adjustments.
Just a minor nit. Otherwise I think this is a very good addition and agree with NAME that we should add some text with regards to the . Proposal for optimistic: The client needs to adapt to the header provided in the intermediate response and fail the request if the file is larger than the maximum allowed size. Proposal for Careful: The client can use the header provided in the to verify that the file is allowed to be uploaded before sending any data.Some editorial suggestions, but one big question: What is the material difference between the first and third strategy? Could these be the same strategy?


This change allows client to discover support for resumable uploads and potential limits via an OPTIONS request. Closes URL A brief comment on this sentence: Once the server has created an upload resource it might have more information available (user details, file metadata etc) and apply different limits than it would have known upfront for an OPTIONS request where this information was not available. We allow this difference but recommend that the upfront announced limits are not looser than the limits after the upload resource has been created. The client can then assume that it can create an upload resource if it obeys to the upfront announced limits even if looser limits will then apply later. Does this make sense? Or is this addition rather unnecessary?
Is OPTIONS a required feature on the server side? What about requests?
Yes, I think a server should be required to support this. If it only wants to provide resumable uploads for specific users, it does not have to announce this for unauthorized users. If a server supports resumable uploads for all target URIs, then it should respond with to an request. But I am unsure how often this actually happens.
introduced for the server to indicate limits on the upload resource (e.g. maximum size). These limits are announced to the client during upload creation in the informational or final responses. However, the client cannot discover these limits without attempting to create an upload resource. A client may want to avoid creating an upload resource if it knows the the server limits cannot be satisfied. A possible solution is to allow clients to send an OPTIONS request upfront to the URL from upload creation. The server might then include the Upload-Limit header in its response, so that the client knows them before trying to create an upload resource. However, a client is not required to send this OPTIONS request upfront. This would also be handy in a possible future where browsers natively implement resumable uploads. The response for a CORS preflight request might then already include information whether the endpoints supports resumable uploads and, if so, under which constraints. The browser can use this data to optimize the uploads.
Early thought: Would it be beneficial to include some kind of marker that this endpoints supports resumable upload in the OPTIONS response? For client's that does not support upgrading requests (e.g. due to lack of support for 1xx responses) who whishes to know upfront if the upload can be split (Upload-Complete: ?0) or if a regular upload (without resumability) should be performed.
Interesting idea! The presence of the Upload-Limit header could be used as an indicator for support of resumable uploads. In theory, its value can be an empty dictionary as well if the server does not impose any limits. If the response does not include the Upload-Limit header, the client can either attempt an upgradable upload or directly fall back to regular uploads.
URL
Too bad we can't use the empty dictionary header as an indicator. It's probably an edge case anyway that a client, not knowing the support of the server, would issue an OPTIONS request to find out? Or maybe not, given CORS? What do others think?
Thanks for the pointer, Lucas! If we cannot use an empty value, a server could also fill in a dummy value, e.g. . This is only necessary if the server does not apply any limit at all (not even ), which I suppose is very rare in reality. In general, the situation where a client wants to know upfront whether the server supports resumable uploads appears like an edge case to me. I assume the client will often know that the server supports it or otherwise will attempt to transparently upgrade a traditional upload into a resumable one. So I don't think we have to spend too much energy on this. How do you feel about this, Stefan?
NAME Yeah I think that is fair enough. The use cases where a generic client (without prior knowledge of server support nor support for informational responses) would send an extra OPTIONS request seems slim to none. The CORS example is interesting but mostly (always?) happens in a browser which does support informational responses and thus would use the 104.
Opened URL for this.


Closes URL A client can express its interest in digests using during upload creation. The server can then respond with after POST and PATCH requests to share its digest with the client. The client computes its digest as well during the upload and compares the digest to spot issues. The representation digest of an upload resource covers the entire bytes that have been uploaded so far (i.e. right until the upload offset). NAME NAME Is this in line with the semantics of RFC 9530?
NAME Need a week for the review :)
Preliminary feedback: I'd simplify the language/split text on multiple lines; Describe processing in separate subsections, e.g.
Thank you for the feedback, Roberto. I added subsections for representation and content digests in URL I think this division helps the reader and also makes sense since representation digests can cover the entire upload, while content digests only cover one request. Let me know if you think differently.
URL added some guidelines on how a client may include integrity fields to upload requests for protecting the integrity of a single request or the entire upload. However, this either requires the client to compute the digest upfront (before sending a request or starting the upload), or the use of request trailers, which is not very widespread and not always possible (e.g. the ). Instead of requiring the client to compute the digest upfront, the server and client could calculate the digests while the upload is ongoing. Once the upload is completed, the server can send its digest to the client in the response to the request that completed the upload. The client can then compare the server's digest to its computed value. If they don't match, it can raise an error to the end user and decide to not continue using the uploaded data. The client should indicate its interest in the server's digest when an upload is created, so the server is not required to compute the digest if the client has no interest. I am wondering if the from can be used for this. Can a client include in the first upload creation request, but the server delivers the digest potentially later in a response for a subsequent request? An upload may be spread across multiple requests.
Should also be supported? For multi request uploads, there's really no use to continue if request 3 of 100 got corrupted.
As NAME mentioned in URL, is not directly applicable here because it asks the server to send the digest corresponding to the response content, not the received request content. Good point that a client may validate digests while a multi-request upload is progressing. A server could respond with the latest digest of the entire uploaded data (not just the data from the last request) using after POST and PATCH requests. The client can then validate and decide to abort if the digests don't match. This usage would be in line with the PATCH example in . The server responds with the digest of the representation after the PATCH has been applied. The example also notes that the server is not required to respond with the representation itself and could also have returned a 204 (as is done for resumable uploads):
Makes sense, thanks for clarifying. is most likely good enough given that it indicates the current checksum of what the server received.
NAME please, stalk me for a review in the next week :)
NAME It took me a bit more than a week, but here we go: URL Happy to hear your feedback :)


This increases the complexity of a minimal server implementation and decreases the complexity of a rich client implementation. Addresses
Another option would be to say that connect-tcp always uses capsules, that would reduce complexity even further
That would reduce the complexity of a rich server implementation, but it would increase the complexity of a minimal client implementation. I don't think that is a wise trade.
Do we think minimal client implementations will exist? I was imagining that they'd use regular CONNECT
The spec probably wants more language to describe what a client should do when a server does not enable the capsule protocol, since thst a failure mode for this optional behaviour.
NAME Template-based configuration is valuable with or without the capsule protocol. Also, if you're not actually using capsules, it adds bandwidth overhead without any benefit. NAME We don't usually say "the server MUST Y". Is there a reason why that is necessary in this case?
It seems weird to me that the server echos something back that the client doesn't check. Since the server has a MUST requirement to process whatever format the client tells it, we could just drop the response header and state that a 2xx is the signal that server accepted connect-tcp with or without capsules.
The response header is for the benefit of intermediaries. Clients don't rely on it. This is also true for CONNECT-UDP.
That's a good point and might be worth saying explicitly in the text. However, we do have a pattern in RFC9298 that levies requirements on request and response message and says "if any of these requirements are not met, the client MUST treat the procying attempt as failed and abort the request". Since connect-tcp levies a MUST about capsule-protocol on the response message, using similar text seems consistent to me.
To move this discussion away from a merged PR and into an issue, I filed
As of draft-ietf-httpbis-connect-tcp-05, connect-tcp now optionally supports the capsule protocol. However, this optionality adds non-trivial complexity to implementations. As currently written, if a client wishes to use capsules, the server can reject the use of capsules, and now the client needs to cache that this server doesn't support them. However it's undefined for how long the client should cache this information, and within what scope. We could potentially figure out recommendations for all these questions, but fundamentally it might be best to instead make capsules mandatory for connect-tcp, like they are for connect-udp and connect-ip. That would simplify implementations and improve consistency.
Individually, I think it would be interesting to try out this direction — probably based on implementation experience for how easy it is to add. On one hand, this makes connect-tcp less of a trivial drop-in. On the other, it opens up more possible use cases.
I heard a good deal of support for this direction in the HTTPBIS meeting.
Thanks!


We started this discussion in the comments of but it deserves its own issue. Now that has landed, capsule support is mandatory for servers but clients can decide whether or not to use it by sending the header. Another way forward would be to just say that connect-tcp always uses capsules. Some reasons why: it simplifies server implementations it simplifies client implementations that want capsules it would be consistent with every other that uses capsules it avoids reusing to mean something different from what was intended in RFC 9297 Since I don't see any real-world reason to use without capsules, this sounds like the best path forward to me cc NAME NAME NAME
I'm sympathetic to the argument that there may be cases where someone wants templates tcp without the overhead of capsules. So how about we move the capsule negotiation to the upgrade token? I.e. one token if you want capsules, another if you don't. Then it makes the use of capsule-protocol header more consistent with how it's already used in other connect-foo docs.
That's a good compromise. That allows me to only implement the upgrade token that enables capsules and not have to deal with the complexity of negotiation. Another option would be to say that if then the capsule extends to the end of the stream. That works because stream offsets only go to UINT64_MAX so that length cannot ever be valid. It's a shame we didn't do that in 9297 though
I think this approach works for the sender to indicate its disinterest to use capsules. But it does not allow the receiver to refuse use of capsules by the sender. By saying using connect-tcp without capsules, I think we are referring to endpoints that do not want to implement capsule support at neither send-side or receive-side. Assuming that is the case, I think using different upgrade token is the way to go, if somebody asks for connect-tcp without capsules.
OK, PR up: URL Please review.
WfmThanks!


This leaves only the Capsule Protocol mode
I do think we should add FINAL_DATA, but I'm fine with merging this first and then discussing that in
As of draft-ietf-httpbis-connect-tcp-05, connect-tcp now optionally supports the capsule protocol. However, this optionality adds non-trivial complexity to implementations. As currently written, if a client wishes to use capsules, the server can reject the use of capsules, and now the client needs to cache that this server doesn't support them. However it's undefined for how long the client should cache this information, and within what scope. We could potentially figure out recommendations for all these questions, but fundamentally it might be best to instead make capsules mandatory for connect-tcp, like they are for connect-udp and connect-ip. That would simplify implementations and improve consistency.
Individually, I think it would be interesting to try out this direction — probably based on implementation experience for how easy it is to add. On one hand, this makes connect-tcp less of a trivial drop-in. On the other, it opens up more possible use cases.
I heard a good deal of support for this direction in the HTTPBIS meeting.
Thanks!


The current text could be interpreted as applying to other HTTP versions, which is not the intent. cc NAME


Adds an example to the Examples section to clarify and illustrate that cookie names are case-sensitive. Additionally adds a clarification that cookie names could be empty to improve readability.
Agreed
What should a user agent do with the "set-cookie-string" ? The current RFC says that the name string can be empty and then in a subsequent step to ignore that cookie. URL
Hi, instructs >If the name-value-pair string lacks a %x3D ("=") character, then the name string is empty, and the value string is the value of name-value-pair. Otherwise, the name string consists of the characters up to, but not including, the first %x3D ("=") character, and the (possibly empty) value string consists of the characters after the first %x3D ("=") character. which covers the case of a nameless cookie such as . I think clarifying, similar to the value string, that the name string could be possibly empty would improve readability, I'll make that change.
Thank you NAME This means there is a change in the recommended parsing algorithm: now empty named cookies are ok. Is that intended?
That's correct. Even though it's strongly not recommended we found that servers still produced unusual set-cookie strings, including empty cookie names, and many UAs accepted them. So the parsing change is meant to standardized behavior so that UAs can at least all do the same thing with them. I still strongly recommend that any sites only use cookies with a non-empty name that conforms to 's syntax. Same for the value field too, now that I'm mentioning it. It'll probably work otherwise, but why make your life harder?
I think the spec would benefit from getting this fact explicitly specified. and are two different cookies.
If you had asked me if this already existed I would have guessed yes. Seems like an obvious improvement
LGTM. It'd be very nice if we could remove the notion of empty cookie names, but since we can't we should document them clearly. :/


If the client gives Upload-Complete=false then the server must respond with Upload-Complete=false and not Upload-Complete=true.
Thank you for this correction!


It's customary to put the terminology and BCP 14 block in the same section, but this works.


NAME thoughts?
Seems fine, with the edits Mark suggested. I'll merge and make those changes unless Paul gets to it first.
Looks pretty good to me, some tweaks here.


Related discussions: URL URL
NAME thanks for the review, updated.. ptal.
Thanks!
LGTM


This changes the grammar and text to permit multiple ALPN tokens in the header. This is necessary for HTTP tunnels ("http/1.1" and "h2") as well as some WebRTC cases. I made some editorial changes at the same time. I'll split those out if we don't accept this.


and .
Looks good overall. changes the semantics of . Not only does it effectively put on subsequent responses from the same origin for the given lifetime, it also suggests that Client Hints should be added to requests to that origin, no matter where the requesting page has come from. Not a problem, but right now it's conveyed almost as an aside. This needs to be clearly specified so it isn't missed / misunderstood.
NAME updated, PTAL.
Thanks Mark!
This is better.


While testing firefox's implementation of this draft Vlad determined that some versions of Chrome throw PROTOCOL_ERROR and close the connection if it receives a type 0xB frame with a non zero length on stream 0. Other frame type numbers, such as 0xC do not seem to have that problem. testing indicates this is fixed in version 59 and is a problem previous to that. (57 is in release as I write this). presumably this is related to historical code around BLOCKED which was proposed to use 0xB. Prudence says to just use a different code point for the moment.
Question for NAME and NAME should we somehow reserve 0xb to avoid this problem?
If this is fixed in Chrome 59, why bother? Actually, I'd also revert this change.
Neither. There are still old Chrome versions deployed (some people lag, for various reasons), so it's not safe to use now, but it should be in the not-to-distant future; we just need to wait a while to use it (presumably for another extension). Reserving it for all time would be a bad precedent.


This changes Expect-CT to be a comma-separated list of directives (which may be name=value pairs), and adds some examples. Also allows the server to send multiple header fields so that they can be combined, as per URL Fixes issue ,
Would it be possible to include an example in the spec? Something like... Expect-CT: enforce; max-age=3600; report-uri="URL" As I believe most developers like to work with examples, rather than just using the spec definition as found under the heading "Response Header Field Syntax". :-)


... for your consideration; feel free to take, discard, or do whatever else you like :)
Thank you so much for the improvements. This PR is a tremendous opportunity for me to improve my skills (both in terms of writing a specification and in terms of English).


Fix one typo. Technically, a set is always a subset of itself. Therefore current wording implies that if two connections have identical Origin Sets, then neither should be used for new requests, and both should be closed. Fix this by saying "proper subset" instead of "subset".
Thanks!


The PR introduces a new SETTINGS parameter that can be used by a server to notify if (and how) it makes use of the CACHEDIGEST frames. You might find the discussion about how to use the parameter interesting. The text of the PR, under the assumption that the underlying transport will be TLS 1.3, suggests the client: to wait for SETTINGS parameter when doing a full handshake retain the value of the parameter in the TLS session cache and reuse the cached settings parameter value when doing 0-RTT resumption
Looks good to me, delta the one comment I made.


. Since I wasn't sure if we can make normative references from an appendix, RFC5234 (ABNF) and RFC4648 (Base64) has been added as informative references.
NAME Thank you for your review. I believe that I have addressed all four issues in the commit above.
NAME Would it make sense to also describe the cookie-based method in this spec?
While we could do that, I am kind of negative against it. We need a specification for Cache-digest using HTTP/2 frames or HTTP/1 headers, since it needs to be defined as an interaction between a server and a client (that has the knowledge of what is being cached). OTOH, we do not need a specification for a Cache-Digest cookie, since it is totally controlled by the server. So it is entirely up to each server implementation to use whatever format it wants. Also, it is not easy to find out the right way to implement a cookie-based Cache-Digest that would work on all web browsers, since the timing the browser consumes the cookies sent from the server depend on the timing of the response. To summarize, the relatively low necessity and the technical difficulty makes me hesitate in pursuing the idea.


The consequences for ORIGIN + Alt-Svc with a port change aren't awesome, but this should at least remove any confusion.
Are you going to take another go at this?
Sorry, I let this one drop because I missed your suggestion, which is good.
Thanks!


The header needs to be registered if it's going to appear in an RFC. Looking at 7540, I think the frame type needs to be registered as well, since "Experimental Use" is private / local only.
Thank you very much!


This is a PR that essentially rebases the change proposed in . To me the change proposed by NAME is a clear improvement. I believe that we should merge this.
Regarding The requirement in the last sentence needs to be re-worded, since includes parsing and the client is clearly parsing those header fields and using them.
I would rewrite the paragraph as
Collision with my suggested edits; closing. Please ask to reopen if you still think there's a problem; we can address it in WGLC.


removes statements that are either redundant or contradicts with RFC 7230, 7231 clarifies how a server is expected to do when generating a response
While it is not required to specify how an intermediary might behave, doing so might be beneficial as suggested in URL We could possibly have something like the following below the example.
works for me.
NAME Thank you for the answer. Adopted the text (with nits) in d0903d6.
NAME Thank you for the review!
Looks good to me!


Clarify that the disappearance of a header field (that once existed in a 103 response) from the following 103 responses does not indicate the retraction of the expectation that the header field will be included in the final response.
ecb2c7e clarifies the general rule that this PR relies on (i.e. the nonexistence of a header field in the 103 response cannot be used as a signal that the header field will be absent in the final response), at the same time addressing Spencer Dawkins' suggestion to clarify that "the server can add header fields in the 200 that were not present in the 103."
In section 2. "103 Early Hints" the last paragraph describes that 103 can be send multiple time. The given example reads as if a server may correct its previous 103 that is made from a cached resource. So the following 103 are correcting the older ones, they are only adding headers to the previous ones or are they replacing them completely? Maybe sentence clarifying this would be good.
This is an interesting question. I think that we should agree on what the expected behavior is before discussing how (or if) we should update the text. Regarding the question, I think that we should consider two (or more) Early Hints responses as expectations from different sources, rather than considering the following one to update the earlier ones. The reason I think so is because nonexistence of a header field in the 103 response does not imply that the header field will not exist in the final response, since we are not required to include in 103 response all the headers that are expected to be included in the final response. So consider the following example: a caching intermediary might generate an Early Hints response from a stale-cached response. That Early Hints response could contain a header field that will never be included within an Early Hints response sent from the origin. In such case, considering both of the Early Hints responses as genuine makes the most sense.


I left the term "request-uri" untouched (it was renamed in the base spec because it's not always a URI). We may want to change that separately, or even refer to the effective request URI (URL)
NAME - ping?
LGTM, thank you for the patch, and for the ping!


(I accidentally put this on master without noticing earlier, sorry.)
Can we just have 425 now that it's official-ish? From martinthomson/http-replay


NAME was confused by the text here and I confess that I don't remember why it was added. There are actually esoteric cases where you get server data that you can read before the handshake is complete (QUIC tends to enable that sort of mess), but there is no circumstance where the client can read the 425 but can't also determine whether that the handshake is not done to its satisfaction. The client might decide that it wants to abort and not provide a certificate or something, but most implementations don't really allow that sort of messing around.


The best and easiest plan is to discard, but we know that throwing out perfectly good packets can have bad effects on performance. It sends a bad signal to a congestion controller for instance. So we need to say either discard, or ensure that the data is properly attributed and consistently processed.
I think you covered it all this way. +1.
The conclusion in the draft is that early data is relatively "safe" (for some definition of the term, see the draft for details) if the handshake is complete. Early data in a transport that provides ordered delivery will always arrive before the handshake completes. But this is not the case in QUIC, or anything like DTLS. We should offer some advice for these situations. From martinthomson/http-replay
But can this really happen ? I mean, my understanding is that the handshake happens in response to a Hello message which also conveys early data. Can we have early data outside of a hello message, or may we receive a handshake response before a hello message ?
With reordering of packets in QUIC, you can get a packet that was sent before the handshake completes after the handshake completes. The order of sending might be "ClientHello, 0-RTT1, 0-RTT2, Client Finished", but the server can get "ClientHello, 0-RTT2, Client Finished, 0-RTT1". Early data is sent in separate packets (this is a hard rule right now, but we might change the format so that the packets can be sent together in future). This is unlike TCP fast open where you only get to send the "early data" in the SYN segment. You can send an awful lot of early data if you want.
OK thanks for the explanation. I tend to think that such behaviours are out of the current scope of the draft but will fall into it as QUIC progresses. Maybe all of this can be summarized (for now) as "In case a TLS implementation supports receiving early data after the handshake completes, such early data has to be silently discarded". That covers this case for QUIC and any possible TLS implementation specificity or even bug.
I think that we might add "... discarded unless the data can be unambiguously identified as being early data" or something like that. After all, it's not replayed data if the handshake completes, so I don't want to prohibit its use. The main problem is that it's hard to properly identify and handle. FWIW, we debated this for our DTLS 1.3 implementation and chose to discard. That is the easiest answer.
I'd still be tempted to simply discard for now, just to avoid the problems of confusing the client with an out-of-order handshake that may make it think the "late" early data were lost while in the end they are considered. I'd rather avoid a client sending a request again as regular data and seeing the server concatenate it with the late arrived early data for example. I really think it's something we can improve later as the out-of-order use cases become more obvious. It would also offer us a safety belt allowing more fancy stuff in QUIC, otherwise we could end up saying "nah don't do this or that, it could cause trouble with early data". The rule might then be relaxed. It will not affect TCP-based implementations anyway.


This text from Kyle Rose highlights the decision process. If any node might make a different decision about processing early data, then this node has to wait.
I like it, +1


Resolves URL
cc NAME NAME
Hey Yoav, Thanks; will take a look. Two immediate things: You're getting a error in the markdown; . I see you've added yourself as an author. That's generally the decision of the chair - NAME in this case.
Hopefully fixed. Is there a way to test it locally? Apologies for the noobness. Removed myself.
See URL for build info.
NAME Thank you for working on the proposal. Am I correct in assuming that changes other than the switch to Cuckoo filters and the introduction are unintentional? For example, I see flag of the frame being removed. Do you have a working code that implements Cuckoo filters? I am curious to see it working. The concept of makes sense to me. Maybe we might want to adjust the codepoints and the naming in relation to .
Oh, I now understand the intent of removing the flag. The motive of the proposal is to build a digest without referring to every response object stored in cache. The fact means that it is not be easy for the client to determine the freshness of the entries that is going to be included in the digest. I am sympathetic to the idea, but I am afraid if the approach works well with the current mechanism of HTTP/2 caching. My understanding is that browsers that exist today only consume a pushed response when it fails to find a freshly cached response in its cache. Otherwise, the pushed response never lands in the browser cache. Unless we change the behavior of the browsers to respect the pushed response even if a freshly cached object already exists in its cache, there's a chance that servers would continually push responses that gets ignored by the client (due to the existence of a freshly cached response in the browser cache with the same URL). NAME Assuming that I correctly understand the motive of removing the distinction between a fresh digest and a stale digest, I would appreciate it if you could clarify your ideas on the problem.
Thanks for reviewing, NAME :) My intent was to include all stored resources in the digest, regardless of them being stale or fresh. Entries are added to the digest when a resource is added to the cache and removed from the digest when a resource is removed. The reason is that I think the distinction doesn't make much sense, and maintaining it adds a lot of complexity, basically forcing browsers to recreate the digest for every connection at O(N) cost. Under this premise what servers should do is: Push all the resources that are known not to be in the cache digest Push 304 responses for resources that are in the cache digest, but are likely to be stale (short freshness lifetime, etc) Don't push resources that are in the cache digest and have a long term freshness lifetime or are immutable. Does that make sense? I'm not sure I understand your reference to the push cache vs. the HTTP cache in your comment. In light of my explanation, is there still an issue there in your view?
URL is the reference implementation. Happy to change it. Do you have any specific changes in mind?
NAME Thank you for the explanation. I now understand the intent better. I think that we need to consider two issues regarding the approach. First is the fact that a browser cache may contain more stale responses than fresh resources. Below are the numbers of cached objects found in my Firefox's cache (to be honest the date is from 2016, I haven't been using Firefox in recent weeks and therefore cannot provide up-to-date data). total ---: 2,273 1,003 As you can see, large scale websites tend to have more stale objects than fresh objects. In other words, including information of stale-cached objects increases the size of the digest roughly three times in this case. Since performance-sensitive resources (that we need to push) are likely to be stored fresh (since they are the most likely ones marked as immutable, or near-immutable), transmitting only the digest of freshly-cached responses makes sense. Second is a configuration issue on the server side. One strategy that can be employed by an H2 server (under the current draft) is to receive a digest of freshly cached resources only, compare the digest against the list of resources the browser should preload by only using the URL, and push the missing resources to the client. It is possible for a H2 server to perform the comparison without actually fetching the resource (from origin or from cache) since only the URL would be required for calculating the digest. The proposal prevents such strategy from being deployed since it requires the ETag values to be always taken into consideration (should they be associated to the HTTP responses). In other words, servers would be required to load response headers of the resources to determine if it needs to be pushed, which could be a huge performance degradation on some deployments. Fortunately, servers could avoid the issue by not including ETags for resources that it may push. I think such change on the server-side configuration would be possible, but we need to make sure if we are to take the path (of removing the fresh vs. stale distinction). Let me explain using an example. Consider the following case: client has with and on server-side, the resource has been updated to When receiving a new request from the client, the server cannot determine if the client has URL in its cache. Therefore, would be pushed. The client, when observing (or equivalent tag), tries to load the resource. Since the fresh resource exists within the browser cache, that would be used. The pushed version is ignored and gets discarded (). This would be repeated every time until the cached object either becomes stale or gets removed from the cache. My understanding is that the browser behavior (explained in ) is true for Firefox and also for Chrome. Am I wrong, or missing something? Thank you for the link. I will try to use it. OTOH, do you have some working code that can actually calculate the cache-digest value taking a list of URLs as an input (something like URL)? I ask this because it would give us a better sense in how the actual size of the digest would be. One way to proceed would be to split the discussion of from Cuckoo filters into a separate issue or a PR. I do not have a strong opinion on the naming or the codepoints. What do you think? NAME
NAME Have you considered the approach using Cuckoo filter to generate GCS? I can understand the fact that you do not want to iterate through the browser cache when sending a cache digest. Per-host Cuckoo hash seems like a good solution to the issue. OTOH, as I described in my previous comment, it seems that sending the hash directly has several issues. That is why I am wondering if it would be viable to generate GCS from the per-host Cuckoo filter that would be maintained within the browser. I can see three benefits in the approach, compared to sending the values of Cuckoo filter directly: the size of the digest will be smaller we can keep the distinction between fresh vs. cache. Sending digest of fresh resources only would end up in even smaller digests. Retaining the distinction lowers the bar to deploy cache-digests on the server side. note: you can store the time when the cached object becomes stale in the data associated to the Cuckoo filter entry (assuming that you would have associated data to handle resize, as we discussed in URL). That information can be used when builiding the GCS to determine if a particular object should go into a GCS of fresh resources or that of stale ones less change to the browser push handling (no need to handle pushes of 304 or replace a freshly cached object when an object with the same URL is being pushed) In case of or in the comment above, sending fresh-only digests using GCS would be about 1/3 the size of sending fresh & stale digests using Cuckoo filter. The biggest cost of calculating GCS from Cuckoo hash would be the sort operation. But I think that the cost could be negligible compared to the ECDH operation that we would be doing for every connection, considering the fact that the number of entries that we would need to sort would be small (e.g., up to 1,000 entries of uint32_t), and the fact that sort algorithms faster than O(n log n) radix sort can be deployed (e.g. radix sort). WDYT?
So have a cuckoo filter digest and then put its fingerprints in a GCS? I have not considered that. Need to give it some thought... At the same time, it's not clear to me how that would enable a "stale" vs. "fresh" digests, or handling of improperly cached resources (fresh resources that were replaced on the server).
I would appreciate it if you could consider. To me it seems it's worth giving a thought. Under the approach proposed in this PR, structure that stores the per-host digest would look like below. is required for resizing the filter (e.g., when doubling or halving ). What I am suggesting that you could change the structure to the following. In addition to the hash value, each entry will contain the moment when the entry becomes stale. The moment can be calculated when the entry is added. For example, if the entry represents a HTTP response with a , can be calculated as . If the entry represents an immutable HTTP response, then should be set to a very large value (e.g.. assuming that underlying type of is ). When building a GCS digest, you would do the following: step 1. prepare an empty list that would contain hashes of fresh responses step 2. prepare an empty list that would contain hashes of stale responses step 3. foreach entry in cuckoofilter: step 3-1. check if the entry is fresh or not, by checking the value of step 3-2. if the entry is fresh, append of the entry to the list of the hashes of fresh responses step 3-3. otherwise, append of the entry to the list of the hashes of stale responses step 4. sort the list of hashes of the fresh responses, encode as GCS, and send step 5. sort the list of hashes of the stale responses, encode as GCS, and send You can skip the operations related to stale objects (i.e. step 2, 3-3, 5) if the server is unwilling to receive stale digests. Whether the approach can be implemented depends on if a client can determine the moment a response becomes stale. I anticipate that it is possible to determine that when you register the entry to Cuckoo filters (which is when you receive the response from the server).
NAME NAME I'm planning to attend the IETF 100 hackathon this weekend in Singapore. (First timer here. ) I'm happy to collaborate on a (URL) implementation of this spec if either of you are around and interested. I'm fairly familiar with the current spec, having implemented it and .
NAME Wonderful! I'll be attending the hackathon on both days (i.e. Saturday and Sunday). I do not think that I would have time to work on Cache Digests, but would love to discuss with you (or help, if you need) about your work on Cache Digests.
I've got an incomplete initial reference implementation at URL It doesn't yet include removal and querying (that's what I'll be adding next), but I did run it on a list of ~3250 URL (which I got out of my main profile chrome://cache/) and it seems to be creating reasonable sized digests. One more advantage, the digests seem to be highly compressible when sparse. Results so far: Digest with 1021 entries (so room for ~4K URLs): 5621 in-memory, 5233 gzipped (when filled with 3250 URLs). Digest with 2503 entries (so room for ~10K URLs): 13772 in-memory, 6879 gzipped (same 3250 URLs). Digest with 7919 entries (so room for ~31K URLs): 43560 in-memory, 9984 gzipped (same 3250 URLs). In practice, I think ~1000 entries is most probably enough, but it's good to know we can increase the digest size (to avoid having to recreate it), without significant over-the-wire penalty.
OK, I now have a complete reference implementation and it seems to be working fine. It also exposed an issue with the initial algorithm, forcing table allocation to accommodate a power of 2 number of entries. Latest results for 3250 URLs taken from my cache: One note: the 1021 entries table had 35 collisions, so it seems like it's insufficient for that number of URLs, unless we're willing to absorb extra pushes for ~1% of the resources.
NAME Interesting! It's good to know that we have numbers now. What is the value of P (the false positive ratio) that you used?
P=8 (so 1/256 false positive)
Note that the numbers here may be possible to further optimize. One example is semi-sorting of the buckets which the Cuckoo-Filters paper mentions, and which I have not yet implemented. It adds some runtime complexity, but can reduce the fingerprint size per resource by a full bit, so could have resulted in ~9% smaller digests in this case.
NAME Awesome! Me not being familiar with these data structures (despite reading Wikipedia article ), why does the 1/256 probability (~4/1000) result in 35 collisions?
The 35 collisions are on top of the false positive rates, and represents resources that we failed to put into the table to begin with (due to both their buckets being full). That rate of collisions seems high compared to the results in the paper, so I need to dig further to see who's wrong...
The collisions are now fixed. It was an algorithm issue, where the entry to be pushed was always the same one at the end of the bucket. I've change that to be a random fingerprint from the bucket, which significantly improved things. The ref implementation is now collision free almost up to the point where the digest is full.


mention the syntax update the reference type the citations consistently
1) It should be RFC 5234, not RFC 4234 2) If it's there, it should actually be mentioned somewhere in the spec.


This makes mitigation mandatory. Either receive a signal that early data is safe for the given resource, or apply the mitigations. That's consistent with the other text.
LGTM


I think that these are the main places where we weren't clear enough about the relevant TLS role.
LGTM


John Mattsson pointed out that we didn't really discuss the risks of use of early data. That's fair, but we don't need to. This rewrites the abstract.
Just one fix : s/mechanisms that allows/mechanisms that allow/ Nice overall.


Hopefully none of this collides with other PRs in flight.
I think it's fine with this. Adding a bit of pragmatism as you did in the last commit also reminds that the low cost of retrying is not worth a lot of trouble :-)


Thanks NAME and NAME for pushing on this.
I think this is good and can even emphasize how to consistently disable 0-RTT across a cluster for any reason then consistently re-enable it.


Note this was manually incorporated into draft-ietf-httpbis-rand-access-live-03


Add SETTINGS prefix to parameter name ENABLECONNECT_PROTOCOL. Fix one typo ("s/a entry/an entry/"). Remove "(type = 0x8)" which is ambigous (what is type?) and redundant with code listed in IANA Considerations section.
thanks


Herve, could you please open an issue for this, so that we can assure that we track the underlying issue? It's OK to merge the proposal if the editor feels that it's helpful, but we need to make sure the WG gets consensus here. Thanks,
Linked to issue .


Port over of PR from mnot repo. Modified last sentence to avoid to much usage of the word "further".
Thanks!


Mention obsoletion in the abstract, tune section title in appendix, and actually cite RFC 3205.


Aligns with other SETTINGS parameter names defined in other docs.
NAME icymi
Thank you for the fix!


This adds a section regarding falsification of Tunnel-Protocol. I didn't expand this to note that WebRTC won't suffer from this problem because the application won't be able to set the header field. That seemed like too much detail.


for encryption. I had these in my local repo. They weren't pushed, but I wanted to make sure that they are OK before merging them.


This algorithm automatically enforces the ABNF syntax, including length limits. It still punts on the actual strtoi/strtof part. ?
Much better, thanks!


This just fixes a typo, but this text has more problems. I'm having trouble connecting this final sentence to the text that precedes it. I think that the point is that including a body on GET is unwise because it might cause explosions. A point about side effects needs far more exposition, especially since GET is defined to be free from overt side effects (safe, idempotent, and all that business). If this were just another statement and the "As a result" is dropped, that's different.
Barg, I think that's a copy/paste error, will fix. thx.


For
Allow a response to associate itself with more than one variant. (moved from mnot/I-D)


There is currently a discrepancy in the spec regarding invalid values in the SameSite cookie attribute. For example, this cookie: Set-Cookie: foo=bar; SameSite=bogus is expected to be dropped entirely according to the "Server Requirements" under Section 4.1.2.7: If the "SameSite" attribute's value is neither of these [ "Lax", "Strict" ], the cookie will be ignored. whereas under Section 5.3.7 of "User Agent Requirements", the cookie is to be kept but the attribute is ignored: If cookie-av's attribute-value is not a case-insensitive match for "Strict" or "Lax", ignore the "cookie-av". Additionally, the end of Section 4.1.2 also matches the behavior described in Section 5.3.7: User agents ignore unrecognized cookie attributes (but not the entire cookie). From a forward-compatibility point of view, the behavior described in section 5.3.7 is the ideal one since it allows for future expansion of this feature such as: Set-Cookie: foo=bar; SameSite=medium
NAME
Done. Let me know if I missed anything else.
URL Section 3.1 of the spec states that "SameSite" alone (without an attribute value) is a valid token: URL Section 4.1 URL states If "cookie-av"'s "attribute-value" is not a case-insensitive match for "Strict" or "Lax", ignore the "cookie-av".
Ah. URL draft is outdated vs. URL The latter no longer allows a bare "SameSite" token, and it calls for the cookie to be dropped in the event that the samesite-value isn't either "strict" or "lax". However, I think this text isn't quite right: If 's is not a case-insensitive match for "Strict" or "Lax", ignore the . The in this case is 'SameSite=whatever' and we want to ignore the whole cookie, not just the invalid attribute?
is my proposal to fix this. We ran into this issue in the Firefox implementation: URL
Thanks!


Also considering if "Security Considerations" can be "Compatibility Considerations".
I haven't renamed "Security Considerations" in this version. And I have no issue with calling this "Compatibility Considerations". But I have some recollection that this is a required section...
Yeah don't do that or various people will start to scream at you :)
Call me a perfectionist, but on reread, I realized there was a cleaner/shorter way to clarify. Let me know if you have any objection...
Thanks Martin!


According to RFC 7231, Section 8.3.1. I didn't include anything here about Vary because I wanted to discuss that first and these seemed uncontroversial. It seems OK to say that Vary is permitted here because we expressly talk about varied reactions based on the presence of Early-Data. If that's right, I will amend this with a second commit. (I'm jetlagged, so being a little cautious.)
Your proposal looks very clear to me, despite your jetlag :-) I agree with you that Vary should be OK. I don't even think we need to say anything about it then.


Our basic list of mitigations missed a fairly obvious one. Mentioning it should help with the class of confusion Magnus N. had with the draft. I decided not to include a note about the server being unable to examine early data before making this decision. That's just something people will need to discover for themselves. Generally, you have to decide whether you want 0-RTT without seeing any of it. Partly this is because it avoids a potential deadlock, but mostly it's because the TLS stack will not even decrypt 0-RTT if it is rejected.
Looks good to me, and indeed clearer. Thanks.
+1


This was a little obtuse in the previous iteration, likely as a result of trying to be parsimonious. Expand it a little and clarify the attack.
It's indeed better like this in my opinion.


URL too wide artwork by changing the second example to use multiple field instances (which, in itself, is good to have in the examples anyway)
Thanks!


Last call discussion identified this as a solution to the confusion that exists around the intent of the header field.
Apparently, this is confusing. Picking a new name might help.


They were both special, but for some reason the first reference wasn't getting brackets in the text output. Putting whitespace between them doesn't affect that, but putting other characters does.
LGTM. (And yes, I tried whitespace as well :-)


NAME had a few comments in the review of this that are easy to solve.
All changes are OK to me as well. Thanks!
43d2c16 does a great job capturing the point that I made poorly off-list; thanks again!


Just editorial suggestions here, but all good ones.
Nice wording, I really like this new version. Thanks!


This will conflict with the changes ekr suggested, but I integrated them, so merging shouldn't lose the important changes.
For "sent in early data on another connection", I think Spencer's proposal "sent in early data on a previous hop" would make it even clearer regarding the hop chain so as not to confuse this with another connection from the same user-agent. OK for the rest.


Follow-on to ; in the process of fixing this, I noticed that I'd added a "server" header but missed adding a corresponding "client" header, so the client discussion appeared in the server section.
For some reason, that seems to be endemic in this doc. Let's do that cleanup separately.
I appreciate the addition of the section heading. Though Section Headings Use Title Case.


Per discussions in fetch [1], update the opt-in example to apply to same-origin resources only, with Accept-CH-Lifetime being processed on navigation requests. [1] URL /cc NAME NAME


Step 4.4 equates "the range %x00-1f or %x7f" to "not in VCHAR", but VCHAR doesn't include %x20; this change adds "or SP" and imports SP from RFC5234.
Thanks!


Serialisation algorithms for dictionary and parameterised-list didn't include commas (?) This copies the equivalent steps from the algorithm
Just noticed that this is
Thx.


Makes "Parse a Number from Text" not be greedy, and return (not fail) when it encounters a non-digit character.
I'm pretty sure it works. Steps 5 & 6 ensure there's always at least one digit. All the calling contexts of "Parsing an Item from Text" (including the top-level algorithm in §4.2) inspect the next character, and handle it appropriately.
SGTM
Section 4.2.6 (Parsing a Number from Text) is ambiguous. It makes no mention of the exit condition from the while loop in step 7 on parsing a number from an input buffer.
I don't see it. The loop terminates with "fail parsing" in 7.4 (invalid character), or 7.5/7.6 (too many digits); or terminates successfully by virtue of "input_string is not empty" becoming untrue.
An example might clarify what I find ambiguous. Let us suppose that the input string is "-11-1, 4, 3", and that we are parsing a list. We'd call parseItem(), which would then invoke parseNumber(). When should parseNumber() stop reading in characters? Should it read in "-11" and then leave the rest to the next parsing function, or should it read in "-11-1" and then fail parsing? The other sections are unambiguous as to when to stop reading in characters (for instance, section 4.2.7 step 4.3 states that parsing of a string would stop when you encounter a DQUOTE, and section 4.2.9 step 3 states that parsing of binary content would stop when you encounter an ending asterisk).
Oh I see what you're saying. Yes, step 7.4 shouldn't be so aggressive.


Serialised wasn't getting appended to
Would it be better to just have each of the steps before return their result?
I did wonder that when writing it. Or make them all . Or however you think looks best. I probably should have just filed it as an Issue. :thinking:


When parsing a number, multiply back in to before testing against the range defined in Section 3.5.
Thanks!


How's that? It looks kind of like the equivalent case in Parse Dict/Parse List, but more descriptive than it was.


Closes URL NAME mind taking a look?
(NAME would you mind adding NAME to the repo as a collaborator so I can ask him to review via the GitHub tool? :) )
Ping.
Invite was sent a while back. Still listed as "pending."
It is an internal flag, but I don't think the PR needs additional documentation. The change we're making is in the middle of an algorithm, and is basically a variable that was set earlier in the process. I think folks reading the document will understand that. :)
Thanks for the review! I'll merge this, and look forward to your PR adding yourself as an editor.
RFC 6265 section 5.3 defines a cookie by the name, domain, and path. However, given these two headers in a response from a request to URL Set-Cookie: mycookie=nothostonly; URL Set-Cookie: mycookie=hostonly Most browsers will have two cookies since they include host-only-flag in their definition of a unique cookie. I'd like to update section 4.1.2 and 5.3 to reflect the behavior of modern browsers with regards to host-only-flag.
Discussed in Berlin; seems reasonable.
I probably need to read more of the context but would it be useful to point out that this is an internal flag for the user agent? It is, right? Inferred from whether the setting of the cookie included the domain or not.


For
from the list Maybe we should be talking about a Variant-Key syntax that lets you specify alternative sets of values, e.g.,


Takes a dependency on URL
I think that it would be better to explicitly reference CERTIFICATE_REQUEST in the cases where a CERTIFICATE is generated directly in response to one. Using the "get context" API from exported authenticators is ugly. This is more important now that exported authenticators folds the request into the response. We probably want to use a flag for whether this is spontaneous so that we can avoid reserving a Request-ID for spontaneous CERTIFICATE messages.
I'm a little dubious about this. Having the same value transported in two different fields and then checking that they're equal seems potentially more error-prone than just transporting it in one place, even if that one place is by calling into a library to interpret for you.
For posterity, discussion in Montreal: Nice not to have to dig out the Request-ID. However, if you duplicate it, then you have to compare and check that they're equal, so you're still calling into TLS. There is an existing duplication in the CERTIFICATE_REQUEST, where the Request-ID is carried outside the EA.


but I'd prefer a better recommendation here than "educate your customers."
Suppose a site is hosted on a particular provider (or CDN, equally). The customer flips the DNS entry to point to a new provider. However, the old provider doesn't know that and might continue to serve traffic via ORIGIN + Secondary Certificates for requests that originally reach the provider via other DNS records. It's current industry practice to leave the old provider configured and hot during a transition to ensure the graceful cutover. However, the DNS record change is considered to be sufficent. We might want to recommend that servers periodically ensure the DNS record points to them before serving Secondary Certificates / ORIGIN references. I think this is Security Considerations prose, not protocol elements or requirements, however.


Serialisation of dictionary and param-list currently say instead of This fixes that.
Blah, good catch; thanks.


Suggested elaboration / re-focusing in the introduction to clarify how CH fits into things.
CI failed due to rfc-URL DNS failure; should be transient.
Feedback addressed, I think.
Thanks for working on this! A few comments...Looks great, thanks Mark! Based on recent direction and discussions, the sections that mention "defines initial set of hints" and reference to User-Agent header may need to be updated, but we'll tackle that in a separate thread.


This change passes 'null' to the content-negotiation algorithm if the request header is absent. For list headers, the algorithms can always turn that into the empty list, but for non-list headers they need to come up with something appropriate to do. This also tries to do better about , but may not completely fix it.
Thanks.
URL says "... For each variant in variants-header ... Let request-value be the field-value(s) associated with field-name in incoming-request." I don't see any handling of the case where there is no header in . The algorithms in URL then also don't handle the case where request-value is empty. Sorry if I've just missed where you handle the empty case.


I've also added examples of the new behavior. This is an attempt to
URL shows an example of an origin server that contains English, French, and German versions of a resource, a cache that has cached the English and French versions, a request that asks for German or Spanish, and, I believe, permission in the spec for the cache to return the English version instead of asking the origin server for the German version. This happens because URL appends the first Variant to the list of requested languages even if the list of requested languages overlaps with the available variants. Is this a mistake in the algorithm, or intentional for some reason?
The language algorithm does that so that an origin can denote a 'default' language -- i.e., what to serve if there isn't any overlap between the client's preferences and what's available. For example, if a user configures their browser to send and the page is only available in English and French, there needs to be a way to say "send English by default." I don't know of any use case where someone would want to serve a hard error when the language doesn't match, but if that's in scope, we could figure something out. If you're looking for something more deterministic in caches (here, the distinction between "these ar the responses that are available and preferred" and "here's the default" is blurred), I'm interested. P.S. in that diff, the line with the comment "prefers French, will accept English(?)" seems wrong.
I think establishes the default language without confusing it with the responses that are available and preferred. I am also not interested in a hard error when the language doesn't match. And thanks, I've fixed the example.
I'm not entirely happy with the optionality of behaviour on the cache's part with those MAYs. The most explicit thing we could do would be to break this information out into a separate header (e.g., ), but sticking with the current syntax for the moment, what if we had special values that, when occurring in that first slot, indicate that the default should NOT match? E.g., where does the magic.
I'm personally happy with turning those MAYs into MUSTs. They're MAYs because that's what was in the older RFCs, but I agree that I don't like making things optional. If I do this in my update tomorrow morning, do you still want to explore the token? I don't really like it because it seems like always the wrong thing for a client to send. I also don't see a particular need to pull the defaults out to a separate header.
I'm a bit interested in , but not adamant. I think it'd be fine if they flipped to MUST as long as the implications were noted.


I think this also but I might have missed something. Misc changes that aren't actually necessary to use Structured Headers: "variant-item" became "variant-axis". I defined what clients ("caches"?) MUST do when a server violates one of its MUSTs. I removed the (incorrect) definition of available-value in favor of allowing arbitrary . It wasn't clear what excluding characters like was gaining us, and the restriction would have required extra words when parsing the headers. I removed {{gen-variant-key}} entirely and had Compute Possible Keys return a list of lists that could be directly compared against the Structured Header parse of . It would be difficult to keep the old behavior of returning a list of strings without re-adding some restriction on an available-value, for example by requiring that they don't contain spaces. I simplified Compute Possible Keys a bit.
URL shows a Variant-Key of But that's now This also has implications for the URL algorithm, which currently ignores semicolons.
looks like a parameterised list is currently a list of strings, but if is adopted, its syntax will need to be changed, as it doesn't fit into parameterised list particularly well (although I guess it could).
Currently waiting on resolution of .
NAME is closed now. Any hints?
Hi NAME - getting to it. Are you planning on implementing or using?


This change switches from "URL" to "URI" specifically for two contexts: URI schemes URI syntax (It might be good to also add an explanation somewhere in the Introduction; I can make a proposal if needed)
It is a PR, no?
Weird, it didn't show the UX for PRs before, but does now. shrug
See URL


Tweak/cleanup in light of
NAME or NAME can you push the green button?
LGTM!


and filed by NAME Given , the AUTOMATIC_USE text is entirely superfluous, and given , the correct wording would be "for a stream."
Not sure if this is an issue or not. Emphasis added in the below quote. This seems to have been left out of the change to stream 0 for all frames (URL)
Section 5.4 contains some left over text that refers to the now removed AUTOMATIC_USE flag. Cf. URL


Signed-off-by: Piotr Sikora
This is effectively editorial, since the error restriction was removed in URL, where this type was renamed from to , but the description didn't reflect that change. cc NAME