Pooshit.Http 0.12.1-preview

• .NETStandard 2.0

  • Pooshit.Json (>= 0.3.40-preview)

• net8.0

  • Pooshit.Json (>= 0.3.40-preview)

Credential-bearing headers are now dropped when a followed redirect crosses origin. This is the change most likely to alter what a working caller observes, and the strip itself is never announced. With FollowRedirects enabled, the resolved target of the hop is compared against the request that produced the redirect: if scheme, host and port do not all match - RFC 6454 origin, so a subdomain is a different origin and a downgrade from https to http is one too - then every header whose name is in the sensitive set is left off the hop. A call that used to succeed can therefore now fail at the hop with a 401, and nothing reports the strip itself: a typed call surfaces it as the ordinary HttpServiceException any 401 would raise, and a call asking for the raw response returns the 401 for the caller to inspect. Neither distinguishes a credential that was deliberately withheld from one the target rejected, so the cause has to be inferred from the redirect having crossed origin. A target whose origin cannot be proven is treated as cross-origin and strips, which covers a response that carries no stamped request uri. A relative Location is mostly not such a case: it resolves against the url of the request that produced it, so a path-relative Location such as /target or ../target, and a bare query string, all stay in origin and keep their headers. The exception is a protocol-relative //host/path Location - relative, because it carries no scheme, but it supplies its own host, so it resolves off-origin and strips like any other cross-origin target. The comparison is made against the final url, after UrlProcessor and after uri combining, so what is judged is what the hop actually sends, and once a header has been stripped it is never restored. This is what browsers and curl -L have always done, and the reason is that Location is a value the remote server chooses: without the strip, a compromised or merely careless upstream can harvest a caller's bearer token by answering with a redirect to a host it names. Two escape hatches remain for a caller who genuinely needs a credential to cross a host boundary. Rewrite the target from UrlProcessor, which sees the resolved url before the hop is issued, so the hop goes to a host the caller named rather than one the server named. Or turn FollowRedirects off, take the raw reply with Get<HttpResponseMessage>, and issue the second call by hand carrying whatever headers it should carry. Unaffected: same-origin redirects; callers who never set FollowRedirects; and cross-origin redirects that carry no credential header to begin with.

HttpServiceException.Message no longer contains the response body. The body is unchanged on HttpServiceException.Body, where it always was, and a caller who logged the message and relied on seeing the body logs .Body instead. There is no option and no way to put it back, and that is the point: an arbitrary vendor payload is something only the application can judge safe to write to a log, so the library stops making that judgement on its behalf. Unaffected: .Body itself, including that it is still null when the response carried no body; the url, the status and the header block in the message; every call that does not fail.

Query parameter values whose name carries a credential word are replaced by <redacted> in the message. The parameter name survives and so does every other parameter's value, so two failing calls to the same endpoint still read differently in a log. The name set is HttpService.SensitiveQueryParameters, open for a consumer to extend or reduce - configure it before the service issues its first request, as with SensitiveHeaders - and it ships with token, key, secret, password, signature, sig, auth and credential. A name is sensitive when it contains an entry anywhere inside it, compared ignoring case, with no word splitting and no anchor at either end, so access_token, X-Amz-Signature, X-Amz-Security-Token, AWSAccessKeyId, apiKey, accesskey, secretkey, authtoken, apitoken and apikey2 are all covered without being listed. Add the word, not a parameter name: sig covers x_vendor_sig and X-Amz-Signature alike, whereas an entry spelled x_vendor_sig matches only names containing that whole string. signature is redundant while sig is present - every name containing signature also contains sig - and ships anyway because sig is the entry most likely to be removed, since it also matches design and assignee: with signature in the set that removal keeps X-Amz-Signature, X-Goog-Signature and every other long-form spelling redacted. It does not keep a bare ?sig= redacted, which is how Azure Storage SAS spells it. An entry only ever covers names at least as long as itself, so the only entries that reach a bare sig are substrings of it - si, ig, g - and every one of those matches so much ordinary traffic (si alone hits design, assignee, version and session) that it costs more than what you removed. If you remove sig and need Azure SAS covered, put it back and live with design being redacted, or add the specific names your own upstream uses. This set is separate from SensitiveHeaders and neither affects the other - SensitiveHeaders also governs which headers are dropped on a cross-origin redirect, and a word added for query redaction must not change what travels to a redirect target. The rule is deliberately biased toward redacting too much, because too little is silent: a parameter named partition_key, sort_key, idempotency_key, sortkey, monkey or keyword now renders <redacted> through key, author and authority through auth, and assignee, design and resign through sig. That costs a diagnostic and you can see it happen; the knob is coarse, since removing key to read keyword also stops secretkey and AWSAccessKeyId being redacted. The direction that carries the real risk is the other one, and it is invisible: a credential whose parameter name contains none of the eight words is printed in full, and nothing in the log says so - a vendor spelling such as x_vendor_nonce or a name that percent-encodes a reserved character inside the credential word itself, access_to/ken, both escape, and the remedy is to add the spelling before the service issues its first request. session and code are deliberately absent - both are far more often ordinary identifiers than credentials. Unaffected: the url's scheme, host and path; a url with no query; the status; HttpServiceException.Body and HttpServiceException.Response, from which Response.RequestMessage.RequestUri still yields the real url for a caller who wants it. Two limits worth naming: a url that appears inside a header value is dumped as part of the header block rather than by this, and is covered only for Location and Referer, by the change described next - a url in any other header, Link for instance, is still dumped verbatim. And a query written with the long-deprecated semicolon separator, ?a=1;access_token=x, is read as a single parameter named a and is not redacted either.

A url carried inside a Location or Referer header value now has its query redacted in the message by the same rule as the request url. The header name survives, and so do the url's scheme, host and path - only the value of a query parameter whose name carries a credential word is replaced by <redacted>, so a Location pointing at a pre-signed object-store target still reads as a target, without its signature. The word set is the same HttpService.SensitiveQueryParameters described above, which means a word added there now changes header rendering as well as url rendering. The header names themselves are not configurable and there is no set to extend: which fields hold a url is fixed by the http specification rather than by a consumer, so it is a private list of two. Within the core http specification Location, Content-Location and Referer are the only fields whose entire value is a single uri reference, and of those Content-Location is excluded anyway because it is a content header: it lives on Response.Content.Headers, which the dump does not walk, so an entry for it could never fire. Outside the core specification WebDAV's Destination also qualifies and is also left out, naming a target resource on the server already being addressed rather than carrying a credential. Redaction is applied to each header value before they are joined, so a header carrying two urls cannot hide a credential in the second one behind the first one's query. Only HeaderDumpMode.Redacted redacts - Full still dumps every header value verbatim, which is what it is for, and Omitted still drops the block whole. Link is deliberately excluded and its value is still dumped verbatim, credentials included. Its RFC 8288 syntax is a comma-separated list of <url>; rel=... entries rather than a bare url, and running such a value through a redactor that reads one query span from the first question mark onwards leaves a credential in the second url legible while looking as though the header had been handled; a paginated api emitting rel="next" and rel="prev" together hits exactly that. Refresh is excluded on the same rule, its value being a delay and a url rather than a url. A consumer who needs a vendor header carrying a callback url protected should put its name in SensitiveHeaders, which replaces the whole value - note that this also drops the header from a cross-origin redirect hop. Unaffected: every header outside the two names; the request url, which was already redacted; the status; HttpServiceException.Body and HttpServiceException.Response, from which the real header values are still available.

A response carrying an unrecognised or absent Content-Type no longer throws InvalidCastException. The JSON family is now matched by structure instead of by exact string - application/json, text/json and any subtype carrying a +json suffix such as application/hal+json or application/vnd.acme.thing+json, all compared ignoring case - and any media type still unrecognised, including none at all, has its body inspected: a payload whose first meaningful character is { or [ is decoded with the configured decoder, and anything else raises HttpServiceException naming the request url, the received media type and the requested type, carrying the raw body on Body. Some paths escape that promise. A Content-Type declaring a character set the runtime does not know raises InvalidOperationException from the string read before the fallback is reached. An empty body does not reach the fallback at all: it returns the default value, unchanged from before and the same under a recognised media type. Nor does a request for string, byte[] or Stream, which hands the body back directly. This is what lets a server answering JSON under a media type it does not declare as JSON be read at all; Meta's Graph API answering text/javascript when no Accept header is sent is the case that prompted it. Caveat, and the one place this fix takes something away: asking for object, MemoryStream or IDisposable against an unrecognised media type used to hand back the undecoded body stream and no longer does - object yields the decoded value, MemoryStream yields an empty stream, and IDisposable raises HttpServiceException. A caller who wants the undecoded body should ask for Stream or byte[], both unaffected and always the right way to ask. Unchanged by that fix: every response whose media type already matched a branch - application/json, application/xml, text/xml, text/plain - and the raw Get<HttpResponseMessage> escape hatch. What the xml and text/plain branches do when the requested type cannot receive what they produce is changed by the item described next.

A response under an explicitly declared media type whose body cannot become the requested type now raises HttpServiceException instead of InvalidCastException. Two branches read an explicit header and produce a fixed type - application/xml and text/xml load an XDocument, text/plain reads the body as a string - and both used to cast that value to the requested type unchecked, so a caller asking for a domain type from a server answering text/plain got an exception naming a string-to-domain-type cast and nothing about the url, the media type, or what had been asked for. The value is now offered only to a requested type that can receive it, which is exactly what the cast itself tested, so every request that succeeded before still succeeds and yields the same value: any requested type the produced value can be assigned to, base class or interface alike, on either branch. That includes object, which yields the XDocument under xml and the body string under text/plain. Only requests that were already guaranteed to fail behave differently, and they now raise the descriptive HttpServiceException the unrecognised-media-type fallback already raised, naming the request url with its query redacted, the media type as received, and the requested type, and carrying the raw body on Body. One consequence beyond the exception type: the refusal leaves the response undisposed, matching the fallback and the error-status path, so a caller who catches it can read Response and owns disposing it. Deliberately not decided here, and therefore unchanged: a JSON body declared as text/plain is refused rather than decoded, the xml branches still consult no configured decoder, and the xml document is still loaded synchronously. Unaffected: application/json and the JSON family, the unrecognised-media-type fallback, the string, byte[], Stream and Get<HttpResponseMessage> escape hatches, and every explicit-media-type request that already succeeded.

HttpServiceException.Message no longer dumps credential-bearing header values. The header name survives and only the value is replaced, so the fact that an Authorization header was set stays visible while the token does not, and an exception logged by the usual reflex no longer writes a credential to the log. The policy is configurable per service through HttpService.HeaderDumpMode - Redacted, which is the new default; Omitted, which drops the header block entirely; or Full, which is the previous behaviour and now has to be asked for - and per call through HttpOptions.HeaderDumpMode, which falls back to the service when unset. The name set is HttpService.SensitiveHeaders, compared ignoring case and open for a consumer to extend or reduce - configure it before the service issues its first request, because it is read on every request and every dump and is not safe to mutate once the service is in use - and it ships with Authorization, Proxy-Authorization, Cookie, Set-Cookie, Api-Key, X-Api-Key, X-Auth-Token and X-Access-Token. WWW-Authenticate and Proxy-Authenticate are deliberately absent: a challenge header carries no client credential and is the most useful thing in a 401 dump. Note that this same name set also governs the cross-origin strip described above - one list, two behaviours - so a name added to it is both redacted out of exception messages and dropped from a cross-origin redirect hop, and a name removed from it loses both protections. Url, status and HttpServiceException.Body are unaffected, as is HttpServiceException.Response: a caller can still walk Response.RequestMessage.Headers, which is explicit access rather than accidental logging.

A followed redirect now inherits the headers of the request that produced it, instead of rebuilding them from the options bag. This is mostly a fix, and it earns a line because it changes what a Send<TResponse> caller observes: the hop used to carry the options bag's headers and none of the caller's own, so an Authorization, Accept or correlation id set on the pre-built request was silently dropped on the hop while the bag's headers - which the original request never carried - were added in their place. The hop now carries what the previous request carried, less the headers that describe a body the redirect does not have - Expect and Transfer-Encoding - and, when the hop leaves the origin, less every name in the sensitive set, which drops the caller's Authorization and Cookie along with it. For a url-based overload whose hop stays in origin the hop is wire-identical, with one observable difference: ITokenProvider.GetTokenAsync is now called once per call rather than once per hop, so a provider that mints single-use tokens will see the first one reused on the hop rather than a fresh one minted. A url-based hop that leaves the origin additionally goes out with no credential at all, where the old code sent a freshly minted one - the first item's change reaches this call shape too, and for most callers it matters more than the token count.