Remove the default
cainfo option on Windows. Providing a CA bundle is not needed anymore because
curl now uses the native schannel SSL backend. For recent versions of libcurl, overriding the CA bundle actually breaks custom trust certificates on corporate networks. (@jeroen, #603)
OAuth2.0 has been made somewhat more flexible in order to support more websites:
init_oauth2.0()) gain a new
query_authorize_extra parameter make it possible to add extra query parameters to the authorization URL. This is needed some APIs (e.g. fitbit) (@cosmomeese, #503).
OAuth 2.0 token refresh gives a more informative error if it fails (#516).
Prior to token retrieval from on-disk cache, scopes are de-duplicated, sorted, and stripped of names before being hashed. This eliminates a source of hash mismatch that causes new tokens to be requested, even when existing tokens had the necessary scope. (@jennybc, #495)
Updates to demos:
cache_info() now handles un-named flags, as illustrated by “private” when the server returns “private, max-age = 0”.
Encoding falls back to UTF-8 if not supplied and content-type parsing fails (#500).
pause_min allows for sub-second delays. (Use with caution! Generally the default is preferred.) (@r2evans)
If the server returns HTTP status code 429 and specifies a
retry-after value, that value will now be used instead of exponential backoff with jitter, unless it’s smaller than
pause_min. (@nielsoledam, #472)
New oauth cache files are always added to
.gitignore and, if it exists,
.Rbuildignore. Specifically, this now happens when option
httr_oauth_cache = TRUE or user specifies cache file name explicitly. (@jennybc, #436)
oauth_app() allows you to specify the
redirect_url if you need to customise it.
oauth2.0_token() gains three new arguments:
credentials argument that allows you to customise the auth flow. For advanced used only (#457)
BROWSER() prints a message telling you to browse to the URL if called in a non-interactive session.
encode = "raw" allows you to do your own encoding for requests with bodies.
http_type() returns the content/mime type of a request, sans parameters.
No longer uses use custom requests for standard
POST requests (#356,
The cross-session OAuth cache is now created with permission 0600, and should give a better error if it can’t be created (#365).
use_basic_auth option is used to obtain a token, token refreshes will now use basic authentication too.
Suppress unhelpful “No encoding supplied: defaulting to UTF-8.” when printing a response (#327).
All auto parser functions now have consistent arguments. This fixes problem where
... is pass on to another function (#330).
Fix in readfunction to close connection when done.
warn_for_status() and (new)
message argument with new
task argument that optionally describes the current task. This allows API wrappers to provide more informative error messages on failure (#277, #302).
warn_for_status() return the response if there were no errors. This makes them easier to use in pipelines (#278).
httr no longer bundles
cacert.pem, and instead it relies on the bundle in openssl. This bundle is only used a last-resort on windows with R <3.2.0.
Switch to ‘openssl’ package for hashing, hmac, signatures, and base64.
content(x) uses xml2 for XML documents and readr for csv and tsv.
content(, type = "text") defaults to UTF-8 encoding if not otherwise specified.
httr no longer uses the RCurl package. Instead it uses the curl package, a modern binding to libcurl written by Jeroen Ooms (#172). This should make httr more reliable and prevent the “easy handle already used in multi handle” error. This change shouldn’t affect any code that uses httr - all the changes have happened behind the scenes.
oauth_listener can now listen on a custom IP address and port (the previously hardwired ip:port of
127.0.0.1:1410 is now just the default). This permits authentication to work under other settings, such as inside docker containers (which require localhost uses
0.0.0.0 instead). To configure, set the system environmental variables
HTTR_PORT respectively (@cboettig, #211).
cookies argument to
handle() is deprecated - cookies are always turned on by default.
brew_dr() has been renamed to
httr_dr() - that’s what it should’ve been in the first place!
context(type = "auto") uses a better strategy for text based formats (#209). This should allow the
encoding argument to work more reliably.
CURL_CA_BUNDLE environment variable to look for cert bundle on Windows (#223).
safe_callback() is deprecated - it’s no longer needed with curl.
proxy() gains an
auth argument which allows you to pick the type of http authentication used by the proxy (#216).
encode arguments so you can generate arbitrary requests with a body.
tumblr added as an
Improved LinkedIn OAuth demo (#173).
Uses R6 instead of RC. This makes it possible to extend the OAuth classes from outside of httr (#113).
Now only set
capath on Windows - system defaults on linux and mac ox seem to be adequate (and in some cases better). I’ve added a couple of tests to ensure that this continues to work in the future.
brew_dr() checks for common problems. Currently checks if your libCurl uses NSS. This is unlikely to work so it gives you some advice on how to fix the problem (thanks to @eddelbuettel for debugging this problem).
Content-Type set to title case to avoid errors in servers which do not correctly implement case insensitivity in header names. (#142, #146) thanks to Håkon Malmedal (@hmalmedal) and Jim Hester (@jimhester).
Correctly parse http status when it only contains two components (#162).
Correctly parse http headers when field name is followed by any amount (including none) of white space.
Default “Accepts” header set to
application/json, text/xml, application/xml, */*: this should slightly increase the likelihood of getting xml back.
application/xml is correctly converted to text before being parsed to
safe_callback() function operator that makes R functions safe for use as RCurl callbacks (#144).
Default to out-of-band credential exchange when
httpuv isn’t installed. (#168)
Default accept header is now “application/json, text/xml, /” - this should encourage servers to send json or xml if they know how.
POST() now specifies Curl options more precisely so that Curl know’s that you’re doing a POST and can respond appropriately to redirects.
parse_http_date() parses http dates according RFC2616 spec.
Requests now print the time they were made.
application/xml is automatically parsed with `
Now possible to specify both handle and url when making a request.
headers() is now a generic with a method for response objects.
parse_media() failed to take into account that media types are case-insenstive - this lead to bad re-encoding for content-types like “text/html; Charset=UTF-8”
Remove redundant arguments
simplifyMatrix for json parser.
PATCH() now use
encode argument to determine how list inputs are encoded. Valid values are “multiple”, “form” or “json”. The
multipart argument is now deprecated (#103). You can stream a single file from disk with
upload_file("path/"). The mime type will be guessed from the extension, or can be supplied explicitly as the second argument to
verbose() now uses a custom debug function so that you can see exactly what data is sent to the server. Arguments control exactly what is included, and the defaults have been selected to be more helpful for the most common cases (#102).
quickstart vignette to help you get up and running with httr.
api-packages vignette describes how best practices to follow when writing R packages that wrap web APIs.
httr_options() lists all known config options, translating between their short R names and the full libcurl names. The
curl_doc() helper function allows you to jump directly to the online documentation for an option.
authenticate() now defaults to
type = "basic" which is pretty much the only type of authentication anyone uses.
cacert.pem to version at 2014-04-22 (#114).
query parameters are now dropped automatically.
print()ing a response, httr will only attempt to print the first few lines if it’s a text format (i.e. either the main type is text or is application/json). It will also truncate each line so that it fits on screen - this should hopefully make it easier to see a little bit of the content, without filling the screen with gibberish.
new_bin() has been removed: it’s easier to see what’s going on in examples with
OAuth 2.0 has recieved a major overhaul in this version. The authentication dance now works in more environments (including RStudio), and is generally a little faster. When working on a remote server, or if R’s internet connection is constrained in other ways, you can now use out-of-band authentication, copying and pasting from any browser to your R session. OAuth tokens from endpoints that regularly expire access tokens can now be refreshed, and will be refresh automatically on authentication failure.
httr now uses project (working directory) based caching: every time you create or refresh a token, a copy of the credentials will be saved in
.httr-oauth. You can override this default for individual tokens with the
cache parameter, or globally with the
httr_oauth_cache option. Supply either a logical vector (
TRUE = always cache,
FALSE = never cache,
NA = ask), or a string (the path to the cache file).
You should NOT include this cache file in source code control - if you do, delete it, and reset your access token through the corresponding web interface. To help, httr will automatically add appropriate entries to
These changes mean that you should only ever have to authenticate once per project, and you can authenticate from any environment in which you can run R. A big thanks go to Craig Citro (@craigcitro) from google, who contributed much code and many ideas to make this possible.
The OAuth token objects are now reference classes, which mean they can be updated in place, such as when an access token expires and needs to be refreshed. You can manually refresh by calling
$refresh() on the object. You can force reinitialisation (to do the complete dance from scratch) by calling
$reinit(force = TRUE).
If a signed OAuth2 request fails with a 401 and the credentials have a
refresh_token, then the OAuth token will be automatically refreshed (#74).
OAuth tokens are cached locally in a file called
.httr-oauth (unless you opt out). This file should not be included in source code control, and httr will automatically add to
.Rbuildignore. The caching policy is described in more detail in the help for the
The OAuth2 dance can now be performed without running a local webserver (#33, thanks to @craigcitro). To make that the default, set
options(httr_oob_default = TRUE). This is useful when running R remotely.
OAuth endpoints can store arbitrary extra urls.
Use the httpuv webserver for the OAuth dance instead of the built-in httpd server (#32, thanks to @jdeboer). This makes the dance work in Rstudio, and also seems a little faster. Rook is no longer required.
oauth_endpoints() includes some popular OAuth endpoints.
The placement of
PUT() has been tweaked so that you must always specify
multipart arguments with their full name. This has always been recommended practice; now it is enforced.
httr includes its own copy of
cacert.pem, which is more recent than the version included in RCurl (#67).
Added default user agent which includes versions of Curl, RCurl and httr.
Switched to jsonlite from rjson.
Content parsers no longer load packages on to search path.
httr now imports the methods package so that it works when called with Rscript.
New automatic parsers for mime types
If you supply multiple headers of the same name, the value of the most recently set header will always be used.
Urls with missing query param values (e.g.
http://x.com/?q=) are now parsed correctly (#27). The names of query params are now also escaped and unescaped correctly when parsing and building urls.
Default html parser is now
XML::htmlParse() which is easier to use with xpath (#66).
OAuth now uses custom escaping function which is guaranteed to work on all platforms (Fixes #21)
When concatenating configs, concatenate all the headers. (Fixes #19)
hmac_sha1 since so many authentication protocols need this
content will automatically guess what type of output (parsed, text or raw) based on the content-type header. It also automatically converts text content to UTF-8 (using the charset in the media type) and can guess at mime type from extension if server doesn’t supply one. Media type and encoding can be overridden with the
encoding arguments respectively.
response objects automatically print content type to aid debugging.
text_content has become
context(, "text") and
content(, "parsed"). The previous calls are deprecated and will be removed in a future version.
oauth_listener, use existing httpd port if help server has already been started. This allows the ouath authentication dance to work if you’re in RStudio. (Fixes #15).
add several functions related to checking the status of an http request. Those are :
url_success as well as
build_url: correctly add params back into full url.