New package: GetQuandlData
Quandl and tidyverse
Introduction
Quandl is one of the best platforms for finding and downloading financial and economic time series. The collection of free databases is solid and I use it intensively in my research and class material.
But, a couple of things from the native package Quandl
always bothered me:
- Multiple data is always returned in the wide (column oriented) format (why??);
- No local caching of data;
- No control for importing error and status;
- Not easy to work within the
tidyverse
collection of packages
As you suspect, I decided to tackle the problem over the weekend. The result is package GetQuandlData
. This is what it does differently:
- It uses the json api (and not the Quandl native function), so that some metadata is also returned;
- The resulting dataframe is always returned in the long format, even for multiple series;
- Users can set custom names for input series. This is very useful when using along
ggplot
or making tables; - Uses package
memoise
to set a local caching system. This means that the second time you ask for a particular time series, it will grab it from your hard drive (and not the internet); - Always compares the requested dates against dates available in the platform.
Installation
# not in CRAN yet (need to test it further)
#install.packages('GetQuandlData')
# from github
devtools::install_github('msperlin/GetQuandlData')
Example 01 - Inflation in the US
Let’s download and plot information about inflation in the US:
library(GetQuandlData)
library(tidyverse)
my_id <- c('Inflation USA' = 'YALE/SP_CPI')
my_api <- readLines('~/GDrive/98-pass-and-bash/.quandl_api.txt') # you need your own API (get it at https://www.quandl.com/sign-up-modal?defaultModal=showSignUp>)
first_date <- '2005-01-01'
last_date <- Sys.Date()
df <- get_Quandl_series(id_in = my_id,
api_key = my_api,
first_date = first_date,
last_date = last_date,
cache_folder = tempdir())
glimpse(df)
## Rows: 11
## Columns: 4
## $ Year <chr> "2005-12-31", "2006-12-31", "2007-12-31", "2008-12-31", "2…
## $ series_name <chr> "Inflation USA", "Inflation USA", "Inflation USA", "Inflat…
## $ value <dbl> 190.700, 198.300, 202.416, 211.180, 211.143, 216.687, 220.…
## $ id_quandl <chr> "YALE/SP_CPI", "YALE/SP_CPI", "YALE/SP_CPI", "YALE/SP_CPI"…
As you can see, the data is in the long format. Let’s plot it:
library(tidyverse)
p <- ggplot(df, aes(x = Year, y = value/100)) +
geom_col() +
labs(y = 'Inflation (%)',
x = '',
title = 'Inflation in the US') +
scale_y_continuous(labels = scales::percent)
p
Beautiful!
Example 02 - Inflation for many countries
Next, lets have a look into a more realistic case, where we need inflation data for several countries:
First, we need to see what are the available datasets from database RATEINF
:
library(GetQuandlData)
library(tidyverse)
db_id <- 'RATEINF'
my_api <- readLines('~/GDrive/98-pass-and-bash/.quandl_api.txt') # you need your own API
df <- get_database_info(db_id, my_api)
knitr::kable(df)
code | name | description | refreshed_at | from_date | to_date | quandl_code | quandl_db |
---|---|---|---|---|---|---|---|
CPI_ARG | Consumer Price Index - Argentina | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2020-10-10 02:03:32 | 1988-01-31 | 2013-12-31 | RATEINF/CPI_ARG | RATEINF |
CPI_AUS | Consumer Price Index - Australia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1948-09-30 | 2022-12-31 | RATEINF/CPI_AUS | RATEINF |
CPI_CAN | Consumer Price Index - Canada | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1989-01-31 | 2023-01-31 | RATEINF/CPI_CAN | RATEINF |
CPI_CHE | Consumer Price Index - Switzerland | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1983-01-31 | 2023-01-31 | RATEINF/CPI_CHE | RATEINF |
CPI_DEU | Consumer Price Index - Germany | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1991-01-31 | 2023-01-31 | RATEINF/CPI_DEU | RATEINF |
CPI_EUR | Consumer Price Index - Euro Area | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1990-01-31 | 2023-01-31 | RATEINF/CPI_EUR | RATEINF |
CPI_FRA | Consumer Price Index - France | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:37 | 1990-01-31 | 2023-01-31 | RATEINF/CPI_FRA | RATEINF |
CPI_GBR | Consumer Price Index - UK | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1988-01-31 | 2023-01-31 | RATEINF/CPI_GBR | RATEINF |
CPI_ITA | Consumer Price Index - Italy | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 2001-01-31 | 2023-01-31 | RATEINF/CPI_ITA | RATEINF |
CPI_JPN | Consumer Price Index - Japan | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1970-01-31 | 2023-01-31 | RATEINF/CPI_JPN | RATEINF |
CPI_NZL | Consumer Price Index - New Zealand | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1926-03-31 | 2022-12-31 | RATEINF/CPI_NZL | RATEINF |
CPI_RUS | Consumer Price Index - Russia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1992-01-31 | 2022-12-31 | RATEINF/CPI_RUS | RATEINF |
CPI_USA | Consumer Price Index - USA | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1913-01-31 | 2023-01-31 | RATEINF/CPI_USA | RATEINF |
INFLATION_ARG | Inflation YOY - Argentina | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2020-10-10 02:03:33 | 1989-01-31 | 2013-12-31 | RATEINF/INFLATION_ARG | RATEINF |
INFLATION_AUS | Inflation YOY - Australia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1949-03-31 | 2022-12-31 | RATEINF/INFLATION_AUS | RATEINF |
INFLATION_CAN | Inflation YOY - Canada | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1990-01-31 | 2023-01-31 | RATEINF/INFLATION_CAN | RATEINF |
INFLATION_CHE | Inflation YOY - Switzerland | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1984-01-31 | 2023-01-31 | RATEINF/INFLATION_CHE | RATEINF |
INFLATION_DEU | Inflation YOY - Germany | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1992-01-31 | 2023-01-31 | RATEINF/INFLATION_DEU | RATEINF |
INFLATION_EUR | Inflation YOY - Euro Area | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1991-01-31 | 2023-01-31 | RATEINF/INFLATION_EUR | RATEINF |
INFLATION_FRA | Inflation YOY - France | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1991-01-31 | 2023-01-31 | RATEINF/INFLATION_FRA | RATEINF |
INFLATION_GBR | Inflation YOY - UK | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1989-01-31 | 2023-01-31 | RATEINF/INFLATION_GBR | RATEINF |
INFLATION_ITA | Inflation YOY - Italy | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 2002-01-31 | 2023-01-31 | RATEINF/INFLATION_ITA | RATEINF |
INFLATION_JPN | Inflation YOY - Japan | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1971-01-31 | 2023-01-31 | RATEINF/INFLATION_JPN | RATEINF |
INFLATION_NZL | Inflation YOY - New Zealand | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 2001-03-31 | 2022-12-31 | RATEINF/INFLATION_NZL | RATEINF |
INFLATION_RUS | Inflation YOY - Russia | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1996-01-31 | 2022-12-31 | RATEINF/INFLATION_RUS | RATEINF |
INFLATION_USA | Inflation YOY - USA | Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. | 2023-02-25 02:03:38 | 1914-01-31 | 2023-01-31 | RATEINF/INFLATION_USA | RATEINF |
Nice. Now we only need to filter the series with YOY inflation:
idx <- stringr::str_detect(df$name, 'Inflation YOY')
df_series <- df[idx, ]
and grab the data:
my_id <- df_series$quandl_code
names(my_id) <- df_series$name
first_date <- '2010-01-01'
last_date <- Sys.Date()
df_inflation <- get_Quandl_series(id_in = my_id,
api_key = my_api,
first_date = first_date,
last_date = last_date)
glimpse(df_inflation)
## Rows: 1,721
## Columns: 4
## $ series_name <chr> "Inflation YOY - Argentina", "Inflation YOY - Argentina", …
## $ ref_date <date> 2010-01-31, 2010-02-28, 2010-03-31, 2010-04-30, 2010-05-3…
## $ value <dbl> 8.24, 9.12, 9.66, 10.21, 10.66, 11.00, 11.20, 11.10, 11.09…
## $ id_quandl <chr> "RATEINF/INFLATION_ARG", "RATEINF/INFLATION_ARG", "RATEINF…
And, finally, an elegant plot:
p <- ggplot(df_inflation, aes(x = ref_date, y = value/100)) +
geom_col() +
labs(y = 'Inflation (%)',
x = '',
title = 'Inflation in the World',
subtitle = paste0(first_date, ' to ', last_date)) +
scale_y_continuous(labels = scales::percent) +
facet_wrap(~series_name)
p