New package: GetQuandlData

Quandl and tidyverse

Introduction

Quandl is one of the best platforms for finding and downloading financial and economic time series. The collection of free databases is solid and I use it intensively in my research and class material.

But, a couple of things from the native package Quandl always bothered me:

  • Multiple data is always returned in the wide (column oriented) format (why??);
  • No local caching of data;
  • No control for importing error and status;
  • Not easy to work within the tidyverse collection of packages

As you suspect, I decided to tackle the problem over the weekend. The result is package GetQuandlData. This is what it does differently:

  • It uses the json api (and not the Quandl native function), so that some metadata is also returned;
  • The resulting dataframe is always returned in the long format, even for multiple series;
  • Users can set custom names for input series. This is very useful when using along ggplot or making tables;
  • Uses package memoise to set a local caching system. This means that the second time you ask for a particular time series, it will grab it from your hard drive (and not the internet);
  • Always compares the requested dates against dates available in the platform.

Installation

# not in CRAN yet (need to test it further)
#install.packages('GetQuandlData')

# from github
devtools::install_github('msperlin/GetQuandlData')

Example 01 - Inflation in the US

Let’s download and plot information about inflation in the US:

library(GetQuandlData)
library(tidyverse)

my_id <- c('Inflation USA' = 'YALE/SP_CPI')
my_api <- readLines('~/GDrive/98-pass-and-bash/.quandl_api.txt') # you need your own API (get it at https://www.quandl.com/sign-up-modal?defaultModal=showSignUp>)
first_date <- '2005-01-01'
last_date <- Sys.Date()

df <- get_Quandl_series(id_in = my_id, 
                        api_key = my_api, 
                        first_date = first_date,
                        last_date = last_date, 
                        cache_folder = tempdir())

glimpse(df)
## Rows: 11
## Columns: 4
## $ Year        <chr> "2005-12-31", "2006-12-31", "2007-12-31", "2008-12-31", "2…
## $ series_name <chr> "Inflation USA", "Inflation USA", "Inflation USA", "Inflat…
## $ value       <dbl> 190.700, 198.300, 202.416, 211.180, 211.143, 216.687, 220.…
## $ id_quandl   <chr> "YALE/SP_CPI", "YALE/SP_CPI", "YALE/SP_CPI", "YALE/SP_CPI"…

As you can see, the data is in the long format. Let’s plot it:

library(tidyverse)

p <- ggplot(df, aes(x = Year, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the US') + 
  scale_y_continuous(labels = scales::percent)

p

Beautiful!

Example 02 - Inflation for many countries

Next, lets have a look into a more realistic case, where we need inflation data for several countries:

First, we need to see what are the available datasets from database RATEINF:

library(GetQuandlData)
library(tidyverse)

db_id <- 'RATEINF'
my_api <- readLines('~/GDrive/98-pass-and-bash/.quandl_api.txt') # you need your own API

df <- get_database_info(db_id, my_api)

knitr::kable(df)
code name description refreshed_at from_date to_date quandl_code quandl_db
CPI_ARG Consumer Price Index - Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2020-10-10 02:03:32 1988-01-31 2013-12-31 RATEINF/CPI_ARG RATEINF
CPI_AUS Consumer Price Index - Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1948-09-30 2022-12-31 RATEINF/CPI_AUS RATEINF
CPI_CAN Consumer Price Index - Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1989-01-31 2023-01-31 RATEINF/CPI_CAN RATEINF
CPI_CHE Consumer Price Index - Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1983-01-31 2023-01-31 RATEINF/CPI_CHE RATEINF
CPI_DEU Consumer Price Index - Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1991-01-31 2023-01-31 RATEINF/CPI_DEU RATEINF
CPI_EUR Consumer Price Index - Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1990-01-31 2023-01-31 RATEINF/CPI_EUR RATEINF
CPI_FRA Consumer Price Index - France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:37 1990-01-31 2023-01-31 RATEINF/CPI_FRA RATEINF
CPI_GBR Consumer Price Index - UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1988-01-31 2023-01-31 RATEINF/CPI_GBR RATEINF
CPI_ITA Consumer Price Index - Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 2001-01-31 2023-01-31 RATEINF/CPI_ITA RATEINF
CPI_JPN Consumer Price Index - Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1970-01-31 2023-01-31 RATEINF/CPI_JPN RATEINF
CPI_NZL Consumer Price Index - New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1926-03-31 2022-12-31 RATEINF/CPI_NZL RATEINF
CPI_RUS Consumer Price Index - Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1992-01-31 2022-12-31 RATEINF/CPI_RUS RATEINF
CPI_USA Consumer Price Index - USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1913-01-31 2023-01-31 RATEINF/CPI_USA RATEINF
INFLATION_ARG Inflation YOY - Argentina Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2020-10-10 02:03:33 1989-01-31 2013-12-31 RATEINF/INFLATION_ARG RATEINF
INFLATION_AUS Inflation YOY - Australia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1949-03-31 2022-12-31 RATEINF/INFLATION_AUS RATEINF
INFLATION_CAN Inflation YOY - Canada Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1990-01-31 2023-01-31 RATEINF/INFLATION_CAN RATEINF
INFLATION_CHE Inflation YOY - Switzerland Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1984-01-31 2023-01-31 RATEINF/INFLATION_CHE RATEINF
INFLATION_DEU Inflation YOY - Germany Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1992-01-31 2023-01-31 RATEINF/INFLATION_DEU RATEINF
INFLATION_EUR Inflation YOY - Euro Area Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1991-01-31 2023-01-31 RATEINF/INFLATION_EUR RATEINF
INFLATION_FRA Inflation YOY - France Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1991-01-31 2023-01-31 RATEINF/INFLATION_FRA RATEINF
INFLATION_GBR Inflation YOY - UK Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1989-01-31 2023-01-31 RATEINF/INFLATION_GBR RATEINF
INFLATION_ITA Inflation YOY - Italy Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 2002-01-31 2023-01-31 RATEINF/INFLATION_ITA RATEINF
INFLATION_JPN Inflation YOY - Japan Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1971-01-31 2023-01-31 RATEINF/INFLATION_JPN RATEINF
INFLATION_NZL Inflation YOY - New Zealand Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 2001-03-31 2022-12-31 RATEINF/INFLATION_NZL RATEINF
INFLATION_RUS Inflation YOY - Russia Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1996-01-31 2022-12-31 RATEINF/INFLATION_RUS RATEINF
INFLATION_USA Inflation YOY - USA Please visit http://www.rateinflation.com/inflation-information/calculate-inflation for more information. 2023-02-25 02:03:38 1914-01-31 2023-01-31 RATEINF/INFLATION_USA RATEINF

Nice. Now we only need to filter the series with YOY inflation:

idx <- stringr::str_detect(df$name, 'Inflation YOY')
df_series <- df[idx, ]

and grab the data:

my_id <- df_series$quandl_code
names(my_id) <- df_series$name
first_date <- '2010-01-01'
last_date <- Sys.Date()

df_inflation <- get_Quandl_series(id_in = my_id, 
                                  api_key = my_api,
                                  first_date = first_date,
                                  last_date = last_date)

glimpse(df_inflation)
## Rows: 1,721
## Columns: 4
## $ series_name <chr> "Inflation YOY - Argentina", "Inflation YOY - Argentina", …
## $ ref_date    <date> 2010-01-31, 2010-02-28, 2010-03-31, 2010-04-30, 2010-05-3…
## $ value       <dbl> 8.24, 9.12, 9.66, 10.21, 10.66, 11.00, 11.20, 11.10, 11.09…
## $ id_quandl   <chr> "RATEINF/INFLATION_ARG", "RATEINF/INFLATION_ARG", "RATEINF…

And, finally, an elegant plot:

p <- ggplot(df_inflation, aes(x = ref_date, y = value/100)) + 
  geom_col() + 
  labs(y = 'Inflation (%)', 
       x = '',
       title = 'Inflation in the World',
       subtitle = paste0(first_date, ' to ', last_date)) + 
  scale_y_continuous(labels = scales::percent) + 
  facet_wrap(~series_name)

p

Marcelo S. Perlin
Marcelo S. Perlin
Associate Professor

My research interests include data analysis, finance and cientometrics.

comments powered by Disqus