Skip to content

Commit

Permalink
Check and cut copyrighted FRED-QD series
Browse files Browse the repository at this point in the history
  • Loading branch information
Nikolas Kuschnig committed Jul 10, 2019
1 parent f874fd1 commit e623e92
Show file tree
Hide file tree
Showing 6 changed files with 58 additions and 5 deletions.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: BVAR
Type: Package
Title: Hierarchical Bayesian Vector Autoregression
Version: 0.1.4
Version: 0.1.5
Date: 2019-06-30
Authors@R: c(person("Nikolas", "Kuschnig", role = c("aut", "cre"), email = "nikolas.kuschnig@wu.ac.at"),
person("Lukas", "Vashold", role = "aut"),
Expand Down
6 changes: 4 additions & 2 deletions R/data.R
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@
#' 1959Q1 until 2018Q4.
#'
#' For further details see McCracken and Ng (2016) or the dataset's appendix at
#' \url{https://research.stlouisfed.org/econ/mccracken/fred-databases/}.
#' \url{https://research.stlouisfed.org/econ/mccracken/fred-databases/}. The
#' dataset included is a subset of the full FRED-QD with the 203 of 248
#' variables that are in public domain.
#'
#' @docType data
#'
#' @format A \code{data.frame} with 240 observations of 248 variables.
#' @format A \code{data.frame} with 240 observations of 203 (248) variables.
#'
#' @keywords datasets fred
#'
Expand Down
8 changes: 8 additions & 0 deletions cran-comments.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
# v0.1.5, CRAN Update 1

- Tried to clarify licensing terms with the Federal Reserve
- Some copyrighted series may have to be removed
- Subset the dataset to only include variables in public domain for now

# v0.1.4, JSS Submission

- Fix addition of prior pdfs to ML
Expand All @@ -6,6 +12,8 @@
- Add normalising constant
- Add lines to all density plots (when supplied via ellipsis)
- Add documentation on using `scale_hess` as a vector
- Add two pre-constructed dummy priors `soc` and `sur`
- Further split up calculation of marginal likelihood

# v0.1.3, CRAN Submission 2

Expand Down
Binary file modified data/fred_qd.rda
Binary file not shown.
6 changes: 4 additions & 2 deletions man/fred_qd.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 41 additions & 0 deletions scripts/dl_fred_qd.R
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@


# Get data ----------------------------------------------------------------

# See https://research.stlouisfed.org/econ/mccracken/fred-databases/
link <- "https://s3.amazonaws.com/files.fred.stlouisfed.org/fred-md/quarterly/"
file <- "2019-03.csv"
Expand All @@ -25,3 +28,41 @@ vapply(fred_qd, function(x) sum(is.na(x)), numeric(1))

# Save fred_qd
save(fred_qd, file = "data/fred_qd.rda")


# Get copyright info ------------------------------------------------------

# Some series in the database are under copyright. We are currently waiting
# for the Federal Reserve to provide us with information on allowed usage. In
# the meantime any questionable series are removed.

library(rvest)

data("fred_qd")

rights <- matrix(NA, nrow = ncol(fred_qd), ncol = 2)
colnames(rights) <- c("copyright", "public_domain")

names_url <- gsub("(.*)x", "\\1", names(fred_qd))

for(i in seq_along(fred_qd)) {

site <- paste0("https://fred.stlouisfed.org/series/", names_url[i])

if(RCurl::url.exists(site)) {
site_txt <- site %>%
read_html() %>%
html_text()

rights[i, ] <- c(grepl("copyrighted: [a-zA-Z]+ required", site_txt),
grepl("public domain: citation requested", site_txt))
}
}

# According to FRED (Adrienne Brennecke) the following series are under copyright:
# VXOCLS, NIKKEI225, NASDAQCOM, SP500, UMCSENT, USEPUINDXM, AAA, BAA

# Here we keep the ones explicitly in public domain.
fred_qd <- fred_qd[, which(rights[, "public_domain"])]

save(fred_qd, file = "data/fred_qd.rda")

0 comments on commit e623e92

Please sign in to comment.