03-data-analysis.Rmd

---
layout: page
title: Using dplyr for data manipulation and analysis
subtitle: dplyr for data analysis
minutes: 10
---

```{r knitr-options, echo=FALSE, purl=FALSE}
knitr::opts_chunk$set(results='hide', fig.path='img/')
```

> ## Learning Objectives {.objectives}
>
> * Learning objective 1
> * Learning objective 2

```{r check-data, echo=FALSE}
if (!file.exists("data/surveys.csv")) {
    download.file("http://files.figshare.com/1919744/surveys.csv",
                  "data/surveys.csv")
}
if (!file.exists("data/species.csv")) {
    download.file("http://files.figshare.com/1919741/species.csv",
                  "data/species.csv")
}
```

```{r load-data}
surveys <- read.csv(file="data/surveys.csv")
species <- read.csv(file="data/species.csv")
```

# Calculating statistics

```{r, echo=FALSE, purl=FALSE}
## Calculating statistics
```

Let's get a closer look at our data. For instance, we might want to know how
many animals we trapped in each plot, or how many of each species were caught.

To get a vector of all the species, we are going to use the `unique()` function
that tells us the unique values in a given vector:

```{r distinct-species, purl=FALSE}
library(dplyr)
distinct_species <- distinct(select(surveys, species_id))
```

```{r count-species, purl=FALSE}
surveys_by_species <- group_by(surveys, species_id) #%>% filter(!is.na(weight))
species_count <- summarise(surveys_by_species, count=n())
```

```{r}
print(species_count, n=50)
```

R has a lot of built in statistical functions, like `mean()`, `median()`,
`max()`, `min()`. Let's start by calculating the average weight of all the
animals using the function `mean()`:

```{r, results='show', purl=FALSE}
species_stats <- summarise(surveys_by_species,
                           count = n(),
                           n_na = sum(is.na(weight)))
```

Hmm, we just get `NA`. That's because we don't have the weight for every animal
and missing data is recorded as `NA`. By default, all R functions operating on a
vector that contains missing data will return NA. It's a way to make sure that
users know they have missing data, and make a conscious decision on how to deal
with it.

When dealing with simple statistics like the mean, the easiest way to ignore
`NA` (the missing data) is to use `na.rm=TRUE` (`rm` stands for remove):

```{r, results='show', purl=FALSE}
mean(surveys$wgt, na.rm=TRUE)
```


In `dplyr` the `group_by()` function allows you to repeat an operation on groups
of data.

```{r}
library(dplyr)
surveys_by_species <- group_by(surveys, species_id) #%>% filter(!is.na(weight))
species_stats <- summarise(surveys_by_species,
                    count = n(),
                    n_na = sum(is.na(weight)))

                    mean_weight = mean(weight, na.rm=TRUE),
                    sd_weight = sd(weight, na.rm=TRUE)
                    )
```