requesting additional info for permute::how() in split-plot design

[katharinehogan]

Thanks for starting this discussion - having people post questions about packages on the repository itself instead of more generic venues like Stack Overflow or less efficient methods like email seems like a much better way for people to hopefully get questions answered.

I'm Katharine Hogan, a PhD candidate in natural resources studying prairie restoration community dynamics. I am struggling with correctly designing the permutation structure on a vegan::adonis2() analysis of a prairie restoration experiment. I'm still unsure after reading the permute::how() documentation and scouring the web for tutorials/examples, and am getting some model results that make me think I'm not accounting for the structure of the experiment correctly.

A very brief overview of the experimental design: It's a 24-ac study, with 24 ca. 1 ac "plots" (6 data measurement points per plot, but that's not crucial to my question). The experiment is a split-plot design with seeding treatments (3-factor) nested within haying management treatments (2-factor) for 6 unique treatment groups (4 reps each for 4x6 = 24 plots). It also has a repeated measures element because I sampled biweekly from late spring to early fall for 3 years (mean of 6 times per year). The 3 seeding treatments (control, low diversity, and high diversity) are randomized within 8 spatial "blocks" of 3 plots each. The haying treatment had to be applied at the block level (4 hayed blocks and 4 unhayed blocks, within which seeding treatments are spatially random), so there is some potential confounding (I believe statistically between management and block) if I don't account for it correctly. I am attaching a schematic my co-advisor made of the experiment for the visual thinkers out there.
plotsGREXHayed2019.pdf

I think my main point of confusion with designing the permutation structure is just terminology. I have been using one set of terms to refer to the different levels of my experiment for years, and am unsure they are analagous to how() - in particular, given my general reading on split-plot experiments, I'm not sure the implicit definitions of blocks and plots in how() are the same as I am used to using. I am also confused why there is a within-plot argument in how(), as I can't think of a case when that would be different from what's already covered in plots (that may just be lack of imagination on my part). Would someone be able to provide a more detailed definition of those terms and how they pertain to the different parts of a split-plot experiment? I can provide more details if helpful. Thanks!

[gavinsimpson]

Can you explain the terminology on the PDF map/diagram you linked to. I only see HAY on 4 plots, not 4 blocks as suggested above. What does the M mean (e.g. in P4 B2 M)?

[katharinehogan]

plotsGREX.pdf
I forgot about the M notation on this particular map - they stand for management (i.e. the haying) though I'm not sure why they were labelled that way. So the "hay" and M notation together show the hayed blocks, but to make it easier to see I've attached an updated image with shaded boxes over the 4 hayed blocks (12 1-ac plots total).

[gavinsimpson]

(You should use dbrda() for this not adonis() or adonis2() — these are the same thing internally but with dbrda() you get more features and you could need Condition() in the formula for the model from what I can see and you can't do that in adonis() etc)

I need to think a little more about this (I'm rusty on some of the details of design-based permutations as I don't really use this stuff now in my research), but…

…in {permute}, blocks and plots can be used for the same thing, to restrict permutations to be within-block (within-plot), i.e. we aren't permuting samples between blocks or plots. The difference between them is that

• blocks are never permuted, while
• plots can be permuted

In your case, one option for a model/test would be to investigate the effects of your main treatment while accounting for (partialling out) the effect of Hay. In that setting you would add treatment + Condition(Hay + block) which will remove the effect of the Hay treatment at the block/plot level (and the between block differences)) and then allow you to test the effect of treatment on your response. For a design-based permutation test you would need to set either

• how(...., blocks = block) or
• how(...., plots = Plots(strata = block)) (default is type = "none", which is what we want)

Either way works and in this instance for this specific test (model + permutation test combination) but the blocks way is simpler.

Instead, if you wanted to test the effect of the Hay treatment, being a variable that varies at the whole plot level, you would need to permute your blocks keeping the samples within the blocks fixed. As this requires permutation of your blocks, you can't use the {permute} blocks option because those are never permuted. to do this (IIRC) you need a model y ~ Hay + Condition(block) and

how(within = Within(type = "none"), plots = Plots(strata = block, type = "free"))

where we are saying to keep the order of samples within blocks (plots) fixed while shuffling the groups of samples (groups at the plot/block level) randomly. This comes with the requirement that your plots contain the same number of observations.

Does that answer your main query? The extra thinking I mentioned above would be around the specific setting you present.

[katharinehogan]

Ok, I think I see what you're saying. Final (I think) follow up question: can you provide any insight as to why the use of strata is discouraged? I.e. is it due to more of a flaw in strata or the fact that permute::how() is more customizable and nuanced and may provide additional inference? Thanks again for all your thoughts.

[jarioksa]

I think strata are just old-fashioned. They were used in vegan before permute, and they were the only structured way of permutation. strata came to vegan from the old classic boot package, but permute provides new and much more powerful schemes of constrained permutation. The strata are discouraged because permute has no strata, and internally vegan must translate strata to corresponding permute constructs, and there is no guarantee such translations will be maintained for ever. Actually, I first dropped the support of strata in adonis2, but @gavinsimpson made me change my mind, and strata are still supported there. The reason why strata still exist is that they were used in 2001, and we do not want to break the user space in vegan.

[jarioksa]

Would permute define strata as a synonym, there would be no problem.(Hint @gavinsimpson.)

[gavinsimpson]

@katharinehogan I only meant we discourage use of strata because using {permute} is so much more capable that what strata offers and it would be better to use a single harmonized system across {vegan} rather than strata for one thing and {permute} when you need something more complex. There's nothing wrong with using the strata argument if all you need is to restrict permutations within blocks of samples.

Essentially strata = foo is equivalent to permutations = how(blocks = foo) (modulo some details about the default number of permutations that I forget) and I believe Jari modified adonis() so that it sets how(...., blocks = strata) if strata is not NULL (supplied by the user).

A separate equivalent is: permutations = how(plots = Plots(strata = foo, type = "none")).

@jarioksa I'm not sure what you hint is about? Would you like the blocks argument to how() to be strata? Perhaps something like

how <- function(...., blocks = strata, strata = NULL) 

say? I'm not sure what that gains us and I purposely chose blocks for this kind of restriction as it is what is used by Cajo ter Braak & Canoco. I preserved the strata idea in Plots() to define groups at that level, but thought it would be confusing to have strata used for two different things.

[katharinehogan]

@gavinsimpson and @jarioksa thanks for the replies, I did not receive a notifcation that you edited your responses. I appreciate all your thought in responding! I think I see what would be a correct way to do what I need to for my situation now.

[jarioksa]

@gavinsimpson I only wanted to say that strata have been in vegan since release 1.4-1 (October 2002). The argument was adopted from the boot package where it had been since R 0.65.2 or something that I used in 1998. In your permute package you had the same functionality but decided to call them Blocks. However, vegan kept backward-compatibility, and therefore it has a special function to interface with permute package, and there you have this piece of code (function getPermuteMatrix):

    ## apply 'strata', but only if possible: ignore silently other cases
    if (!missing(strata) && !is.null(strata)) {
        if (inherits(perm, "how") && is.null(getBlocks(perm)))
            setBlocks(perm) <- strata
    }

So strata are "obsolescent" because you ditched them and used Blocks, but they are still supported in vegan. Probably they will be supported for ever (well, that's too much to say, but as long as there is vegan in R and I maintain it). I intended to remove strata in favour of how(), but when I tried to do this in adonis2, people did not notice this and either complained or didn't notice that their use of non-existent (in adonis2) argument strata had no effect. So strata are supported also in adonis2 now.