Regarding factors in mixture3p model

[yoojchoo]

Hi Dr. Frischkorn & Dr.Popov,
Thanks for the great toolbox! I'm playing with this toolbox to see if I can apply Bays' three parameter model on my dataset.

In my experimental paradigm, subjects did a flanker task during the retention period so that there were two trial-types derived at the probe stage(e.g., Reproduction) : a trial where they made an error on the flanker task vs. a trial where they responded correctly on the flanker task. The set-size was fixed as 3.

I was interested whether/how action errors during the retention period would affected the memory representation by implementing three-parameter models, expecting that action errors would increase the P(non-target) compared to correct condition.

however, it seems that mixture3p function only asks to add three major arguments : resp_error, nt_features, and set_size. Does this mean that I can't use the mixture3p to compare correct vs. error trials and need to use mixture2p model (just to compare two conditions without adding the non-target features)?

+) I just tried to feed-in setsize as correctedness (e.g., 1=correct, 2=error) but ended up getting an error:
Error: The number of columns for non-target values in the argument 'nt_features' should equal max(set_size)-1 .

Thank you.

Best,
Yoojeong

[GidonFrischkorn]

Hi Yoojeong,
thanks for you post and questions, and great that you are trying our package!

Any predictor of the model parameters (i.e. thetat, thetant, and kappa) should go into the bmmformula you specify to predict the model parameters. In your case this might look something like:

flanker_formula <- bmmformula(
   thetat ~ 1 + flanker_error + (1 + flanker_error | ID),
   thetant ~ 1 + flanker_error + (1 + flanker_error | ID),
   kappa ~ 1 + (1 | ID)
)

This would specify that the mixture weights that capture the probability of target responses (thetat) and the probability of non-target responses (thetant) should vary as a function of the flanker_error variable. This would likely be coded as a factor.

The arguments passed to mixture3p are only relevant to identify the model properly, but they do not include any predictions of model parameters by experimental manipulations. When your set_size is constant, then you can just pass the value of your constant set size to the argument. So setting up the model object might look something like this:

flanker_model <- mixture3p(
   resp_error = "error",
   nt_features = paste("nt_col",1:2),
   set_size = 3
)

The error is the response relative to the target color, whereas the two nt_col are the other colors in your memory set relative to the target color. All these variables should be coded in radians. And then the set_size is just the constant set size of 3 in your experiment. The variable names obviously depend on your data set.

Using the bmmformula and the bmmodel object you should be able to fit the model and evaluate the results calling the bmm function:

flanker_fit <- bmm(
   model = flanker_model,
   data = flanker_data,
   formula = flanker_formula
)

In addition, I would recommend to supply the file argument to save your results once sampling is done. And if your machine allows, I would also pass the cores argument to specify over how many cores the sampling should be run in parallel.

I hope my response is helpful.
Cheers, Gidon

[yoojchoo]

Hi Dr. Frischkorn,
Thanks for your kind reply. It took me to run the model for three days as I mistakenly set the # of chains as 32, but it seems that my model worked great.
Your suggested model was very similar to what I set-up:

flanker_formula <- bmmformula(
   thetat ~ 0 + flkacc + (0 + flkacc || ID),
   thetant ~ 0 + flkacc + (0 + flkacc || ID),
   kappa ~ 0 + flkacc + (0 + flkacc || ID)
)

and the results seem to be what I expected. Following errors, that precision of the flanker_error is lower than that of flanker_correct condition.

             Estimate Est.Error     Q2.5    Q97.5
kappa_flkaccflkCorr 7.870516  1.074922 6.802424 9.055717
kappa_flkaccflkErr  6.715926  1.107846 5.460089 8.191594

> pmem 
             Estimate Est.Error              Q2.5     Q97.5
thetat_flkaccflkCorr 0.6907493 0.3250798 0.6370255 0.7283446
thetat_flkaccflkErr  0.5847848 0.3173381 0.5525286 0.6047095

It seems that p(mem) is more decreased and p(guess) increases following errors, and the p(target) and p(non-target) are somewhat I expeted. I have to run statistical inference further, but it's promising.

My follow-up questions is, 1) Do you have any idea regarding how to set-up the nubmer of chains for this type of mixture model? I heard that some setup the number of chains as 3 x # of parameters to be estimated.

2. Should I set-up the cores argument outside of the mixture model something like:

># Set up parallel sampling of mcmc chains
>options(mc.cores =  parallel::detectCores())
>
>nChains <- 6 # number of chains
>
>if (nChains >  parallel::detectCores()) {
>  nChains <-  parallel::detectCores()
>} 

Thans much!

Best,
Yoojeong

[venpopov]

The number of chains is not related to the number of parameters. Usually the default of 4 is a good choice. In bayesian MCMC models we run multiple chains so that we can be sure that the parameter estimates have converged from different starting values.

The cores argument specifies how many chains you want to run in parallel on your machine. Specifying

options(mc.cores = parallel::detectCores())

tells brms to run as many chains in parallel as you have CPU cores, but less than the number of chains you specify. So if you call the model like this:

options(mc.cores = parallel::detectCores())
flanker_fit <- bmm(
   model = flanker_model,
   data = flanker_data,
   formula = flanker_formula
)

will run 4 chains, and if your machine has 4 cores, it will run all chains in parallel. You can achieve the same by directly specifying the arguments to bmm:

flanker_fit <- bmm(
   model = flanker_model,
   data = flanker_data,
   formula = flanker_formula,
   chains = 6,
   cores = 3
)

the above code for example will run 6 chains total, with 3 chains running in parallel - so 3 chains will run, and when they finish, the other 3 will run next. Usually using the default number of 4 chains with cores = 4, if your machine has enough cores, is a good approach.

model

Good to see that your model is running. Be sure to investigate whether the model has converged - all parameter estimates should have an Rhat value less than 1.01, and you should not get warnings about divergent transitions.

just a clarfication - p_target and p_mem is the same within the contenxt of the 3-parameter model, and this corresponds to the thetat parameter, which is on the logit scale. You can find more information in the online vignette here: https://venpopov.github.io/bmm/articles/bmm_mixture_models.html

[venpopov]

btw, to help see your code better, it helps to wrap it in ``` marks like this:

```r
options(mc.cores = parallel::detectCores())
```

which displays it a nice separate box like you see in the comments above

(I edited your comment to add these)