Scale reliability: items within occasions within persons

[nickmoberly]

Hello all,

I would like to know the internal consistency reliability (Cronbach's alpha equivalent) of a five-item Likert scale that a sample of people fill out on ten occasions, taking into account the nested data structure.

One way this has been done (e.g., Nezlek & Gable, 2001) is to construct an unconditional (empty) 3-level model (persons = Level 3, occasions = Level 2, scale items = Level 1) with an intercept that is random at the occasion and person level. Reliability estimates at the occasion and person level can then be obtained for the scale. HLM software automatically produces these values as outputs (under "RELIABILITY ESTIMATES"), but I do not know how to obtain these values from MLwiN, or how to calculate them from the variances in the standard output.

Can anybody help?

Many thanks,

Nick

Nezlek, J. B., & Gable, S. L. (2001). Depression as a moderator of relationships between positive daily events and day-to-day psychological adjustment. Personality And Social Psychology Bulletin, 27(12), 1692-1704.

[nickmoberly]

A-ha, I just checked (Snijders & Bosker, 1999, pp. 24-26) on 'Reliability of aggregated variables'.

For an empty two-level random-intercept model, where 'number of items' is Level 1 cluster size and constant:

Reliability = Level 2 var / (Level 2 var + [(Level 1 var)/number of items])

So I'm assuming, for an empty three-level random-intercept model:

Reliability = (Level 2 var + Level 3 var) / (Level 2 var + Level 3 var + [(Level 1 var)/number of items])

This is [Method 1] which appears to produce the same result as calculating Cronbach’s alpha while ignoring the person-nesting, and is a function of the expected correlation between two (Level 1) items within the same Level 2 unit, taking into account that they will be from the same Level 3 unit.

However one could [Method 2] calculate reliability at Level 2 only:

Reliability = Level 2 var / (Level 2 var + [(Level 1 var)/number of items])

Reliability [Method 1] > Reliability [Method 2].

It seems more meaningful to report reliability using Method 1 given that the expected correlation with which it is associated is more relevant. Or is it?

Any thoughts much appreciated,

Nick