Aim

To provide a clear understanding of some of the key measures or reliability and their implementation in python.

How

To do that I will follow references as close as possible while providing the python code of the implementation of the ideas and examples.

Revelle has greatly created and write many papers in the matter and the library psychin R with the corresponding implementations. We would follow Revelle 2021 and 2017, in particlar Revelle 2021 very closely.

We would expand a bit when calculating the omegas and in particular when using the Schmid-Leiman solution, following Wolff and Presing 2005, as it constains a clear explanations of the theory and the estimation, as in this paper authors describe the implementation in SPSS and SAS, which something similar we are trying to do here in python.

References

  • Revelle, Willian. Manuscrip. 2021. An introduction to psychometric theory with applications in R. https://personality-project.org/r/book/Chapter7.pdf
  • Revelle, William R. “psych: Procedures for personality and psychological research.” (2017).
  • Wolff, Hans-Georg, and Katja Preising. “Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS.” Behavior Research Methods 37.1 (2005): 48-58.

Examples in R.

This work: * Rafael Valero Fernández. (2022). reliabiliPy: measures of survey domain reliability in Python with explanations and examples. Cronbach´s Alpha and Omegas. (v0.0.0). Zenodo.DOI

library("psych")
correlation_matrix <- data.frame(
   v0 = c(1.0, 0.5, 0.5, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v1 = c(0.5, 1.0, 0.5, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v2 = c(0.5, 0.5, 1.0, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v3 = c(0.5, 0.5, 0.5, 1.0, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v4 = c(0.5, 0.5, 0.5, 0.5, 1.0, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v5 = c(0.5, 0.5, 0.5, 0.5, 0.5, 1.0, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v6 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 1.0, 0.6, 0.6, 0.6, 0.2, 0.2), 
   v7 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 1.0, 0.6, 0.6, 0.2, 0.2),
   v8 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 0.6, 1.0, 0.6, 0.2, 0.2), 
   v9 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 0.6, 0.6, 1.0, 0.2, 0.2),
   v10 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 1.0, 0.7), 
   v11 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.7, 1.0)
   )
correlation_matrix

Now run the analysis

 omega(correlation_matrix)
Loading required namespace: GPArotation
Omega 
Call: omega(m = correlation_matrix)
Alpha:                 0.84 
G.6:                   0.88 
Omega Hierarchical:    0.54 
Omega H asymptotic:    0.6 
Omega Total            0.9 

Schmid Leiman Factor loadings greater than  0.2 

With eigenvalues of:
  g F1* F2* F3* 
2.4 1.8 1.6 1.0 

general/max  1.33   max/min =   1.8
mean percent general =  0.36    with sd =  0.05 and cv of  0.13 
Explained Common Variance of the general factor =  0.35 

The degrees of freedom are 33  and the fit is  0 

The root mean square of the residuals is  0 
The df corrected root mean square of the residuals is  0

Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 54  and the fit is  2.38 

The root mean square of the residuals is  0.2 
The df corrected root mean square of the residuals is  0.22 

Measures of factor score adequacy             
                                                 g  F1*  F2*  F3*
Correlation of scores with factors            0.74 0.77 0.81 0.81
Multiple R square of scores with factors      0.55 0.60 0.66 0.66
Minimum correlation of factor score estimates 0.10 0.20 0.31 0.33

 Total, General and Subset omega for each subset
                                                 g  F1*  F2*  F3*
Omega total for total scores and subscales    0.90 0.86 0.86 0.82
Omega general for total scores and subscales  0.54 0.34 0.29 0.24
Omega group for total scores and subscales    0.36 0.51 0.57 0.59

library("psych")
correlation_matrix <- data.frame(
   v0 = c(1.0, 0.5, 0.5, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v1 = c(0.5, 1.0, 0.5, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v2 = c(0.5, 0.5, 1.0, 0.5, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v3 = c(0.5, 0.5, 0.5, 1.0, 0.5, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v4 = c(0.5, 0.5, 0.5, 0.5, 1.0, 0.5, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2), 
   v5 = c(0.5, 0.5, 0.5, 0.5, 0.5, 1.0, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2),
   v6 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 1.0, 0.6, 0.6, 0.6, 0.2, 0.2), 
   v7 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 1.0, 0.6, 0.6, 0.2, 0.2),
   v8 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 0.6, 1.0, 0.6, 0.2, 0.2), 
   v9 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.6, 0.6, 0.6, 1.0, 0.2, 0.2),
   v10 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 1.0, 0.7), 
   v11 = c(0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.7, 1.0)
   )
omega(correlation_matrix)
Omega 
Call: omega(m = correlation_matrix)
Alpha:                 0.84 
G.6:                   0.88 
Omega Hierarchical:    0.54 
Omega H asymptotic:    0.6 
Omega Total            0.9 

Schmid Leiman Factor loadings greater than  0.2 

With eigenvalues of:
  g F1* F2* F3* 
2.4 1.8 1.6 1.0 

general/max  1.33   max/min =   1.8
mean percent general =  0.36    with sd =  0.05 and cv of  0.13 
Explained Common Variance of the general factor =  0.35 

The degrees of freedom are 33  and the fit is  0 

The root mean square of the residuals is  0 
The df corrected root mean square of the residuals is  0

Compare this with the adequacy of just a general factor and no group factors
The degrees of freedom for just the general factor are 54  and the fit is  2.38 

The root mean square of the residuals is  0.2 
The df corrected root mean square of the residuals is  0.22 

Measures of factor score adequacy             
                                                 g  F1*  F2*  F3*
Correlation of scores with factors            0.74 0.77 0.81 0.81
Multiple R square of scores with factors      0.55 0.60 0.66 0.66
Minimum correlation of factor score estimates 0.10 0.20 0.31 0.33

 Total, General and Subset omega for each subset
                                                 g  F1*  F2*  F3*
Omega total for total scores and subscales    0.90 0.86 0.86 0.82
Omega general for total scores and subscales  0.54 0.34 0.29 0.24
Omega group for total scores and subscales    0.36 0.51 0.57 0.59

LS0tCnRpdGxlOiAiY2hlY2tpbmdfY2xhc3MiCm91dHB1dDogaHRtbF9ub3RlYm9vawotLS0KIyBBaW0KClRvIHByb3ZpZGUgYSBjbGVhciB1bmRlcnN0YW5kaW5nIG9mIHNvbWUgb2YgdGhlIGtleSBtZWFzdXJlcyBvciByZWxpYWJpbGl0eSBhbmQgdGhlaXIgaW1wbGVtZW50YXRpb24gaW4gYHB5dGhvbmAuCgojIEhvdwoKVG8gZG8gdGhhdCBJIHdpbGwgZm9sbG93IHJlZmVyZW5jZXMgYXMgY2xvc2UgYXMgcG9zc2libGUgd2hpbGUgcHJvdmlkaW5nIHRoZSBweXRob24gY29kZSBvZiB0aGUgaW1wbGVtZW50YXRpb24gb2YgdGhlIGlkZWFzIGFuZCBleGFtcGxlcy4KClJldmVsbGUgaGFzIGdyZWF0bHkgY3JlYXRlZCBhbmQgd3JpdGUgbWFueSBwYXBlcnMgaW4gdGhlIG1hdHRlciBhbmQgdGhlIGxpYnJhcnkgYHBzeWNoYGluIGBSYCB3aXRoIHRoZSBjb3JyZXNwb25kaW5nIGltcGxlbWVudGF0aW9ucy4gV2Ugd291bGQgZm9sbG93IFJldmVsbGUgMjAyMSBhbmQgMjAxNywgaW4gcGFydGljbGFyIFJldmVsbGUgMjAyMSB2ZXJ5IGNsb3NlbHkuCgpXZSB3b3VsZCBleHBhbmQgYSBiaXQgd2hlbiBjYWxjdWxhdGluZyB0aGUgb21lZ2FzIGFuZCBpbiBwYXJ0aWN1bGFyIHdoZW4gdXNpbmcgdGhlIGBTY2htaWQtTGVpbWFuYCBzb2x1dGlvbiwgZm9sbG93aW5nIFdvbGZmIGFuZCBQcmVzaW5nIDIwMDUsIGFzIGl0IGNvbnN0YWlucyBhIGNsZWFyIGV4cGxhbmF0aW9ucyBvZiB0aGUgdGhlb3J5IGFuZCB0aGUgZXN0aW1hdGlvbiwgYXMgaW4gdGhpcyBwYXBlciBhdXRob3JzIGRlc2NyaWJlIHRoZSBpbXBsZW1lbnRhdGlvbiBpbiBgU1BTU2AgYW5kIGBTQVNgLCB3aGljaCBzb21ldGhpbmcgc2ltaWxhciB3ZSBhcmUgdHJ5aW5nIHRvIGRvIGhlcmUgaW4gYHB5dGhvbmAuCgoKIyMgUmVmZXJlbmNlcwoqIFJldmVsbGUsIFdpbGxpYW4uIE1hbnVzY3JpcC4gMjAyMS4gQW4gaW50cm9kdWN0aW9uIHRvIHBzeWNob21ldHJpYyB0aGVvcnkgd2l0aCBhcHBsaWNhdGlvbnMgaW4gUi4gaHR0cHM6Ly9wZXJzb25hbGl0eS1wcm9qZWN0Lm9yZy9yL2Jvb2svQ2hhcHRlcjcucGRmCiogUmV2ZWxsZSwgV2lsbGlhbSBSLiAicHN5Y2g6IFByb2NlZHVyZXMgZm9yIHBlcnNvbmFsaXR5IGFuZCBwc3ljaG9sb2dpY2FsIHJlc2VhcmNoLiIgKDIwMTcpLgoqIFdvbGZmLCBIYW5zLUdlb3JnLCBhbmQgS2F0amEgUHJlaXNpbmcuICJFeHBsb3JpbmcgaXRlbSBhbmQgaGlnaGVyIG9yZGVyIGZhY3RvciBzdHJ1Y3R1cmUgd2l0aCB0aGUgU2NobWlkLUxlaW1hbiBzb2x1dGlvbjogU3ludGF4IGNvZGVzIGZvciBTUFNTIGFuZCBTQVMuIiBCZWhhdmlvciBSZXNlYXJjaCBNZXRob2RzIDM3LjEgKDIwMDUpOiA0OC01OC4KCkV4YW1wbGVzIGluIFIuCgpUaGlzIHdvcms6IAoqIFJhZmFlbCBWYWxlcm8gRmVybsOhbmRlei4gKDIwMjIpLiByZWxpYWJpbGlQeTogbWVhc3VyZXMgb2Ygc3VydmV5IGRvbWFpbiByZWxpYWJpbGl0eSBpbiBQeXRob24gd2l0aCBleHBsYW5hdGlvbnMgYW5kIGV4YW1wbGVzLiBDcm9uYmFjaMK0cyBBbHBoYSBhbmQgT21lZ2FzLiAodjAuMC4wKS4gWmVub2RvLlshW0RPSV0oaHR0cHM6Ly96ZW5vZG8ub3JnL2JhZGdlLzQ0NTg0NjUzNy5zdmcpXShodHRwczovL3plbm9kby5vcmcvYmFkZ2UvbGF0ZXN0ZG9pLzQ0NTg0NjUzNykgCmBgYHtyfQpsaWJyYXJ5KCJwc3ljaCIpCmBgYAoKYGBge3J9CmNvcnJlbGF0aW9uX21hdHJpeCA8LSBkYXRhLmZyYW1lKAogICB2MCA9IGMoMS4wLCAwLjUsIDAuNSwgMC41LCAwLjUsIDAuNSwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiksIAogICB2MSA9IGMoMC41LCAxLjAsIDAuNSwgMC41LCAwLjUsIDAuNSwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiksCiAgIHYyID0gYygwLjUsIDAuNSwgMS4wLCAwLjUsIDAuNSwgMC41LCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yKSwgCiAgIHYzID0gYygwLjUsIDAuNSwgMC41LCAxLjAsIDAuNSwgMC41LCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yKSwKICAgdjQgPSBjKDAuNSwgMC41LCAwLjUsIDAuNSwgMS4wLCAwLjUsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIpLCAKICAgdjUgPSBjKDAuNSwgMC41LCAwLjUsIDAuNSwgMC41LCAxLjAsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIpLAogICB2NiA9IGMoMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMS4wLCAwLjYsIDAuNiwgMC42LCAwLjIsIDAuMiksIAogICB2NyA9IGMoMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC42LCAxLjAsIDAuNiwgMC42LCAwLjIsIDAuMiksCiAgIHY4ID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjYsIDAuNiwgMS4wLCAwLjYsIDAuMiwgMC4yKSwgCiAgIHY5ID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjYsIDAuNiwgMC42LCAxLjAsIDAuMiwgMC4yKSwKICAgdjEwID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDEuMCwgMC43KSwgCiAgIHYxMSA9IGMoMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjcsIDEuMCkKICAgKQpjb3JyZWxhdGlvbl9tYXRyaXgKYGBgCgpOb3cgcnVuIHRoZSBhbmFseXNpcwpgYGB7cn0KIG9tZWdhKGNvcnJlbGF0aW9uX21hdHJpeCkKCmBgYAoKYGBge3J9CmxpYnJhcnkoInBzeWNoIikKY29ycmVsYXRpb25fbWF0cml4IDwtIGRhdGEuZnJhbWUoCiAgIHYwID0gYygxLjAsIDAuNSwgMC41LCAwLjUsIDAuNSwgMC41LCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yKSwgCiAgIHYxID0gYygwLjUsIDEuMCwgMC41LCAwLjUsIDAuNSwgMC41LCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yKSwKICAgdjIgPSBjKDAuNSwgMC41LCAxLjAsIDAuNSwgMC41LCAwLjUsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIpLCAKICAgdjMgPSBjKDAuNSwgMC41LCAwLjUsIDEuMCwgMC41LCAwLjUsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIpLAogICB2NCA9IGMoMC41LCAwLjUsIDAuNSwgMC41LCAxLjAsIDAuNSwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiksIAogICB2NSA9IGMoMC41LCAwLjUsIDAuNSwgMC41LCAwLjUsIDEuMCwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiksCiAgIHY2ID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAxLjAsIDAuNiwgMC42LCAwLjYsIDAuMiwgMC4yKSwgCiAgIHY3ID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjYsIDEuMCwgMC42LCAwLjYsIDAuMiwgMC4yKSwKICAgdjggPSBjKDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuNiwgMC42LCAxLjAsIDAuNiwgMC4yLCAwLjIpLCAKICAgdjkgPSBjKDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuNiwgMC42LCAwLjYsIDEuMCwgMC4yLCAwLjIpLAogICB2MTAgPSBjKDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMS4wLCAwLjcpLCAKICAgdjExID0gYygwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuMiwgMC4yLCAwLjIsIDAuNywgMS4wKQogICApCm9tZWdhKGNvcnJlbGF0aW9uX21hdHJpeCkKYGBg