|
Data for this example are based on 32 items from a science assessment test in the
subjects of biology, chemistry, and physics administered to twelfth-grade students
near the end of the school year. The items were classified by subject matter for
purposes of the bifactor analysis.
The first five cases from the data file exampl04.dat are shown below. The
FILE keyword on the INPUT command denotes this file as the data source and the SCORES
keyword indicates that it contains item scores.
Case001 14523121312421534414334135131545
Case002 34283328312821524114338184145848
Case003 14543223322131554134331134134441
Case004 24423324322421524134315254134242
Case005 24523221122421544514333115131241
The case identification is given in the first 7 columns, and is listed first in
the variable format statement. The length of this field is also indicated by the
NIDW keyword on the INPUT command. After using the 'T' operator to tab to column
11, the 32 item responses are read as single characters (32A1).
(7A1,T11,32A1)
32 items from the science test are used as indicated by the NITEM keyword on the
PROBLEM command, and the RESPONSE keyword denotes the number of possible responses.
The six responses are listed in the RESPONSE command. Naming of the items is done
using the NAMES command, while the KEY command lists the correct response to each
item.
The BIFACTOR command is used to request full-information estimation of loadings
on a general factor in the presence of item-group factors. Three item-group factors
are present (NIGROUP = 3), with assignation of the items to these groups as specified
with the IGROUP keyword. The CPARMS keyword lists the probabilities of chance success
on each item. By setting the LIST keyword to 3, the bifactor loadings will be printed
in both item and in item-group order in the output file. A total of 30 EM cycles
(CYCLES = 30) will be performed in the bifactor solution.
The SCORE command is used to obtain, for each distinct pattern, the EAP score of
the general factor of the bifactor model and to obtain the standard error estimate
of the general factor score allowing for conditional dependence introduced by the
group factors. Factor scores for the first 10 cases will be printed to the output
file (LIST = 10) and the guessing model will be used in the computation of the factor
scores (CHANCE option).
>TITLE
ITEM BIFACTOR ANALYSIS OF A TWELFTH-GRADE
SCIENCE ASSESSMENT TEST
THE
GENERAL FACTOR WILL BE SCORED
>PROBLEM NITEM=32,RESPONSE=6;
>NAMES
CHEM01 PHYS02,CHEM03,PHYS04,PHYS05,CHEM06,BIOL07,CHEM08,BIOL09,BIOL10,BIOL11, PHYS12,BIOL13,PHYS14,BIOL15,CHEM16,BIOL17,BIOL18,PHYS19,PHYS20,BIOL21,BIOL22,
PHYS23,BIOL24,PHYS25,PHYS26,BIOL27,PHYS29,CHEM29,PHYS30,BIOL31,CHEM32;
>RESPONSE 8,1,2,3,4,5;
>KEY 14523121312421534414334135131545;
>BIFACTOR NIGROUP=3,LIST=3,CYCLES=30,
IGROUPS=(2,3,2,3,3,2,1,2,1,1,1,3,1,3,1,2,1,1,3,3,1,1,
3,1,3,3,1,3,2,3,1,2),
CPARMS=(0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,
0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,
0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1,0.1);
>SCORE LIST=10,CHANCE;
>SAVE PARM,FSCORES;
>INPUT NIDW=4,SCORES,FILE='EXAMPL04.DAT';
(7A1,T11,32A1)
>STOP;
 |
Exampl04.tsf illustrates the extension of a one-factor model to a so-called
bifactor model by the inclusion of group factors. The bifactor model is applicable
when an achievement test contains more than one subject matter content area. The
data set exampl04.dat consists of the results of a 32-item science assessment
test in the subjects biology, chemistry, and physics. Items are classified according
to subject matter where 1 = biology, 2 = chemistry and 3 = physics. Note that TESTFACT
does not estimate guessing parameters, but does allow the user to specify the values
(see CPARMS keyword) in which case a 3-parameter
model that provides for the effect of guessing is fitted to the data.
The analysis specified in exampl04.tsf produces Phase 0, Phase 1, Phase 2, Phase
6 and Phase 7 output. The interpretation of
Phases 0, 1, and 2 are omitted here since a detailed discussion of these parts
of the output is given before.
Display 1 lists the chance and initial intercept and slope estimates. Note that
the initial intercept estimates are set equal to zero, the initial slope estimates
are set to 1.414 for the general factor and 1.00 for the group factors. These initial
values are routinely used in TESTFACT for bifactor models.
DISPLAY 1. CHANCE
AND INITIAL INTERCEPT AND SLOPE ESTIMATES
CHANCE INTERCEPT
SLOPES
1 2
1 CHEM01 0.100 0.000 1.414 1.000
2 PHYS02 0.100 0.000 1.414 1.000
3 CHEM03 0.100 0.000 1.414 1.000
4 PHYS04 0.100 0.000 1.414 1.000
5 PHYS05 0.100 0.000 1.414 1.000
6 CHEM06 0.100 0.000 1.414 1.000
…
31 BIOL31 0.100 0.000 1.414 1.000
32 CHEM32 0.100 0.000 1.414 1.000
One may optionally include the TETRACHORIC command (see exampl03.tsf) when
fitting a bifactor model. This command is required if a printout of residuals is
requested. If a TETRACHORIC command is used, tetrachoric correlations are computed
pairwise for the  =
496 pairs of items. There are a total of 20 item pairs that cannot be used since
their corresponding  frequency
tables contain zero or near-zero off-diagonal or marginal frequencies. In these cases,
a tetrachoric correlation of 1 is substituted in the matrix of tetrachoric correlations.
The inclusion or exclusion of the TETRACHORIC command has no effect on the estimation
procedure, since the starting values for the marginal maximum likelihood procedure
are fixed, and do not depend on the matrix of tetrachoric coefficients.
Display 2-3: EM estimation and quadrature points and weights
The bifactor procedure uses the 9 quadrature points and weights listed below. MML
estimation for the bifactor model requires quadrature in only two dimensions. For
a more detailed discussion, see the Phase 7 part of the output.
DISPLAY 2. THE
EM ESTIMATION OF PARAMETERS
9 QUADRATURE POINTS
DISPLAY 3. 9
QUADRATURE POINTS AND WEIGHTS
1 -4.000000 0.000134
2 -3.000000 0.004432
3 -2.000000 0.053991
4 -1.000000 0.241971
5 0.000000 0.398942
6 1.000000 0.241971
7 2.000000 0.053991
8 3.000000 0.004432
9 4.000000 0.000134
The number of cycles for the EM algorithm is set equal to 30 (CYCLES = 30 in
the BIFACTOR command). At each cycle, the value
of ? log L (see Example 3 output) as well as the maximum change in the intercept
and slope parameters are given. At cycle 30 the maximum change in intercept is 0.0050.
The general factor slope estimates for the 32 items changed at most by 0.0047 while
the corresponding value for the group factor equals 0.0095. These values indicate
that, although convergence was not attained within the specified 30 cycles, the solution
after 30 cycles is probably acceptable for all practical purposes.
CYCLE 30 -2 X MARGINAL LOG-LIKELIHOOD = 0.1882667932D+05
CHANGE
= 0.4390039691D-01
MAXIMUM CHANGE OF ESTIMATES
INTERCEPT
= 0.004952 SLOPE
= 0.004758
Display 4: Chi-square and degrees of freedom
DISPLAY 4. CHI-SQUARE
= 11150.36 DF = 503.00 P
= 0.000
The  -value
is 1150.36 with degrees of freedom equal to 503. The number of degrees of freedom
is calculated as
where N is the number of distinct patterns, n is the number
of items, and  is
the number of items assigned to group factors. For this example, N = 600,
n = 32 and, since all the items are assigned to group factors, =
32.
The -statistic
is only correct when all possible  patterns
are observed. For the present sample, since  ,
the -statistic
is too inaccurate to be used as a goodness-of-fit test statistic. The difference
in the -statistics
for alternative models, however, yields a valid test statistic for judging whether
the inclusion of additional parameters results in a significant improvement of model
fit.
Example
It is hypothesized that the 12 physics items are indicators of a general factor
only, while the biology and chemistry items are indicators of a general and two uncorrelated
group factors. We wish to test
 :
The 32 items are indicators of a general factor, but the 13 biology and 7
chemistry items are also indicators of two uncorrelated group factors. :
The 32 items are indicators of a general as well as three uncorrelated group factors.
To obtain the -statistic
and degrees of freedom under ,
the BIFACTOR command is modified as follows:
>BIFACTOR NIGROUP=2, LIST=3, CYCLES=30,
IGROUPS=(2,0,2,0,0,2,1,2,1,1,1,0,1,0,1,2,1,1,0,0,1,1,0,1,0,0,1,0,2,0,1,2), CPARMS=(0.1,
0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1,
0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1,
0.1, 0.1, 0.1, 0.1);
Note that the NIGROUP keyword is set equal to
2 and that each value of 3, corresponding to the position of the physics items in
the data set, is substituted by a value of 0 in the IGROUPS keyword.
A symbol indicates that the corresponding items are not assigned to any group factors.
A graphical presentation of the model
is shown below.
If we run exampl04.tsf with the changes to the BIFACTOR command
as discussed above, the -statistic
value and degrees of freedom shown below is obtained.
DISPLAY 4. CHI-SQUARE
= 11179.71 DF=515.00 P=0.000
To test against ,
one computes the difference in the corresponding s
and degrees of freedom. Hence =
11179.71 ?11150.36 = 29.35 with degrees of freedom equal to 515 ?503 = 12.
Since  , is
rejected and it is concluded that items from all 3 subjects should be used for the
group factors.
Display 5: Untransformed item parameters
The estimates for the intercept and slope parameters are listed below.
DISPLAY 5. UNTRANSFORMED
ITEM PARAMETERS
CHANCE
INTERCEPT SLOPES
1 2
1 CHEM01 0.100 -1.054 0.709 0.417
2 PHYS02 0.100 0.126 1.019 0.548
3 CHEM03 0.100 -1.360 1.265 -0.182
4 PHYS04 0.100 -0.578 0.469 0.377
5 PHYS05 0.100 0.263 0.635 0.337
6 CHEM06 0.100 -2.729 1.706 0.308
…
31
BIOL31 0.100 1.608 1.447 -0.005
32 CHEM32 0.100 -1.522 0.190 0.066
An alternative way to present these estimated parameters are shown below for the
first 10 items.
Item Chance Intercept General Group1 Group2 Group3
--------------------------------------------------------------------
1 CHEM01 0.100 -1.054 0.709 0.000 0.417 0.000
2 PHYS02 0.100 0.126 1.019 0.000 0.000 0.548
3 CHEM03 0.100 -1.360 1.265 0.000 -0.182 0.000
4 PHYS04 0.100 -0.578 0.469 0.000 0.000 0.377
5 PHYS05 0.100 0.263 0.635 0.000 0.000 0.337
6 CHEM06 0.100 -2.729 1.706 0.000 0.308 0.000
7 BIOL07 0.100 0.839 0.586 0.636 0.000 0.000
8 CHEM08 0.100 -2.220 1.144 0.000 0.929 0.000
9 BIOL09 0.100 1.287 0.212 0.476 0.000 0.000
10 BIOL10 0.100 -0.464 0.762 0.444 0.000 0.000
Display 6: Percent of variance
DISPLAY 6. PERCENT
OF VARIANCE
----------------------------
GENERAL 0 31.7580
ITEM GROUP 1 3.8018
ITEM GROUP 2 2.7716
ITEM GROUP 3 2.9551
UNIQUENESS 58.7134
----------------------------
The percentage variance explained by each of the four factors is calculated as follows:
Let  denote
the j-th slope parameter for item i, i = 1, 2, ? 32. If we define
then slopes are transformed to factor loadings (see Display 9 in the discussion
of the Example 3 output) using the relationship
 .
Example
For item 7,
The item 7 loadings are therefore 0.586/1.322 = 0.443 and 0.636/1.322 = 0.481 respectively.
Let  be
a  matrix
of factor loadings with elements (see Display 7)
The percentage variance explained by factor j is calculated as
where n = 32 and  the j-th
characteristic root of  (See
also the discussion of the Example 3 output, Display 15). The uniqueness component
is calculated as
Display 7: Standardized difficulties, communalities and
bifactor loadings
The bifactor loadings are derived from the slope estimates using the formula  (see
Display 6 above). The standardized item i difficulty equals  .
Example
For item 7,  =
1.322 so that the standardized difficulty is 
Communalities are equal to the sum of the squares of the factor loadings. For example,
the item 1 communality is equal to
DISPLAY 7. BIFACTOR
RESULTS IN ITEM SEQUENTIAL ITEM ORDER
ITEM GROUP
DIFFICULTY COMMUNALITY GENERAL SPECIFIC
----------------------------------------------------------------
1
CHEM01 2 0.8138 0.4036 0.5475 0.3222
2
PHYS02 3 -0.0821 0.5725 0.6663 0.3585
3
CHEM03 2 0.8380 0.6204 0.7797 -0.1120
4
PHYS04 3 0.4952 0.2659 0.4022 0.3226
5
PHYS05 3 -0.2137 0.3408 0.5156 0.2739
6
CHEM06 2 1.3636 0.7504 0.8524 0.1540
…
31 BIOL31 1 -0.9140 0.6768 0.8227 -0.0029
32
CHEM32 2 1.4923 0.0388 0.1859 0.0649
----------------------------------------------------------------
Display 8: Bifactor results in item group order
The printout below shows the same information as for Display 7, except that the
items are re-ordered by group number. All 32 items have positive loadings on the
general factor, while the group factor loadings for BIOL31, CHEM03 and PHYS30 are
negative, but relatively small.
DISPLAY 8. BIFACTOR
RESULTS IN ITEM GROUPORDER
ITEM GROUP
DIFFICULTY COMMUNALITY GENERAL SPECIFIC
----------------------------------------------------------------
7
BIOL07 1 -0.6347 0.4277 0.4430 0.4812
9
BIOL09 1 -1.1417 0.2136 0.1882 0.4221
10 BIOL10 1 0.3483 0.4372 0.5713 0.3330
11 BIOL11 1 -2.0915 0.3887 0.5390 0.3133
13 BIOL13 1 -0.3328 0.3918 0.5249 0.3411
15 BIOL15 1 -0.8412 0.4683 0.5493 0.4081
17 BIOL17 1 -1.7506 0.3391 0.5395 0.2192
18 BIOL18 1 0.5366 0.7089 0.8358 0.1022
21 BIOL21 1 -1.3192 0.2177 0.2152 0.4140
22 BIOL22 1 -1.4669 0.3685 0.5956 0.1175
24 BIOL24 1 -0.5344 0.3935 0.5922 0.2069
27 BIOL27 1 -1.0345 0.5244 0.7042 0.1686
31 BIOL31 1 -0.9140 0.6768 0.8227 -0.0029
1 CHEM01 2 0.8138 0.4036 0.5475 0.3222
3 CHEM03 2 0.8380 0.6204 0.7797 -0.1120
6 CHEM06 2 1.3636 0.7504 0.8524 0.1540
8 CHEM08 2 1.2465 0.6847 0.6422 0.5219
16 CHEM16 2 0.3761 0.3262 0.4820 0.3065
29 CHEM29 2 0.6132 0.6825 0.5533 0.6135
32 CHEM32 2 1.4923 0.0388 0.1859 0.0649
2 PHYS02 3 -0.0821 0.5725 0.6663 0.3585
4 PHYS04 3 0.4952 0.2659 0.4022 0.3226
5 PHYS05 3 -0.2137 0.3408 0.5156 0.2739
12 PHYS12 3 0.3828 0.0437 0.0999 0.1836
14 PHYS14 3 -0.5195 0.4537 0.4988 0.4527
19 PHYS19 3 -0.0128 0.2476 0.4967 0.0289
20 PHYS20 3 -1.0889 0.4226 0.6200 0.1954
23 PHYS23 3 0.7102 0.3164 0.4571 0.3279
25 PHYS25 3 0.5005 0.4205 0.4814 0.4345
26 PHYS26 3 0.2243 0.6128 0.7500 0.2242
28 PHYS29 3 0.0337 0.3544 0.5948 0.0255
30 PHYS30 3 0.3034 0.0979 0.2912 -0.1146
----------------------------------------------------------------
The factor scores are so-called expected a-posteriori estimates of the general ability
factor under the assumption of normality (see Phase 7, exampl03.out).
Let  denote
the general ability for examinee i. The EAP score is the conditional expectation  where  is
the item j score for examinee i. From well-known results for conditional
distributions it follows that
where
The marginal probability function  is
obtained in the EM step using a two-dimensional quadrature formula. Suppose  denotes
the set of ordered item scores for the three groups. Under the assumption of uncorrelated
group factors, it follows for Example 4 that
where denotes
the general ability for examinee i,  is
the group 1 (biology),  the
group 2 (chemistry) and  the
group 3 (physics) ability respectively. Note that (see Display 9)  ,  ,
?  .
Under the independence assumption, it follows that
 .
Each term in this product can be expressed as a two-dimensional integral. The first
term, for example, can be evaluated from
This integral can be approximated by
where  and  are
the weights and  and  the
points shown as Display 9 of the output.
Display 10: General factor scores and standard errors
The ability scores for each case and corresponding standard error estimates are
tabulated. Examinee 5, for example, selected the correct answers to 22 of the 32
items. Therefore, the percentage correct is 
The estimated ability score for this candidate is 0.8 with a standard error of 0.411.
Candidate 7 also obtained correct answers to 22 of the 32 items, but the ability
estimate is 0.691. It is evident that the ability estimated depend on the number
of correct items as well as which subset of items was answered correctly.
DISPLAY 10. GENERAL
FACTOR SCORE AND STANDARD ERROR (S.E.)
CASE HEADER:
CASE NUMBER PERCENT
PERCENT CASE ID
PRESENTED CORRECT
OMITTED
SCORE
AND S.E.
============================================================================
1 32 100.0 0.0 Case001
2.507 0.591
2 32 53.1 0.0 Case002
-0.066 0.323
3 32 56.2 0.0 Case003
0.032 0.380
4 32 50.0 0.0 Case004
-0.559 0.505
5 32 68.8 0.0 Case005
0.800 0.411
6 32 62.5 0.0 Case006
0.342 0.491
7 32 68.8 0.0 Case007
0.691 0.469
8 32 65.6 0.0 Case008
0.286 0.463
9 32 28.1 0.0 Case009
-1.434 0.518
10 32 46.9 0.0 Case010
-0.965 0.342
Summary statistics for score estimates
The number of cases scored is equal to 600, with a mean of ?.0258 and standard deviation
of 0.9011. Note that the ability scores are estimated under the assumption that the
general factor ability has a normal distribution with mean 0 and standard deviation
1. For large data sets, one ideally wants the estimated ability scores to have mean
0 and standard deviation 1.
The root-mean-square posterior standard deviations are calculated as follows.
where  ,  etc.
The RMS value of 0.4394 is relatively large, and indicates that, in general, 95%
confidence intervals for the estimated scores will be wide. For example, a 95% confidence
interval for examinee 5 is
The empirical reliability is a measure of how close the observed scores are to the
true, but unobserved, scores. A reliability of 1, for example, implies that one can
safely substitute the observed test scores for the unknown true scores.
SUMMARY STATISTICS FOR SCORE ESTIMATES
======================================
CASES SCORED 600
MEAN: -0.0258
S.D.: 0.9011
VARIANCE: 0.8119
ROOT-MEAN-SQUARE POSTERIOR STANDARD DEVIATIONS
RMS: 0.4344
VARIANCE: 0.1887
EMPIRICAL
RELIABILITY: 0.8114
|
