필사이언스 - IRT 예제

필 사이언스 | Systat | CambridgeSoft | EndNote

Parscale: DIF Analysis Using a Partial Credit Model

This example illustrates differential item functioning (DIF) analysis of multiple category item responses. The SCORE command is required, and was included in the syntax. For the DIF model, however, no scoring is done and there is no Phase 3 output.

Syntax

This example illustrates differential item functioning (DIF) analysis of multiple category item responses.

EXAMPLE 6:  DIF MODEL:             SAMPLE1 GROUP1 N=500 N=(0,1)
            PARTIAL CREDIT MODEL: SAMPLE2 GROUP2 N=500 N=(0,1)
>FILE   DFNAME='PSLDAT\EXAMPL06.DAT', NFNAME='PSLDAT\EXAMPL06.NFL' ;
>INPUT  NIDCH=8, NTOT=6, LENGTH=6, NTEST=1, NFMT=1, MGROUP=2 ;
(8A1,5X,A1,/6A1)
>TEST   TNAME=PARV3E, ITEM=(1,2,3,4,5,6),
        INAME=('I001','I002','I003','I004','I005','I006'), NBLOCK=6 ;
>BLOCK1 REPEAT=6, NIT=1, NCAT=3, ORIGINAL=(1,2,3) ;
>MGROUP DIF=(0,1,0,0), GNAME=(MALE,FEMALE), GCODE=('1','2') ;
>CALIB  LOGISTIC, PARTIAL, NQPT=25, CYCLES=(100,1,1,1,1,1), NEWTON=2,
        CRIT=0.01, POSTERIOR ;
>SCORE  ;

Raw data are read from the file exampl06.dat using the DFNAME keyword on the FILES command. The NFNAME keyword is used to indicate the file in which the not-presented key is given. The data file contains responses to 6 items, as indicated on the INPUT command, where NTOTAL is set to 6.

One test, 6 items in length, is considered. This information is provided to the program using the NTEST and LENGTH keywords on the INPUT command. The format statement occupies one line in the syntax file as indicated by the NFMT keyword. The final keyword used on the INPUT command, MGROUP, indicates that a multiple-group analysis is to be performed and that there are two groups. Note that the MGROUP keyword is used in combination with the MGROUP command, which must follow directly after the BLOCK command(s).

The format statement includes information on three fields in the raw data file. The subject ID (8A1) and group identification field (A1) are read on the first of the two records per respondent, while the 6 item responses (6A1) are read from the second record.

On the TEST command, a name for the test is provided using the TNAME keyword. The items on this test are listed using the ITEMS keyword, while the INAME keyword is used to provide names for the items. Finally, by setting NBLOCK to 6, it is indicated that 6 BLOCK commands will follow the TEST command.

In this example, there is one item with three categories originally coded 1, 2, and 3 in each block as indicated by the NITEMS, NCAT and ORIGINAL keywords respectively. Because the rating scale model is not used here, separate category parameters are estimated for each item, and the REPEAT keyword indicates that the BLOCK command should be repeated six times.

The second value (1) assigned to the DIF keyword of the MGROUP command requests DIF analysis of the item threshold parameters. All other values in this keyword are equal to zero, indicating that only thresholds are allowed to differ between the groups. The GNAME and GCODE keywords are used to assign names and codes to the two groups. By default, the first group will be used as the reference group. To change the reference group, the REFERENCE keyword on the MGROUP command may be used.

A partial credit model with logistic response function is requested through the use of the PARTIAL and LOGISTIC keywords on the CALIB command. The default number of quadrature points is 30. In this case, NQPT is set to 25, because fewer points are needed when the number of items is small. By setting the CYCLES keyword to 100, a maximum of 100 EM cycles will be performed, followed by two Newton cycles (NEWTON =2). The convergence criterion is somewhat relaxed by setting CRIT to 0.01 instead of using the default convergence criterion of 0.001. Finally, the POSTERIOR option is added to the CALIB command. As a default, the posterior distribution is computed after the computation of expected proportions during the E-step as their by-product. Therefore, these expected sample sizes and expected frequencies of categorical responses are computed based on the posterior distribution in the previous EM cycle. Adding the POSTERIOR option forces the program to compute the posterior distribution again after the M-step. Therefore, the expected proportions can be computed during the E-step based on an updated posterior distribution. This was added to be consistent with the BILOG-MG program in the case of two categories.

Phase 0 output

When the MGROUP keyword and MGROUP command are used or multiple TEST/BLOCK commands are used, additional information is written to the phase 0 output file.

NUMBER OF SUBGROUPS: 2

FORMAT OF DATA INPUT IS
(8A1,5X,A1,/,6A1)

>TEST TNAME=PARV3E, ITEM=(1,2,3,4,5,6),
INAME=('I001','I002','I003','I004','I005','I006'), NBLOCK=6 ;

BLOCK CARD: 1

>BLOCK1 REPEAT=6, NIT=1, NCAT=3, ORIGINAL=(1,2,3) ;

BLOCK CARD:   2 IS COPIED FROM BLOCK   1
BLOCK CARD:   3 IS COPIED FROM BLOCK   1
BLOCK CARD:   4 IS COPIED FROM BLOCK   1
BLOCK CARD:   5 IS COPIED FROM BLOCK   1
BLOCK CARD:   6 IS COPIED FROM BLOCK   1

>MGROUP DIF=(0,1,0,0), GNAME=(MALE,FEMALE), GCODE=('1','2') ;

In the next few lines, the program echoes the information on parameters allowed to be different between groups as specified with the DIF keyword: in this case, only the thresholds are allowed to differ between the two groups. The MALE group will be used as reference group.

MULTIPLE GROUP MODEL: DIF (TREND) MODEL

GROUP PARAMETER FOR SLOPE:                 NO
GROUP PARAMETER FOR THRESHOLD:             YES
GROUP PARAMETER FOR CATEGORY:              NO
GROUP PARAMETER FOR GUESSING:              NO
REFERENCE GROUP FOR DIF MODEL:              1 MALE

SUBGROUP NAME AND CODE
======================

1 MALE 1
2 FEMALE 2

DIF OR COMMON BLOCK
===================
   1 BLOCK     DIF BLOCK
   2 BLOCK     DIF BLOCK
    3 BLOCK     DIF BLOCK
   4 BLOCK     DIF BLOCK
   5 BLOCK     DIF BLOCK
   6 BLOCK     DIF BLOCK

Phase 1 output

The only difference between the Phase 1 output for a single group analysis and for a multiple group analysis is that the summary item statistics are first given by subgroup and then for the total group. The output for the first item is shown below for all three cases. We see that females were more likely to respond in category 3 and less likely to respond in category 1 than the males. In general, 76% of the total responses were in category 3.

1 SUBGROUP: MALE

2 SUBGROUP: FEMALE

TOTAL

Item means, initial slope estimates, and Pearson and polyserial item-test correlations are given in the next table.

----------------------------------------------------------------------------
  BLOCK     | RESPONSE   TOTAL SCORE | PEARSON  &  | INITIAL      INITIAL
     ITEM   |   MEAN         MEAN   | POLYSERIAL  |  SLOPE       LOCATION
            |   S.D.*       S.D.*   | CORRELATION |
---------------------------------------------------------------------------
   BLOCK   |                         |             |
     1 I001 |     2.539     13.162   |     0.714  |     1.000       0.000
            |     0.831*     3.765*  |     0.976  |
----------------------------------------------------------------------------
   CATEGORY | SCORING   |   MEAN   |     S.D.   | PARAMETER
      1     |    1.000  |     8.190 |     2.235   |    0.000
      2     |    2.000  |   11.263 |     2.899   |   -0.155
      3     |    3.000  |   14.655 |     2.735   |    1.596
----------------------------------------------------------------------------

Phase 2 output

For the DIF model, a separate prior distribution is used for each group member, and the prior distribution is updated after each estimation cycle based on the posterior distribution from the previous cycle.

For the DIF model, it is assumed that different groups have different distributions with mean and standard deviation . The distributions are not necessarily normal. These empirical posterior distributions are estimated simultaneously with the estimation of the item parameters. To obtain those, parameters, the following constraint is imposed for the DIF model:

This constraint implies the overall difficulty levels of a test or a set of common items given to both the reference group and focal group, indicated by subscripts R and F, respectively, are the same. Therefore, the item difficulty parameters for the focal groups are adjusted. Any overall difference in terms of test difficulty will be assumed to be the difference in ability level for subgroups. The ability level difference among groups can then be estimated by the posterior distributions.

As no scoring function was specified on the CALIB command, the default scoring function 1, 2. will be used.

Under the partial credit model, the step parameters, also known as the item step difficulties or category intersections, correspond to the points on the ability scale where two successive item response category characteristic curves (IRCCC) intersect. Increasing difficulty of a step relative to other steps within an item is associated with higher values of the step parameters. In this example, where each item has 3 categories, 2 "steps" are needed to move from the first category to the third category: a respondent needs to move from category 1 to category 2, and a second step is needed to move from category 2 to category 3. From the second step parameters of items 1 and 2 (see below) moving from category 2 to category 3 is harder to do in the case of item 2 for the male respondents.

The IRCCC for items 1 and 5 are shown below. Vertical lines were added to indicate the trait level at which the curves for step 0 and step 1 intersect. The most likely response for a male with trait level of -2 would be to complete 0 steps in both cases. For a male with trait level of approximately 1.5, completing the step from category 2 to category 3 would be more likely in the case of item 5. Although there is little difference between the two graphs, it would appear that completing the first step is somewhat easier for item 1 than for item 5, while completing the second step is easier for item 5. This is in agreement with the second step parameters for these items: 1.769 for item 1 and 1.517 for item 5.

MULTIPLE GROUP MODEL [DIF(TREND) MODEL]
---------------------------------------

[GROUP: 1 MALE ]

   ITEM BLOCK   1 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER      :    0.000   -1.769     1.769
   S.E.                :     0.000    0.167     0.162

   ITEM BLOCK   2 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.565     1.565
   S.E.                :    0.000    0.138     0.139

   ITEM BLOCK   3 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.652     1.652
   S.E.                :     0.000    0.138     0.132

   ITEM BLOCK   4 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.667     1.667
   S.E.                :     0.000    0.139     0.137

   ITEM BLOCK   5 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.517     1.517
   S.E.                :     0.000    0.156     0.140

   ITEM BLOCK   6 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.724     1.724
   S.E.                :     0.000    0.157     0.165

The step parameter information is followed by the item parameter estimates for the male group. Standard errors are computed from the empirical information matrix in the final Newton cycle.

+------+-----+---------+---------+---------+---------+---------+---------+
| ITEM |BLOCK|  SLOPE  |  S.E.  |LOCATION |   S.E. |GUESSING |   S.E.  |
+======+=====+=========+=========+=========+=========+=========+=========+
| I001 |   1 |   0.843 |  0.053 |  -0.582 |   0.070 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I002 |   2 |   1.027 |  0.067 |   0.506 |   0.064 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I003 |   3 |   0.653 |  0.036 |  -0.529 |   0.074 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I004 |   4 |   0.645 |  0.036 |   0.532 |   0.075 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I005 |   5 |   0.447 |  0.027 |  -0.638 |   0.094 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I006 |   6 |   0.381 |  0.023 |   0.618 |   0.101 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+

Similar information for the female group is given next. Note that the slope for each item is common across the two groups. This implies that the same item discrimination is assumed over the groups.

[GROUP: 2 FEMALE ]

   ITEM BLOCK   1 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.769     1.769
   S.E.                :     0.000    0.167     0.162

   ITEM BLOCK   2 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.565     1.565
   S.E.                :     0.000    0.138     0.139

   ITEM BLOCK   3 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.652     1.652
   S.E.                :     0.000    0.138     0.132

   ITEM BLOCK   4 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.667     1.667
   S.E.                 :    0.000     0.139     0.137

   ITEM BLOCK   5 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.517     1.517
   S.E.                :     0.000    0.156     0.140

   ITEM BLOCK   6 BLOCK
   SCORING FUNCTION   :    1.000     2.000     3.000
   STEP PARAMTER       :    0.000   -1.724     1.724
   S.E.                :     0.000    0.157     0.165

+------+-----+---------+---------+---------+---------+---------+---------+
| ITEM |BLOCK|  SLOPE  |  S.E.  |LOCATION |   S.E. |GUESSING |   S.E.  |
+======+=====+=========+=========+=========+=========+=========+=========+
| I001 |   1 |   0.843 |  0.053 |  -0.609 |   0.085 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I002 |   2 |   1.027 |  0.067 |   0.624 |   0.056 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I003 |   3 |   0.653 |  0.036 |   0.014 |   0.073 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I004 |   4 |   0.645 |  0.036 |  -0.332 |   0.082 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I005 |   5 |   0.447 |  0.027 |  -0.608 |   0.112 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+
| I006 |   6 |   0.381 |  0.023 |   0.819 |   0.090 |  0.000 |   0.000 |
+------+-----+---------+---------+---------+---------+---------+---------+

DIF contrasts are given next. In the table below, the CONTRAST column gives the differences between item locations between the groups and the associated standard error. The STD column contains standardized contrasts, obtained by dividing each contrast by its standard error. The probability that a normal variate exceeds the absolute value of the standardized difference is also given. This is a one-sided test.

MULTIPLE GROUP MODEL [DIF(TREND) MODEL]
---------------------------------------

CONTRAST OF ITEM LOCATIONS:

GROUP 2: FEMALE MINUS REFERENCE GROUP 1: MALE

+---------------------------------+
  |ITEM |BLOCK| CONTRAST |  STD    |
  |    |     | (S.E.)   | (PROB. ) |
  +=====+=====+==========+==========+
  |I001 |   1 |  -0.027 |   -0.245 |
  |    |     |(   0.110)|(  0.403)|
  |    |     |          |          |
  |I002 |   2 |   0.119 |   1.393 |
  |    |     |(   0.085)|(  0.082)|
  |    |     |          |          |
  |I003 |   3 |   0.543 |   5.193 |
  |    |     |(   0.105)|(  0.000)|
  |    |     |          |          |
  |I004 |   4 |  -0.864 |   -7.743 |
  |    |     |(   0.112)|(  0.000)|
  |    |     |          |          |
  |I005 |   5 |   0.029 |   0.199 |
  |    |     |(   0.146)|(  0.421)|
  |    |     |          |          |
  |I006 |   6 |   0.201 |   1.481 |
  |    |     |(   0.135)|(  0.069)|
  |    |     |          |          |
  +---------------------------------+

-test statistics for the item location contrasts are given in the next section of the output file. In this case, with only one degree of freedom

This is a two-sided test. In the next table, these -test statistics and exceedance probabilities are summarized.

CHI-SQUARES OF ITEM LOCATION CONTRASTS:

+---------------------------------+
  |ITEM BLOCK  CHI-SQRS   D.F.   |
  |                       PROB.   |
  +=====+=====+==========+==========+
  |I001 |  1 |   0.060 |       1. |
  |    |     |          |   0.793 |
  |    |     |          |          |
  +---------------------------------+
  |I002 |   2 |   1.940 |       1. |
  |    |     |          |   0.160 |
  |    |     |          |         |
  +---------------------------------+
  |I003 |   3 |  26.966 |       1. |
  |    |     |          |   0.000 |
  |    |     |          |          |
  +---------------------------------+
  |I004 |   4 |  59.956 |       1. |
  |    |     |         |   0.000 |
  |    |     |          |          |
  +---------------------------------+
  |I005 |   5 |   0.040 |       1. |
  |    |     |          |   0.823 |
  |    |     |          |          |
  +---------------------------------+
  |I006 |   6 |   2.194 |       1. |
  |    |     |          |   0.134 |
  |    |     |          |          |
  +---------------------------------+
  |TOTAL|     |  91.156 |       6. |
  |    |     |          |   0.000 |
  +---------------------------------+

When the summary statistics for the 2 groups are compared, we see that only the standard deviation of the threshold differs. Recall that for this example, the DIF keyword on the MGROUP command was used to allow only threshold parameters to differ between the groups. Overall, no large difference between groups over all items is observed.

SUMMARY STATISTICS OF PARAMETER ESTIMATES

1 GROUP NAME: MALE

     +----------+---------+---------+----+
      |PARAMETER |   MEAN | STN DEV |  N |
      +==========+=========+=========+====+
      |SLOPE     |   0.666|   0.241|   6|
      |LOG(SLOPE)|   -0.463|   0.373|   6|
      |THRESHOLD |   -0.016|   0.624|   6|
      |GUESSING  |   0.000|   0.000|   0|
      +----------+---------+---------+----+

2 GROUP NAME: FEMALE

     +----------+---------+---------+----+
      |PARAMETER |   MEAN | STN DEV |  N |
     +==========+=========+=========+====+
      |SLOPE     |   0.666|   0.241|   6|
      |LOG(SLOPE)|   -0.463|   0.373|   6|
      |THRESHOLD |   -0.016|   0.618|   6|
      |GUESSING  |   0.000|   0.000|   0|
     +----------+---------+---------+----+

The final output is the estimated latent distribution by group. The origin and unit of the scale is set so that the mean of the reference group is 0 and the standard deviation 1.

A plot of the estimated latent distributions is given below. The solid line represents the distribution for the male group. If there is appreciable DIF, the latent distributions do not represent the same latent variable and no meaningful comparison of the two distributions is possible. If there is no DIF, significant differences between the latent distributions represent real differences between the populations sampled.

QUADRATURE POINTS AND POSTERIOR WEIGHTS:
GROUP 1 GROUP NAME: MALE

               1           2           3           4           5
POINT   -0.4000E+01 -0.3667E+01 -0.3333E+01 -0.3000E+01 -0.2667E+01
WEIGHT   0.5632E-04  0.2014E-03  0.6431E-03 0.1833E-02  0.4652E-02

               6           7           8           9          10
POINT   -0.2333E+01 -0.2000E+01 -0.1667E+01 -0.1333E+01 -0.1000E+01
WEIGHT   0.1049E-01  0.2095E-01  0.3688E-01 0.5725E-01  0.7941E-01

              11          12         13          14          15
POINT   -0.6667E+00 -0.3333E+00 0.3331E-15  0.3333E+00  0.6667E+00
WEIGHT   0.1012E+00  0.1194E+00  0.1275E+00 0.1217E+00  0.1049E+00

              16          17          18          19          20
POINT     0.1000E+01 0.1333E+01  0.1667E+01  0.2000E+01 0.2333E+01
WEIGHT   0.8245E-01  0.5801E-01  0.3610E-01 0.1987E-01  0.9722E-02

              21          22          23          24          25
POINT    0.2667E+01  0.3000E+01  0.3333E+01 0.3667E+01  0.4000E+01
WEIGHT   0.4239E-02  0.1650E-02  0.5745E-03 0.1789E-03  0.4983E-04

TOTAL WEIGHT: 1.00000
MEAN : 0.00000
S.D. : 0.99974

QUADRATURE POINTS AND POSTERIOR WEIGHTS:

GROUP 2 GROUP NAME: FEMALE

               1          2          3           4           5
POINT   -0.4000E+01 -0.3667E+01 -0.3333E+01 -0.3000E+01 -0.2667E+01
WEIGHT   0.1378E-04  0.4995E-04  0.1625E-03 0.4743E-03  0.1244E-02

               6           7           8           9          10
POINT   -0.2333E+01 -0.2000E+01 -0.1667E+01 -0.1333E+01 -0.1000E+01
WEIGHT   0.2933E-02  0.6227E-02  0.1196E-01 0.2110E-01  0.3537E-01

              11          12          13          14          15
POINT   -0.6667E+00 -0.3333E+00 0.3331E-15  0.3333E+00  0.6667E+00
WEIGHT   0.5780E-01  0.8749E-01  0.1148E+00 0.1349E+00  0.1471E+00

              16          17          18          19          20
POINT    0.1000E+01  0.1333E+01  0.1667E+01 0.2000E+01  0.2333E+01
WEIGHT   0.1373E+00  0.1047E+00  0.6725E-01 0.3758E-01  0.1857E-01

              21          22          23          24          25
POINT    0.2667E+01  0.3000E+01  0.3333E+01 0.3667E+01  0.4000E+01
WEIGHT   0.8168E-02  0.3205E-02  0.1123E-02 0.3516E-03  0.9841E-04

TOTAL WEIGHT: 1.00000
MEAN : 0.00000
S.D. : 0.99974


Copyright ® 2008, Scientific Software International, Inc., All rights reserved. 필 사이언스㈜ 경기도 고양시 일산동구 백석동1141-2 유니테크빌 825 전화 031-905-7754 팩스 031-905-7724 이메일: