필사이언스 - IRT 예제

필 사이언스 | Systat | CambridgeSoft | EndNote

Bilog-mg: analysis of variant items

In this example, responses to 50 items are read from those of 100 items in the data file using the format statement

(11A1,T39,25A1/T13,25A1).

The first few lines of the data file are shown below. Each position in the item response fields for each examinee corresponds to the same item. In other examples, the association between response and item may depend on the group/form membership of an examinee.

The answer key (KFNAME keyword on the INPUT command) is given first, and is given in the first two lines of the raw data file in the same format as the item responses.

KEY 1 0000000000000000000000000 0000000000000000000000000
KEY 2 0000000000000000000000000 0000000000000000000000000
01021119001 0000040102010000200110100 2024030005001000000000233
01021119002 0004002014062000012000100 0303020013400310000002000 1 20 0

From the 50, 20 are selected as Main Test items and 4 as Variant Test items. This is indicated by setting NITEM to 24 and NVARIANT to 4 on the LENGTH command. Items for the main test are selected by name in the TESTM command; items for the variant test are selected by name in the TESTV command. The item names correspond to the sequence numbers in the original set of 100 items. Here the short form of naming and numbering is used ?the set of items forms an arithmetic progression of integer or decimal numbers allowing use of the short form (first (increment) last). A similar abbreviation may be used for consecutive item names (INAME keyword on the ITEMS command).

The analysis is performed on a sample of 200 students randomly drawn from the original sample of 660 (SAMPLE=200 on the INPUT command). The EAP scale scores of Phase 3 are computed from the responses to items in the main test.

EXAMPLE 06: ANALYSIS OF VARIANT ITEMS IN A SPELLING TEST OF RANDOMLY
SELECTED WORDS  SUBJECTS: 660 UNDERGRADUATE STUDENTS;  2PL MODEL
>GLOBAL  DFNAME='EXAMPL06.DAT', NTEST=1,NVTEST=1,NPARM=2,SAVE;
>SAVE  PARM='EXAMPL06.PAR',SCORE='EXAMPL06.SCO';
>LENGTH  NITEM=24,NVARIANT=4;
>INPUT  NTOTAL=50,KFNAME='EXAMPL06.DAT',SAMPLE=200,NIDCHAR=11;
>ITEMS  INUMBERS=(1(1)50),INAME=(I26(1)I75);
>TESTM  TNAME=MAINTEST,
        INAMES=(I26,I27,I28,I29,I31,I33,I34,
        I35,I36,I38,I39,I47,I48,I49,I50,I54,I60,I64,I68,I72);
>TESTV  TNAME=VARIANT,
      INAMES=(I53,I59,I69,I73);
      (11A1,T39,25A1/T13,25A1)
>CALIB  CRIT=.005,CYCLES=10,NEWTON=2,FLOAT,ACCEL=0.5;
>SCORE  METHOD=2,NOPRINT;

Phase 1 output

Phase 1 lists the test specifications and the assignment of items to the main test and the variants.

>ITEMS INUMBERS=(1(1)50), INAME=(I26(1)I75);

TEST SPECIFICATIONS
===================

  >TESTM  TNAME=MAINTEST,
        INAMES=(I26,I27,I28,I29,I31,I33,I34,
        I35,I36,I38,I39,I47,I48,I49,I50,I54,I60,I64,I68,I72);

TEST NUMBER: 1 TEST NAME: MAINTEST
NUMBER OF ITEMS: 20

    ITEM  ITEM      ITEM  ITEM      ITEM  ITEM      ITEM  ITEM
    NUMBER NAME    NUMBER  NAME   NUMBER NAME    NUMBER NAME
-----------------------------------------------------------------------
        1 I26          9  I34      23 I48     43 I68
        2 I27     10  I35      24 I49     47 I72
        3 I28     11  I36      25 I50
        4 I29     13  I38      29 I54
        6 I31     14  I39      35 I60
        8 I33     22  I47      39 I64
-----------------------------------------------------------------------

>TESTV TNAME=VARIANT,
INAMES=(I53,I59,I69,I73);

TEST NUMBER: 2 TEST NAME: VARIANT
NUMBER OF ITEMS: 4

    ITEM  ITEM      ITEM  ITEM      ITEM  ITEM      ITEM  ITEM
    NUMBER NAME    NUMBER  NAME   NUMBER NAME    NUMBER NAME
-----------------------------------------------------------------------
      28 I53     34  I59      44 I69     48 I73
  -----------------------------------------------------------------------

Responses of 660 examinees are read from the data records, but only 200 randomly sampled cases are included in the Phase 1 and Phase 2 analysis. The classical item statistics are shown separately for main and variant items. The test scores for the item-test correlations are based on the test scores from the main test items only.

660 OBSERVATIONS READ FROM FILE: EXAMPL06.DAT
660 OBSERVATIONS WRITTEN TO FILE: MF.DAT

REPORT ON SUBJECT SAMPLING:

LEVEL OF SAMPLING = 0.3030

660 SUBJECTS READ FROM FILE: MF.DAT
200 SUBJECTS WRITTEN TO FILE: CF.DAT

ITEM STATISTICS FOR SUBTEST MAINTEST

               ITEM*TEST CORRELATION
ITEM  NAME   #TRIED #RIGHT  PCT LOGIT/1.7 PEARSON BISERIAL
------------------------------------------------------------------------
  1  I26      200.0    134.0  67.0 -0.42    0.188  0.244
  2  I27      200.0    102.0  51.0 -0.02    0.421  0.527
  3  I28      200.0   78.0  39.0    0.26    0.294  0.374
  4  I29      200.0    147.0  73.5 -0.60    0.444  0.598
  5  I31      200.0   29.0  14.5    1.04    0.096  0.148
  6  I33      200.0   97.0  48.5    0.04    0.246  0.309
  7  I34      200.0   99.0  49.5    0.01    0.418  0.524
  8  I35   200.0    31.0 15.5    1.00  0.282 0.428
  9  I36      200.0    141.0  70.5 -0.51    0.212  0.280
  10  I38      200.0    139.0  69.5 -0.48    0.289  0.380
  11  I39      200.0    133.0  66.5 -0.40    0.521  0.675
  12  I47      200.0   74.0  37.0    0.31    0.289  0.369
  13  I48      200.0    100.0  50.0    0.00    0.497  0.623
  14  I49      200.0   84.0  42.0    0.19    0.490  0.618
  15  I50      200.0   91.0  45.5    0.11    0.429  0.539
  16  I54      200.0   77.0  38.5    0.28    0.440  0.561
  17  I60      200.0   32.0  16.0    0.98    0.245  0.369
  18  I64      200.0    100.0  50.0    0.00    0.146  0.183
  19  I68      200.0    160.0  80.0 -0.82    0.312  0.446
  20  I72      200.0   99.0  49.5    0.01    0.418  0.524
------------------------------------------------------------------------

ITEM STATISTICS FOR SUBTEST VARIANT

               ITEM*TEST CORRELATION
ITEM  NAME   #TRIED #RIGHT  PCT LOGIT/1.7 PEARSON  BISERIAL
-------------------------------------------------------------------------
1  I53      200.0    139.0 69.5 -0.48    0.454 0.596
2  I59      200.0    135.0 67.5 -0.43    0.456 0.594
3  I69      200.0   53.0 26.5  0.60    0.379 0.510
4  I73      200.0   50.0 25.0  0.65    0.069 0.094
-------------------------------------------------------------------------

Phase 2 output

Calibration of the main test items is computed as in the other examples. Without altering the item parameter estimates of those items, parameter estimates for the variants are computed with respect to the latent dimension determined by the main items.

>CALIB CRIT=.005, CYCLES=10, NEWTON=2, FLOAT, ACCEL=0.5;

(Output omitted)

SUBTEST MAINTEST; ITEM PARAMETERS AFTER CYCLE 6

ITEM INTERCEPT SLOPE THRESHOLD  LOADING  ASYMPTOTE CHISQ  DF
        S.E.      S.E.      S.E.      S.E.      S.E.    (PROB)
-------------------------------------------------------------------------
I26|  0.451 |  0.356  | -1.267  |  0.336 |  0.000  |    3.2  7.0
  |  0.648* |  0.087* |  1.794* |  0.082* |  0.000* | (0.8671)
  |    |    |    |    |    |
I27|  0.028 |  0.745  | -0.038  |  0.598 |  0.000  |    2.0  6.0
  |  0.692* |  0.150* |  0.928* |  0.120* |  0.000* | (0.9187)
  |    |    |    |    |    |
  (Similar output omitted)
I68|  0.963 |  0.561  | -1.716  |  0.490 |  0.000  |    4.7  6.0
  |  0.922* |  0.138* |  1.466* |  0.120* |  0.000* | (0.5863)
  |    |    |    |    |    |
I72|  -0.018 |  0.718  |  0.026  |  0.583 |  0.000  | 11.5  6.0
  |  0.684* |  0.148* |  0.952* |  0.120* |  0.000* | (0.0731)
-------------------------------------------------------------------------
                  * STANDARD ERROR

LARGEST CHANGE = 0.002542 2.8 129.0
0.9995)
-------------------------------------------------------------------------

  PARAMETER      MEAN STN DEV
  -----------------------------------
  SLOPE    0.609 0.218
  LOG(SLOPE) -0.558 0.368
  THRESHOLD   0.145 1.269

          ******************************
         CALIBRATION OF VARIANT ITEMS
         VARIANT
         ******************************

-2 LOG LIKELIHOOD = 4545.534

SUBTEST VARIANT ; ITEM PARAMETERS AFTER CYCLE 6

ITEM INTERCEPT  SLOPE THRESHOLD  LOADING  ASYMPTOTE CHISQ    DF
   S.E.      S.E.      S.E.      S.E.      S.E.   (PROB)
-------------------------------------------------------------------------
I53|  0.585 |  0.607 |  -0.963  |  0.519  |  0.000 |    0.0  0.0
  |  0.104* |  0.109* |  0.204* |  0.093* |  0.000* | (1.0000)
  |    |    |    |    |    |
I59|  0.516 |  0.597  | -0.865  |  0.513 |  0.000  |    0.0  0.0
  |  0.101* |  0.107* |  0.197* |  0.092* |  0.000* | (1.0000)
  |    |    |    |    |    |
I69|  -0.696 |  0.580  |  1.200  |  0.501 |  0.000  |    0.0  0.0
  |  0.000* |  0.000* |  0.000* |  0.000* |  0.000* | (1.0000)
  |    |    |    |    |    |
I73|  -0.649 |  0.093  |  6.975  |  0.093 |  0.000  |    0.0  0.0
  |  0.000* |  0.000* |  0.000* |  0.000* |  0.000* | (1.0000)
-------------------------------------------------------------------------

Phase 3 output

In Phase 3, scores for all 660 examinees are computed from the main test item response and saved to an external file. Printing of the scores is suppressed, except for the first three cases. The latent distribution estimated from all 660 cases is computed and printed. Scores are based on the unrescaled Phase 2 parameters, which are then saved to an external file.

>SCORE METHOD=2,NOPRINT;

SCORES WILL NOT BE COMPUTED FOR VARIANT ITEM SUBTESTS
PARAMETERS FOR SCORING, RESCALING, AND TEST AND ITEM INFORMATION

  METHOD OF SCORING SUBJECTS:       EXPECTATION A POSTERIORI
                 (EAP; BAYES ESTIMATION)
  TYPE OF PRIOR:          NORMAL
  SCORES WRITTEN TO FILE          EXAMPL06.SCO
  TYPE OF RESCALING:            NONE REQUESTED
  ITEM AND TEST INFORMATION:       NONE REQUESTED
  DOMAIN SCORE ESTIMATION:       NONE REQUESTED

       QUAD
TEST NAME POINTS
  -----------------------
1  MAINTEST  10
  -----------------------

GROUP  SUBJECT IDENTIFICATION                MARGINAL
WEIGHT  TEST   TRIED RIGHT  PERCENT    ABILITY   S.E.  PROB
-------------------------------------------------------------------------
  1  01021119001          |          |
  1.00  MAINTEST    20   8 40.00 |  -0.4077 0.3699 |  0.000000
  1  01041122001          |          |
  1.00  MAINTEST    20   8 40.00 |  -0.4104 0.3695 |  0.000000
  1  01051219001          |          |
  1.00  MAINTEST    20   3 15.00 |  -1.2375 0.4662 |  0.000000
-------------------------------------------------------------------------

SUMMARY STATISTICS FOR SCORE ESTIMATES
======================================

CORRELATIONS AMONG TEST SCORES

MAINTEST
MAINTEST 1.0000

MEANS, STANDARD DEVIATIONS, AND VARIANCES OF SCORE ESTIMATES

  TEST:       MAINTEST
  MEAN:      0.0919
  S.D.:      0.8984
  VARIANCE:    0.8071

ROOT-MEAN-SQUARE POSTERIOR STANDARD DEVIATIONS

  TEST:       MAINTEST
  RMS:       0.4526
  VARIANCE:    0.2048

EMPIRICAL RELIABILITY: 0.7976

MARGINAL LATENT DISTRIBUTION(S)
===============================

  MARGINAL LATENT DISTRIBUTION FOR TEST MAINTEST
MEAN   =    0.092
S.D.    =    0.979

       1     2      3      4      5
  POINT -0.4000E+01 -0.3111E+01 -0.2222E+01 -0.1333E+01 -0.4444E+00
  WEIGHT 0.1235E-03  0.2576E-02 0.2490E-01  0.1234E+00  0.3154E+00

       6      7      8      9    10
  POINT  0.4444E+00  0.1333E+01  0.2222E+01 0.3111E+01  0.4000E+01
  WEIGHT 0.3313E+00  0.1585E+00 0.3875E-01  0.4743E-02  0.2767E-03


Copyright ® 2008, Scientific Software International, Inc., All rights reserved. 필 사이언스㈜ 경기도 고양시 일산동구 백석동1141-2 유니테크빌 825 전화 031-905-7754 팩스 031-905-7724 이메일: