Computerized Adaptive Testing

Computerized Adaptive Testing.

class torch_measure.cat.AdaptiveTester(model, strategy='fisher', n_spanning=10)

Run a Computerized Adaptive Test to efficiently estimate ability.

Given a calibrated item bank (known difficulties/discriminations), adaptively selects items and updates the ability estimate after each response.

Parameters:

• model (IRTModel) – A fitted IRT model with item parameters.

• strategy (str or SelectionStrategy) – Item selection strategy: “fisher” (default), “spanning”, “random”, or a custom SelectionStrategy instance.

• n_spanning (int) – Number of spanning items (only for “spanning” strategy).

run(responses, budget=None, lr=0.1, n_steps=50)

Run adaptive testing on a single subject.

Parameters:

• responses (torch.Tensor) – Full response vector for the subject (n_items,). Only items selected by the algorithm will be “seen”.

• budget (int | None) – Maximum number of items to administer. Defaults to all items.

• lr (float) – Learning rate for ability estimation.

• n_steps (int) – Number of optimization steps per ability update.

Returns:

Results with: - ‘ability’: Final ability estimate (scalar) - ‘administered’: List of item indices in administration order - ‘responses’: List of responses to administered items - ‘ability_trajectory’: Ability estimate after each item

Return type:

dict

torch_measure.cat.fisher_information(ability, difficulty, discrimination=None)

Compute Fisher information for each item at given ability levels.

For the 2PL model: I(theta) = a^2 * P(theta) * (1 - P(theta)) For the Rasch model (a=1): I(theta) = P(theta) * (1 - P(theta))

Higher information means the item is more useful for estimating ability at that level.

Parameters:

• ability (torch.Tensor) – Subject ability values, shape (N,) or scalar.

• difficulty (torch.Tensor) – Item difficulty values, shape (M,).

• discrimination (torch.Tensor | None) – Item discrimination values, shape (M,). Defaults to 1 (Rasch).

Returns:

Fisher information matrix, shape (N, M) or (M,) if ability is scalar.

Return type:

torch.Tensor

class torch_measure.cat.MaxInfoStrategy

Select the item with maximum Fisher information at current ability estimate.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int

class torch_measure.cat.RandomStrategy

Select items randomly from the unadministered pool.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int

class torch_measure.cat.SpanningStrategy(n_spanning=10)

Select items spanning the difficulty range before switching to max-info.

First selects items evenly across the difficulty range to get a rough ability estimate, then switches to maximum information selection.

Parameters:

n_spanning (int) – Number of items to select in the spanning phase.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int

reset()

Reset spanning count for a new test session.

class torch_measure.cat.AdaptiveTester(model, strategy='fisher', n_spanning=10)

Run a Computerized Adaptive Test to efficiently estimate ability.

Given a calibrated item bank (known difficulties/discriminations), adaptively selects items and updates the ability estimate after each response.

Parameters:

• model (IRTModel) – A fitted IRT model with item parameters.

• strategy (str or SelectionStrategy) – Item selection strategy: “fisher” (default), “spanning”, “random”, or a custom SelectionStrategy instance.

• n_spanning (int) – Number of spanning items (only for “spanning” strategy).

run(responses, budget=None, lr=0.1, n_steps=50)

Run adaptive testing on a single subject.

Parameters:

• responses (torch.Tensor) – Full response vector for the subject (n_items,). Only items selected by the algorithm will be “seen”.

• budget (int | None) – Maximum number of items to administer. Defaults to all items.

• lr (float) – Learning rate for ability estimation.

• n_steps (int) – Number of optimization steps per ability update.

Returns:

Results with: - ‘ability’: Final ability estimate (scalar) - ‘administered’: List of item indices in administration order - ‘responses’: List of responses to administered items - ‘ability_trajectory’: Ability estimate after each item

Return type:

dict

torch_measure.cat.fisher_information(ability, difficulty, discrimination=None)

Compute Fisher information for each item at given ability levels.

For the 2PL model: I(theta) = a^2 * P(theta) * (1 - P(theta)) For the Rasch model (a=1): I(theta) = P(theta) * (1 - P(theta))

Higher information means the item is more useful for estimating ability at that level.

Parameters:

• ability (torch.Tensor) – Subject ability values, shape (N,) or scalar.

• difficulty (torch.Tensor) – Item difficulty values, shape (M,).

• discrimination (torch.Tensor | None) – Item discrimination values, shape (M,). Defaults to 1 (Rasch).

Returns:

Fisher information matrix, shape (N, M) or (M,) if ability is scalar.

Return type:

torch.Tensor

class torch_measure.cat.MaxInfoStrategy

Select the item with maximum Fisher information at current ability estimate.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int

class torch_measure.cat.SpanningStrategy(n_spanning=10)

Select items spanning the difficulty range before switching to max-info.

First selects items evenly across the difficulty range to get a rough ability estimate, then switches to maximum information selection.

Parameters:

n_spanning (int) – Number of items to select in the spanning phase.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int

reset()

Reset spanning count for a new test session.

class torch_measure.cat.RandomStrategy

Select items randomly from the unadministered pool.

select(ability_estimate, difficulty, discrimination, administered)

Select the next item to administer.

Parameters:

• ability_estimate (torch.Tensor) – Current ability estimate (scalar).

• difficulty (torch.Tensor) – Item difficulties (M,).

• discrimination (torch.Tensor | None) – Item discriminations (M,).

• administered (torch.Tensor) – Boolean mask of already-administered items (M,).

Returns:

Index of the selected item.

Return type:

int