Skip to content

Forgetting is not fixed in the latest release? #291

@Newspaper8459

Description

@Newspaper8459

In metrics.py, there was a problem in the range, where it previously ranged from 0 to k-2. This had apparently been fixed in the past, and I can check that in this commit.

Below is the current code I can find in the repository:

def forgetting(all_preds, all_targets, all_tasks):
    ...

    for j in range(k - 1):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(
            _get_R_ij(l, j, all_preds, all_targets, all_tasks)
            for l in range(k - 1)
            if l >= j
        )
    ...

However, it hasn't been fixed in the latest release, which is 1.2.7.
Below is the code in the latest release 1.2.7.

def forgetting(all_preds, all_targets, all_tasks):
    ...
    for j in range(k - 2):
        # Accuracy on task j after learning current task k
        a_kj = _get_R_ij(k - 1, j, all_preds, all_targets, all_tasks)
        # Best previous accuracy on task j
        max_a_lj = max(_get_R_ij(l, j, all_preds, all_targets, all_tasks) for l in range(k - 2) if l >= j)
        f += max_a_lj - a_kj  # We want this results to be as low as possible
    ...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions