DD-Ranking Leaderboard

The score for ranking (DD-Ranking Score, DDRS) in the following table is computed by $DDRS = \frac{e^{w IOR - (1 - w) HLR} - e^{-1}}{e - e^{-1}}$, where $w$ is the weight for the HLR metric. You can specify the weight $w$ below.

$\text{Acc.}$: The accuracy of models trained on different samples.

$\text{full-hard}$: Full dataset with hard labels.

$\text{syn-hard}$: Synthetic dataset with hard labels.

$\text{syn-any}$: Synthetic dataset with personalized evaluation methods (hard or soft labels).

$\text{rdm-any}$: Randomly selected dataset (under the same compression ratio) with the same personalized evaluation methods.

$\text{HLR} = \text{Acc.} \text{full-hard} - \text{Acc.} \text{syn-hard}$: The degree to which the original dataset is recovered under hard labels (hard label recovery).

$\text{IOR} = \text{Acc.} \text{syn-any} - \text{Acc.} \text{rdm-any}$: The improvement over random selection when using personalized evaluation methods (improvement over random).