UniER

UniER consists of two main types of models: Item-level Exercise Recommendation and Path-level Exercise Recommendation. Below is a summary of the individual model names and their corresponding execution commands. We use the ASSISTments2017 dataset as an example.

Environment Setup

conda create -n unirec python=3.9
conda activate unirec
pip install -r requirements.txt

Data Preprocessing

Navigate to pykt-toolkit/examples and process the dataset to obtain train_valid.csv, test.csv, train_valid_sequences.csv, and test_sequences.csv:

git clone https://github.com/pykt-team/pykt-toolkit.git
cd pykt-toolkit/examples
python data_preprocess.py --dataset_name=assist17

Item-level Exercise Recommendation

Following the instructions in Data Process, obtain pkm.pth, pkc.pth, and qid_test_questions.txt for subsequent exercise recommendation processing.

python wandb_dkt_train.py --dataset_name=assist2017
python wandb_predict.py --save_dir=saved_model/

1. DRE

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv, test_sequences.csv and pkm.pth into the folder data/assist2017. Related execution commands:

# Data processing 
cd data && python data_process.py
cd .. && python Q.py

# Run DRE script:
python run_this_GPU.py

# Evaluate metrics 
python evaluate_acc.py
python evaluate_ndcg.py
python evaluate_proximity.py
cd data && python get_final_pro_rec.py
python process.py
python filter_rec.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

2. AKT

Dataset preparation: First, obtain qid_test_questions.txt. Place train_valid.csv, train_valid_sequences.csv, test.csv and test_sequences.csv into the folder dataset/assist2017. Place qid_test_questions.txt into model/akt/assist2017 Related execution commands:

# Run script and evaluate metrics 
python recommend_ex.py
python recommend_ex_list.py
cd Ep && python filter_over1_test.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

3. SimpleKT

Dataset preparation: First, obtain qid_test_questions.txt. Place train_valid.csv, train_valid_sequences.csv, test.csv and test_sequences.csv into the folder dataset/assist2017. Place qid_test_questions.txt into model/simplekt/assist2017 Related execution commands:

# Run script and evaluate metrics 
python recommend_ex.py
python recommend_ex_list.py
cd Ep && python filter_over1_test.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

4. MMER

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv and test_sequences.csv into the folder datasets/assist2017. Related execution commands:

# Data processing
python to_json.py
python Q.py

# Run MMER script
python main.py --file_dir=datasets/assist2017 --cpt_num=102

# Evaluate metrics
cd .. && python test_ep_all_MMER.py
python test_ep_por_MMER.py

5. KCP-ER

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv, test_sequences.csv, pkm.pth and pkc.pth into the folder datasets/assist2017/select. Related execution commands:

# Data processing
python select_test300.py

# Run KCP-ER script
python eb_filter.py
python REL_generator.py

# Evaluate metrics
python evaluate4acc_etc.py
python evaluate4ndcg_etc.py
python evaluate_proximity.py
cd datasets && python get_rec_20.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

6. MulOER-SAN

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv and test_sequences.csv into the folder PYKT/data/assist2017. Related execution commands:

# Preprocessing, training, and predicting based on pykt-toolkit/examples:
python data_preprocess.py --dataset_name assist2017
python wandb_akt_train.py --use_wandb 0 --add_uuid 0 --dataset_name assist2017
python wandb_predict.py --use_wandb 0 --save_dir saved_model/assist2017_akt_qid_saved_model #Place qid_test_question_predictions into the folder PYKT/data/assist2017.

# Obtaining coverage:
cd kc_cover && python transformer.py

# Further filtering and recommending exercises:
cd ../select && python chuli.py
python rec.py

# Evaluate metrics
python generate_q_matrix.py
python evaluate_acc.py
python evaluate_ndcg.py
python evaluate_proximity.py
python filter_over4_test.py
python get_original_ex_ques.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

7. KG4EX

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv, test_sequences.csv, pkm.pth and pkc.pth into the folder data/assist2017. Related execution commands:

# Data processing
cd get_KG_graph && python preprocess.py
python process.py

# Training, evaluation, and recommendation testing:
cd ../codes && python run.py --do_train --cuda --data_path ../data/assist2017 --model TransE -b 1024 -d 1000 -g 12.0 -a 1.0 -lr 0.001 -adv -save models/assist2017/TransE_adv
python test_TransE.py

# Evaluate metrics
python evaluate.py
python evaluate_proximity.py
cd Ep && python get_rec_20.py
python get_unique_last.py
cd ../../.. && python test_ep_all.py
python test_ep_por.py

8. ER-TGA

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv and test_sequences.csv into the folder datasets/assist2017. Place pkm.pth into the folder evaluate/assist2017. Related execution commands: Data processing

# Data processing
cd code && python Preprocess.py

#Run ER-TGA script
./Auto_Run_Scripts.sh

# Evaluate metrics
cd Ep && python Q.py
cd .. && python evaluate4acc_etc.py
python evaluate4ndcg_etc.py
python evaluate_proximity.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

9. NR4DER

Dataset preparation: Place train_valid.csv, train_valid_sequences.csv, test.csv, test_sequences.csv, pkm.pth into the folder dataset/data_200/assist2017. Related execution commands:

# After obtaining the Q matrix, proceed with multi-stage processing sequentially:
cd module && python Q.py
cd ../main && python stage1_main.py 
cd ../module && python EB_filter.py 
cd ../main && python stage2_main.py 

# Evaluate metrics
cd ../module && python evaluate4acc_etc.py
python evaluate4ndcg_etc.py
python evaluate_proximity.py
python rec_top20.py
cd ../.. && python test_ep_all.py
python test_ep_por.py

Path-level Exercise Recommendation

Before running the code for Path-level Exercise Recommendation, move Data Process/generate_graph_vertex.py into the pykt-toolkit/examples folder and execute it to generate graph_vertex.json. Then place it into the folder data/dataProcess/assist17.

cd pykt-toolkit/examples
python generate_graph_vertex.py