Add files via upload by moatdan · Pull Request #252 · FlagAI-Open/OpenSeek

moatdan · 2026-05-29T02:37:23Z

No description provided.

gemini-code-assist

Code Review

This pull request implements an In-Context Learning (ICL) automatic data annotation solution using Qwen3-4B optimized for Huawei Ascend 910C, including prompt strategies, caching mechanisms for token calculation, and task-specific configurations. Key feedback points out a logical discrepancy in the Chain-of-Thought (CoT) prompt selection between main.py and method.py (Task 4 vs Task 8), redundant tokenizer loading that ignores the command-line argument, hardcoded absolute paths that hinder portability, and an incorrect return type annotation for annotate_ascend.

gemini-code-assist · 2026-05-29T02:38:47Z

+    elif task_id == 8:
+        # Task 8: Kernel Generation - Code Generation


There is a logical discrepancy between main.py and method.py regarding the Chain-of-Thought (CoT) strategy. In main.py (lines 78-81), build_prompt_cot is called for task_id in [3, 4], and the comments state that CoT is harmful for Task 8. However, in method.py, build_prompt_cot implements CoT for task_id == 8 and has no implementation for task_id == 4 (which falls back to the standard prompt). Please update build_prompt_cot to implement CoT for Task 4 (String concatenation) instead of Task 8, or align the task IDs between both files.

gemini-code-assist · 2026-05-29T02:38:47Z

+def precompute_all_examples_length(all_examples: list[dict]):
+    """
+    M15优化：预先计算所有示例的长度
+    """
+    global _example_length_cache
+    tokenizer = get_tokenizer()
+
+    for i, example in enumerate(all_examples):
+        try:
+            compute_example_length(example, tokenizer)
+        except (KeyError, IndexError) as e:
+            print(f"警告：示例{i}长度计算失败，跳过")
+            continue
+
+def select_examples(all_examples: list[dict], task_description: str, text2annotate: str) -> str:
+    """
+    M15优化版本：示例缓存与长度预计算方案
+    通过预先计算和缓存示例长度，提高运行效率
+
+    Parameters:
+        all_examples: 所有示例列表，每个示例包含'input'和'output'键
+        task_description: 任务描述
+        text2annotate: 待标注文本
+    """
+    # M15优化：预先计算所有示例长度（仅第一次调用时执行）
+    if not _example_length_cache:
+        print("M15优化：开始预计算所有示例长度...")
+        precompute_all_examples_length(all_examples)
+        print(f"M15优化：长度预计算完成，已缓存{len(_example_length_cache)}个示例")
+
+    # 获取缓存的tokenizer
+    tokenizer = get_tokenizer()


The tokenizer is currently loaded twice: once in main.py (which is then unused) and once in method.py (via get_tokenizer()). Additionally, the --tokenizer_path CLI argument is ignored because get_tokenizer() always uses the hardcoded path "/root/Qwen3-4B". Passing the tokenizer instance from evaluate to select_examples avoids redundant loading and honors the CLI argument.

def precompute_all_examples_length(all_examples: list[dict], tokenizer): """ M15优化：预先计算所有示例的长度 """ global _example_length_cache for i, example in enumerate(all_examples): try: compute_example_length(example, tokenizer) except (KeyError, IndexError) as e: print(f"警告：示例{i}长度计算失败，跳过") continue def select_examples(all_examples: list[dict], task_description: str, text2annotate: str, tokenizer=None) -> str: """ M15优化版本：示例缓存与长度预计算方案通过预先计算和缓存示例长度，提高运行效率 Parameters: all_examples: 所有示例列表，每个示例包含'input' and 'output'键 task_description: 任务描述 text2annotate: 待标注文本 tokenizer: 可选的分词器实例 """ # 获取缓存的tokenizer tokenizer = tokenizer or get_tokenizer() # M15优化：预先计算所有示例长度（仅第一次调用时执行） if not _example_length_cache: print("M15优化：开始预计算所有示例长度...") precompute_all_examples_length(all_examples, tokenizer) print(f"M15优化：长度预计算完成，已缓存{len(_example_length_cache)}个示例")

gemini-code-assist · 2026-05-29T02:38:47Z

+            prompt = build_prompt(task_description, text2annotate)
+
        if examples_str is None:
            examples_str = select_examples(icl_examples, task_description, text2annotate)


Pass the loaded qwen_tokenizer to select_examples to avoid redundant tokenizer loading and to respect the --tokenizer_path command-line argument.

Suggested change

examples_str = select_examples(icl_examples, task_description, text2annotate)

examples_str = select_examples(icl_examples, task_description, text2annotate, qwen_tokenizer)

gemini-code-assist · 2026-05-29T02:38:47Z

+    1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',
+    2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',
+    3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',
+    4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',
+    5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',
+    6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',
+    7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',
+    8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',


Hardcoding absolute paths like /root/OpenSeek/... makes the code non-portable and prone to failure on other environments. Since the data directory is located relative to the script directory, it is highly recommended to construct these paths dynamically using os.path.

Suggested change

1: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-1_closest_integers.json',

2: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-2_count_nouns_verbs.json',

3: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-3_collatz_conjecture.json',

4: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-4_conala_concat_strings.json',

5: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json',

6: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-6_mnli_same_genre_classification.json',

7: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-7_jeopardy_answer_generation_all.json',

8: '/root/OpenSeek/openseek/competition/LongContext-ICL-Annotation/data/openseek-8_kernel_generation.json',

1: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-1_closest_integers.json')),

2: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-2_count_nouns_verbs.json')),

3: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-3_collatz_conjecture.json')),

4: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-4_conala_concat_strings.json')),

5: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-5_semeval_2018_task1_tweet_sadness_detection.json')),

6: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-6_mnli_same_genre_classification.json')),

7: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-7_jeopardy_answer_generation_all.json')),

8: os.path.abspath(os.path.join(os.path.dirname(__file__), '../data/openseek-8_kernel_generation.json'))

gemini-code-assist · 2026-05-29T02:38:47Z

    return prediction

-def annotate_ascend(input_prompt:str)->list[str]:
+def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:


The return type annotation of annotate_ascend is list[str], but the function actually returns a single string (whole_result.strip() or prediction which is answer[0]). Update the return type annotation to str or Optional[str] to prevent type checker warnings and avoid misleading other developers.

Suggested change

def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:

def annotate_ascend(input_prompt:str, task_id:int=None)->str:

Add files via upload

f7c6623

gemini-code-assist Bot reviewed May 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add files via upload - #252

Add files via upload#252
moatdan wants to merge 1 commit into
FlagAI-Open:mainfrom
moatdan:main

moatdan commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

gemini-code-assist Bot May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		elif task_id == 8:
		# Task 8: Kernel Generation - Code Generation

	examples_str = select_examples(icl_examples, task_description, text2annotate)
	examples_str = select_examples(icl_examples, task_description, text2annotate, qwen_tokenizer)

	def annotate_ascend(input_prompt:str, task_id:int=None)->list[str]:
	def annotate_ascend(input_prompt:str, task_id:int=None)->str:

Uh oh!

Conversation

moatdan commented May 29, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant