Skip to content

_compare_rows_by_primary_key_streaming 并非流式,还是全量缓存了 #4

Description

@cobolbaby

diffTableTool/table_diff.py

Lines 1050 to 1066 in 38a6869

for row in cursor1:
row_dict = dict(zip(comparison_fields, row))
key = tuple(row_dict[pk] for pk in primary_keys)
rows1_data[key] = row_dict
row_count1 += 1
# 读取第二个表的数据
row_count2 = 0
for row in cursor2:
row_dict = dict(zip(comparison_fields, row))
key = tuple(row_dict[pk] for pk in primary_keys)
rows2_data[key] = row_dict
row_count2 += 1
# 收集所有主键值
all_keys = set(rows1_data.keys()) | set(rows2_data.keys())
logger.info(f"总共 {len(all_keys)} 个唯一主键值")

1kw+ 的表对比比较吃力,太吃内存资源

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions