Просмотр исходного кода

fix(bgs_new_daily_spider): 修正任务查询语句以优化数据获取

- 取消了原有的auth_code过滤条件,改为根据id倒序排序
- 限制查询结果为最近的10000条任务
- 提高查询效率,避免遗漏较新任务数据
- 注释保留旧的查询语句便于日后参考
- 修复可能导致任务遗漏的逻辑问题
charley 4 недель назад
Родитель
Сommit
be5682b8ee
1 измененных файлов с 2 добавлено и 1 удалено
  1. 2 1
      bgs_spider/bgs_new_daily_spider.py

+ 2 - 1
bgs_spider/bgs_new_daily_spider.py

@@ -204,7 +204,8 @@ def bgs_main(log):
         while True:
             sql_bgs_id_list = sql_pool.select_all(
                 # "SELECT auth_code FROM bgs_task WHERE state!=1 AND auth_code > 15000000 LIMIT 5000"
-                "SELECT auth_code FROM bgs_task WHERE state = 0 AND auth_code > 18606718 LIMIT 50000"
+                # "SELECT auth_code FROM bgs_task WHERE state = 0 AND auth_code > 18606718 ORDER BY id DESC LIMIT 10000"
+                "SELECT auth_code FROM bgs_task WHERE state = 0 ORDER BY id DESC LIMIT 10000"
             )
             sql_bgs_id_list = [bid[0] for bid in sql_bgs_id_list]