Spaces:
Runtime error
Runtime error
refactor(check_db): 简化并重构数据库检查脚本
Browse files合并两次数据库查询为单次查询,移除总记录数统计与amount、weight字段的查询,删除冗余中文注释,将输出字段名改为英文并使用索引访问行数据
check_db.py
CHANGED
|
@@ -4,26 +4,13 @@ from sqlalchemy import text
|
|
| 4 |
|
| 5 |
async def main():
|
| 6 |
async with AsyncSessionLocal() as session:
|
| 7 |
-
|
| 8 |
-
r = await session.execute(text("select count(1) from standard_trade_records"))
|
| 9 |
-
print(f"总记录数: {r.scalar()}")
|
| 10 |
-
|
| 11 |
-
# 2. 检查几条真实数据的翻译情况
|
| 12 |
-
query = """
|
| 13 |
-
select hs_code, product_name, origin_country, destination_country,
|
| 14 |
-
departure_port, arrival_port, transport_mode, amount, weight
|
| 15 |
-
from standard_trade_records
|
| 16 |
-
limit 5
|
| 17 |
-
"""
|
| 18 |
-
r = await session.execute(text(query))
|
| 19 |
-
print("\n数据样例:")
|
| 20 |
for row in r:
|
| 21 |
-
print(f"
|
| 22 |
-
print(f"
|
| 23 |
-
print(f"
|
| 24 |
-
print(f"
|
| 25 |
-
print(f"
|
| 26 |
-
print(f" 金额/重量: ${row.amount} / {row.weight}KG")
|
| 27 |
print("---")
|
| 28 |
|
| 29 |
asyncio.run(main())
|
|
|
|
| 4 |
|
| 5 |
async def main():
|
| 6 |
async with AsyncSessionLocal() as session:
|
| 7 |
+
r = await session.execute(text("select hs_code, product_name, origin_country, destination_country, departure_port, arrival_port, transport_mode from standard_trade_records limit 5"))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
for row in r:
|
| 9 |
+
print(f"HS: {row[0]}")
|
| 10 |
+
print(f"Product: {row[1]}")
|
| 11 |
+
print(f"Origin -> Dest: {row[2]} -> {row[3]}")
|
| 12 |
+
print(f"Port Dep -> Arr: {row[4]} -> {row[5]}")
|
| 13 |
+
print(f"Transport: {row[6]}")
|
|
|
|
| 14 |
print("---")
|
| 15 |
|
| 16 |
asyncio.run(main())
|
infrastructure/monitoring/__pycache__/__init__.cpython-314.pyc
ADDED
|
Binary file (173 Bytes). View file
|
|
|
infrastructure/monitoring/__pycache__/alert.cpython-314.pyc
ADDED
|
Binary file (1.06 kB). View file
|
|
|
infrastructure/monitoring/__pycache__/quality.cpython-314.pyc
ADDED
|
Binary file (3.13 kB). View file
|
|
|
logs/app_2026-06-10.log
CHANGED
|
@@ -56,3 +56,32 @@
|
|
| 56 |
2026-06-10 23:26:15.300 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv
|
| 57 |
2026-06-10 23:26:15.300 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv
|
| 58 |
2026-06-10 23:26:15.302 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv | Attempt: 2/3
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
2026-06-10 23:26:15.300 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv
|
| 57 |
2026-06-10 23:26:15.300 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv
|
| 58 |
2026-06-10 23:26:15.302 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/EXP_2026.csv | Attempt: 2/3
|
| 59 |
+
2026-06-10 23:26:53.877 | INFO | packages.connectors.base:run:141 - [BR] Saved 10 raw records
|
| 60 |
+
2026-06-10 23:26:53.963 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/PAIS.csv | Attempt: 1/3
|
| 61 |
+
2026-06-10 23:26:55.629 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/tabelas/PAIS.csv
|
| 62 |
+
2026-06-10 23:26:55.630 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/tabelas/PAIS.csv
|
| 63 |
+
2026-06-10 23:26:55.632 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/PAIS.csv | Attempt: 2/3
|
| 64 |
+
2026-06-10 23:26:57.987 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/URF.csv | Attempt: 1/3
|
| 65 |
+
2026-06-10 23:26:58.909 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/tabelas/URF.csv
|
| 66 |
+
2026-06-10 23:26:58.909 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/tabelas/URF.csv
|
| 67 |
+
2026-06-10 23:26:58.912 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/URF.csv | Attempt: 2/3
|
| 68 |
+
2026-06-10 23:27:01.611 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/VIA.csv | Attempt: 1/3
|
| 69 |
+
2026-06-10 23:27:02.541 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/tabelas/VIA.csv
|
| 70 |
+
2026-06-10 23:27:02.542 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/tabelas/VIA.csv
|
| 71 |
+
2026-06-10 23:27:02.544 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/VIA.csv | Attempt: 2/3
|
| 72 |
+
2026-06-10 23:27:05.032 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/NCM.csv | Attempt: 1/3
|
| 73 |
+
2026-06-10 23:27:06.159 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/tabelas/NCM.csv
|
| 74 |
+
2026-06-10 23:27:06.159 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/tabelas/NCM.csv
|
| 75 |
+
2026-06-10 23:27:06.161 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/tabelas/NCM.csv | Attempt: 2/3
|
| 76 |
+
2026-06-10 23:27:11.947 | INFO | packages.connectors.base:run:151 - [BR] Saved 10 standard records
|
| 77 |
+
2026-06-10 23:27:11.953 | INFO | infrastructure.monitoring.quality:check_batch_quality:12 - Starting quality check for batch: TEST_BATCH_001
|
| 78 |
+
2026-06-10 23:27:11.966 | INFO | infrastructure.monitoring.quality:check_batch_quality:35 - Batch TEST_BATCH_001 quality metrics - Total: 10, Missing HS: 0 (0.00%)
|
| 79 |
+
2026-06-10 23:27:12.029 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/IMP_2026.csv | Attempt: 1/3
|
| 80 |
+
2026-06-10 23:27:12.879 | WARNING | packages.core.http_client:request:79 - Request Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081) for https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/IMP_2026.csv
|
| 81 |
+
2026-06-10 23:27:12.879 | WARNING | packages.core.http_client:request:82 - SSL 证书校验失败,降级为 verify=False 重试: https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/IMP_2026.csv
|
| 82 |
+
2026-06-10 23:27:12.882 | DEBUG | packages.core.http_client:request:55 - Request: GET https://balanca.economia.gov.br/balanca/bd/comexstat-bd/ncm/IMP_2026.csv | Attempt: 2/3
|
| 83 |
+
2026-06-10 23:27:52.220 | INFO | packages.connectors.base:run:141 - [BR] Saved 10 raw records
|
| 84 |
+
2026-06-10 23:27:52.234 | INFO | packages.connectors.base:run:151 - [BR] Saved 10 standard records
|
| 85 |
+
2026-06-10 23:27:52.234 | INFO | infrastructure.monitoring.quality:check_batch_quality:12 - Starting quality check for batch: TEST_BATCH_001
|
| 86 |
+
2026-06-10 23:27:52.237 | INFO | infrastructure.monitoring.quality:check_batch_quality:35 - Batch TEST_BATCH_001 quality metrics - Total: 20, Missing HS: 0 (0.00%)
|
| 87 |
+
2026-06-10 23:27:52.238 | INFO | packages.connectors.base:run:157 - [BR] Job completed successfully
|