我后续测试了一下,即使不使用并发,单线程请求,在有多个任务同时活动下载的时候,平均请求耗时会快速上涨,接近每个请求 4 秒。多个任务的情况下,会导致无法在合理时间内完成所有请求。
如果我停止所有的任务,这个耗时会立刻下降到合理区间,以相当快的速度完成响应。
这个问题我在 2.10测试版 - #112 2908803755 这里提到过,但随着最近的版本更新,性能似乎越来越糟糕了。
能把任务详情 /api/task/summary/get
的数据拉平到 /api_v2/task_list/get
就好,这对 WebUI 也有好处,避免了浏览器发起大量的请求,并可以显示足够多的基本信息。
这是 qbittorrent 的响应,提供的数据恰到好处:
"c15e9b4ed3d8410b2dc479a7d74916bc6233eab0": {
"added_on": 1728112882,
"amount_left": 0,
"auto_tmm": false,
"availability": -1,
"category": "",
"comment": "",
"completed": 5287520256,
"completion_on": 1728112892,
"content_path": "E:\\Download\\zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso",
"dl_limit": 0,
"dlspeed": 0,
"download_path": "",
"downloaded": 0,
"downloaded_session": 0,
"eta": 8640000,
"f_l_piece_prio": false,
"force_start": false,
"has_metadata": true,
"inactive_seeding_time_limit": -2,
"infohash_v1": "b84e74c1dbcc88a02c5b24a6f84383f353a2e1dd",
"infohash_v2": "c15e9b4ed3d8410b2dc479a7d74916bc6233eab0497c29b7e1c58cc9e10f54b0",
"last_activity": 1729172163,
"magnet_uri": "magnet:?xt=urn:btih:b84e74c1dbcc88a02c5b24a6f84383f353a2e1dd&xt=urn:btmh:1220c15e9b4ed3d8410b2dc479a7d74916bc6233eab0497c29b7e1c58cc9e10f54b0&dn=zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.tracker.cl%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.theoks.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker-udp.gbitt.info%3A80%2Fannounce&tr=udp%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=https%3A%2F%2Ftracker.tamersunion.org%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker2.dler.org%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker1.myporn.club%3A9337%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.dler.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.bittor.pw%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.0x7c0.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker01-msk-virt.corbina.net%3A80%2Fannounce&tr=udp%3A%2F%2Fopentracker.io%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.free-tracker.ga%3A6969%2Fannounce&tr=udp%3A%2F%2Fnew-line.net%3A6969%2Fannounce",
"max_inactive_seeding_time": -1,
"max_ratio": 1.5,
"max_seeding_time": -1,
"name": "zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso",
"num_complete": 71,
"num_incomplete": 80,
"num_leechs": 0,
"num_seeds": 0,
"popularity": 27.251718417546623,
"priority": 0,
"private": false,
"progress": 1,
"ratio": 0.22004520165756883,
"ratio_limit": -2,
"reannounce": 5,
"root_path": "",
"save_path": "E:\\Download",
"seeding_time": 21235,
"seeding_time_limit": -2,
"seen_complete": 1729171677,
"seq_dl": false,
"size": 5287520256,
"state": "stalledUP",
"super_seeding": false,
"tags": "",
"time_active": 21234,
"total_size": 5287520256,
"tracker": "udp://tracker.dler.org:6969/announce",
"trackers_count": 20,
"up_limit": 0,
"uploaded": 1163493461,
"uploaded_session": 0,
"upspeed": 0
}
如果需要模拟测试,这里有一段示例代码:
@Override
public List<Torrent> getTorrents() {
Map<String, String> requirements = new HashMap<>(); // 配置基本参数
requirements.put("group_state", "ACTIVE");
requirements.put("sort_key", "");
requirements.put("sort_order", "unsorted");
HttpResponse<String> request;
try {
request = httpClient.send(
MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_LIST.getEndpoint(),
HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(requirements)))
.header("Authorization", "Bearer " + this.deviceToken)
, HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); // 获取任务列表
} catch (Exception e) {
throw new IllegalStateException(e);
}
if (request.statusCode() != 200) {
throw new IllegalStateException(tlUI(Lang.DOWNLOADER_BC_FAILED_REQUEST_TORRENT_LIST, request.statusCode(), request.body()));
}
var response = JsonUtil.standard().fromJson(request.body(), BCTaskListResponse.class);
Semaphore semaphore = new Semaphore(16); // 16 线程并发,但测试过程中发现,不管并发调多大,WebAPI 的性能都很糟糕,任务一多就非常慢,所以并发一点用都没有
List<BCTaskTorrentResponse> torrentResponses = Collections.synchronizedList(new ArrayList<>(response.getTasks().size()));
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
response.getTasks().stream().filter(t -> t.getType().equals("BT")) // 过滤只看 BT 任务,忽略 HTTP 等
.forEach(torrent -> executor.submit(() -> { // 刷取每个任务的任务详情,因为仅仅获取任务列表,得到的数据和信息并不足够,这些额外的请求在任务较多的时候会显著延长耗时
try {
semaphore.acquire();
Map<String, String> taskIds = new HashMap<>();
taskIds.put("task_id", torrent.getTaskId().toString());
HttpResponse<String> fetch = httpClient.send(MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_SUMMARY.getEndpoint(), HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(taskIds)))
.header("Authorization", "Bearer " + this.deviceToken),
HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); // 请求 /api/task/summary/get 端点,卡住的问题也就发生在这里
System.out.println(System.currentTimeMillis() + " Received a RESP");
var torrentResp = JsonUtil.standard().fromJson(fetch.body(), BCTaskTorrentResponse.class);
torrentResponses.add(torrentResp);
} catch (IOException | InterruptedException e) {
log.warn(tlUI(Lang.DOWNLOADER_BITCOMET_UNABLE_FETCH_TASK_SUMMARY), e);
} finally {
semaphore.release();
}
}));
}
return torrentResponses.stream().map(torrent -> new TorrentImpl(torrent.getTask().getTaskId().toString(),
torrent.getTask().getTaskName(),
torrent.getTaskDetail().getInfohash() != null ? torrent.getTaskDetail().getInfohash() : torrent.getTaskDetail().getInfohashV2(),
torrent.getTaskDetail().getTotalSize(),
torrent.getTaskStatus().getDownloadPermillage() / 1000.0d,
torrent.getTask().getUploadRate(),
torrent.getTask().getDownloadRate(),
torrent.getTaskDetail().getTorrentPrivate()
)).collect(Collectors.toList());
}
在获取完上面的任务列表和任务详情后,需为每个种子都获取一次 Peers,这一部分耗时似乎还行,考虑到大部分下载器都需要单独调用,这里可以不做修改:
@Override
public List<Peer> getPeers(Torrent torrent) {
HttpResponse<String> resp;
try {
Map<String, Object> requirements = new HashMap<>();
requirements.put("groups", List.of("peers_connected")); // 2.11 Beta 3 可以限制获取哪一类 Peers,注意下面仍需要检查,因为旧版本不支持
requirements.put("task_id", torrent.getId());
requirements.put("max_count", String.valueOf(Integer.MAX_VALUE)); // 获取全量列表,因为我们需要检查所有 Peers
resp = httpClient.send(MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_PEERS.getEndpoint(),
HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(requirements)))
.header("Authorization", "Bearer " + this.deviceToken),
HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
} catch (Exception e) {
throw new IllegalStateException(e);
}
if (resp.statusCode() != 200) {
throw new IllegalStateException(tlUI(Lang.DOWNLOADER_BC_FAILED_REQUEST_PEERS_LIST_IN_TORRENT, resp.statusCode(), resp.body()));
}
var peers = JsonUtil.standard().fromJson(resp.body(), BCTaskPeersResponse.class);
//noinspection UnusedAssignment
resp = null; // 立即手动释放 resp 的对象引用,部分版本 BitComet 一个响应能高达 12 MB,考虑到 PBH 的设计运行内存上限仅 386MB ,所以辅助 GC 完成垃圾回收是值得的。
if (peers.getPeers() == null) {
return Collections.emptyList();
}
var noGroupField = peers.getPeers().stream().noneMatch(dto -> dto.getGroup() != null); // 2.10 的一些版本没有 group 字段
var stream = peers.getPeers().stream();
if (!noGroupField) { // 对于 2.10+ 新版本,添加一个 group 过滤
stream = stream.filter(dto -> dto.getGroup().equals("connected") // 2.10 正式版
|| dto.getGroup().equals("connected_peers") // 2.11 Beta 1-2
|| dto.getGroup().equals("peers_connected")); // 2.11 Beta 3
}
return stream.map(peer -> new PeerImpl(parseAddress(peer.getIp(), peer.getRemotePort(), peer.getListenPort()),
peer.getIp(),
new String(ByteUtil.hexToByteArray(peer.getPeerId()), StandardCharsets.ISO_8859_1),
peer.getClientType(),
peer.getDlRate(),
peer.getDlSize() != null ? peer.getDlSize() : -1, // 兼容 2.10
peer.getUpRate(),
peer.getUpSize() != null ? peer.getUpSize() : -1, // 兼容 2.10
peer.getPermillage() / 1000.0d, null, Collections.emptyList())
).collect(Collectors.toList());
}
额外的话题:
BitComet 的 WebUI 的列表是不是不支持分页?在 qBittorrent 那里有个问题就是当种子数量过多的时候,因为没有分页,导致种子数量达到一定程度后,过多的 DOM 引发打开 WebUI 时浏览器卡死的问题。BitComet 的 WebUI 刚刚起步,现在改是个不错的时机。