2.11测试版

我后续测试了一下,即使不使用并发,单线程请求,在有多个任务同时活动下载的时候,平均请求耗时会快速上涨,接近每个请求 4 秒。多个任务的情况下,会导致无法在合理时间内完成所有请求。

如果我停止所有的任务,这个耗时会立刻下降到合理区间,以相当快的速度完成响应。

这个问题我在 2.10测试版 - #112 2908803755 这里提到过,但随着最近的版本更新,性能似乎越来越糟糕了。

能把任务详情 /api/task/summary/get 的数据拉平到 /api_v2/task_list/get 就好,这对 WebUI 也有好处,避免了浏览器发起大量的请求,并可以显示足够多的基本信息。
这是 qbittorrent 的响应,提供的数据恰到好处:

		"c15e9b4ed3d8410b2dc479a7d74916bc6233eab0": {
			"added_on": 1728112882,
			"amount_left": 0,
			"auto_tmm": false,
			"availability": -1,
			"category": "",
			"comment": "",
			"completed": 5287520256,
			"completion_on": 1728112892,
			"content_path": "E:\\Download\\zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso",
			"dl_limit": 0,
			"dlspeed": 0,
			"download_path": "",
			"downloaded": 0,
			"downloaded_session": 0,
			"eta": 8640000,
			"f_l_piece_prio": false,
			"force_start": false,
			"has_metadata": true,
			"inactive_seeding_time_limit": -2,
			"infohash_v1": "b84e74c1dbcc88a02c5b24a6f84383f353a2e1dd",
			"infohash_v2": "c15e9b4ed3d8410b2dc479a7d74916bc6233eab0497c29b7e1c58cc9e10f54b0",
			"last_activity": 1729172163,
			"magnet_uri": "magnet:?xt=urn:btih:b84e74c1dbcc88a02c5b24a6f84383f353a2e1dd&xt=urn:btmh:1220c15e9b4ed3d8410b2dc479a7d74916bc6233eab0497c29b7e1c58cc9e10f54b0&dn=zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.tracker.cl%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.demonii.com%3A1337%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker.torrent.eu.org%3A451%2Fannounce&tr=udp%3A%2F%2Fexodus.desync.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.theoks.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker-udp.gbitt.info%3A80%2Fannounce&tr=udp%3A%2F%2Fexplodie.org%3A6969%2Fannounce&tr=https%3A%2F%2Ftracker.tamersunion.org%3A443%2Fannounce&tr=udp%3A%2F%2Ftracker2.dler.org%3A80%2Fannounce&tr=udp%3A%2F%2Ftracker1.myporn.club%3A9337%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.dler.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.bittor.pw%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.0x7c0.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fretracker01-msk-virt.corbina.net%3A80%2Fannounce&tr=udp%3A%2F%2Fopentracker.io%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.free-tracker.ga%3A6969%2Fannounce&tr=udp%3A%2F%2Fnew-line.net%3A6969%2Fannounce",
			"max_inactive_seeding_time": -1,
			"max_ratio": 1.5,
			"max_seeding_time": -1,
			"name": "zh-cn_windows_11_enterprise_ltsc_2024_x64_dvd_cff9cd2d.iso",
			"num_complete": 71,
			"num_incomplete": 80,
			"num_leechs": 0,
			"num_seeds": 0,
			"popularity": 27.251718417546623,
			"priority": 0,
			"private": false,
			"progress": 1,
			"ratio": 0.22004520165756883,
			"ratio_limit": -2,
			"reannounce": 5,
			"root_path": "",
			"save_path": "E:\\Download",
			"seeding_time": 21235,
			"seeding_time_limit": -2,
			"seen_complete": 1729171677,
			"seq_dl": false,
			"size": 5287520256,
			"state": "stalledUP",
			"super_seeding": false,
			"tags": "",
			"time_active": 21234,
			"total_size": 5287520256,
			"tracker": "udp://tracker.dler.org:6969/announce",
			"trackers_count": 20,
			"up_limit": 0,
			"uploaded": 1163493461,
			"uploaded_session": 0,
			"upspeed": 0
		}

如果需要模拟测试,这里有一段示例代码:

@Override
    public List<Torrent> getTorrents() {
        Map<String, String> requirements = new HashMap<>(); // 配置基本参数
        requirements.put("group_state", "ACTIVE");
        requirements.put("sort_key", "");
        requirements.put("sort_order", "unsorted");
        HttpResponse<String> request;
        try {
            request = httpClient.send(
                    MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_LIST.getEndpoint(),
                                    HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(requirements)))
                            .header("Authorization", "Bearer " + this.deviceToken)
                    , HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); // 获取任务列表
        } catch (Exception e) {
            throw new IllegalStateException(e);
        }
        if (request.statusCode() != 200) {
            throw new IllegalStateException(tlUI(Lang.DOWNLOADER_BC_FAILED_REQUEST_TORRENT_LIST, request.statusCode(), request.body()));
        }
        var response = JsonUtil.standard().fromJson(request.body(), BCTaskListResponse.class);

        Semaphore semaphore = new Semaphore(16); // 16 线程并发,但测试过程中发现,不管并发调多大,WebAPI 的性能都很糟糕,任务一多就非常慢,所以并发一点用都没有
        List<BCTaskTorrentResponse> torrentResponses = Collections.synchronizedList(new ArrayList<>(response.getTasks().size())); 
        try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
            response.getTasks().stream().filter(t -> t.getType().equals("BT")) // 过滤只看 BT 任务,忽略 HTTP 等
                    .forEach(torrent -> executor.submit(() -> { // 刷取每个任务的任务详情,因为仅仅获取任务列表,得到的数据和信息并不足够,这些额外的请求在任务较多的时候会显著延长耗时
                        try {
                            semaphore.acquire();
                            Map<String, String> taskIds = new HashMap<>();
                            taskIds.put("task_id", torrent.getTaskId().toString());
                            HttpResponse<String> fetch = httpClient.send(MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_SUMMARY.getEndpoint(),                                                 HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(taskIds)))
                                            .header("Authorization", "Bearer " + this.deviceToken),
                                    HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8)); // 请求 /api/task/summary/get 端点,卡住的问题也就发生在这里
                            System.out.println(System.currentTimeMillis() + " Received a RESP");
                            var torrentResp = JsonUtil.standard().fromJson(fetch.body(), BCTaskTorrentResponse.class);
                            torrentResponses.add(torrentResp);
                        } catch (IOException | InterruptedException e) {
                            log.warn(tlUI(Lang.DOWNLOADER_BITCOMET_UNABLE_FETCH_TASK_SUMMARY), e);
                        } finally {
                            semaphore.release();
                        }
                    }));
        }
        return torrentResponses.stream().map(torrent -> new TorrentImpl(torrent.getTask().getTaskId().toString(),
                torrent.getTask().getTaskName(),
                torrent.getTaskDetail().getInfohash() != null ? torrent.getTaskDetail().getInfohash() : torrent.getTaskDetail().getInfohashV2(),
                torrent.getTaskDetail().getTotalSize(),
                torrent.getTaskStatus().getDownloadPermillage() / 1000.0d,
                torrent.getTask().getUploadRate(),
                torrent.getTask().getDownloadRate(),
                torrent.getTaskDetail().getTorrentPrivate()
        )).collect(Collectors.toList());
    }

在获取完上面的任务列表和任务详情后,需为每个种子都获取一次 Peers,这一部分耗时似乎还行,考虑到大部分下载器都需要单独调用,这里可以不做修改:

    @Override
    public List<Peer> getPeers(Torrent torrent) {
        HttpResponse<String> resp;
        try {
            Map<String, Object> requirements = new HashMap<>();
            requirements.put("groups", List.of("peers_connected")); // 2.11 Beta 3 可以限制获取哪一类 Peers,注意下面仍需要检查,因为旧版本不支持
            requirements.put("task_id", torrent.getId());
            requirements.put("max_count", String.valueOf(Integer.MAX_VALUE)); // 获取全量列表,因为我们需要检查所有 Peers
            resp = httpClient.send(MutableRequest.POST(apiEndpoint + BCEndpoint.GET_TASK_PEERS.getEndpoint(),
                                    HttpRequest.BodyPublishers.ofString(JsonUtil.standard().toJson(requirements)))
                            .header("Authorization", "Bearer " + this.deviceToken),
                    HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));
        } catch (Exception e) {
            throw new IllegalStateException(e);
        }
        if (resp.statusCode() != 200) {
            throw new IllegalStateException(tlUI(Lang.DOWNLOADER_BC_FAILED_REQUEST_PEERS_LIST_IN_TORRENT, resp.statusCode(), resp.body()));
        }
        var peers = JsonUtil.standard().fromJson(resp.body(), BCTaskPeersResponse.class);
        //noinspection UnusedAssignment
        resp = null; // 立即手动释放 resp 的对象引用,部分版本 BitComet 一个响应能高达 12 MB,考虑到 PBH 的设计运行内存上限仅 386MB ,所以辅助 GC 完成垃圾回收是值得的。
        if (peers.getPeers() == null) {
            return Collections.emptyList();
        }
        var noGroupField = peers.getPeers().stream().noneMatch(dto -> dto.getGroup() != null); // 2.10 的一些版本没有 group 字段
        var stream = peers.getPeers().stream();

        if (!noGroupField) { // 对于 2.10+ 新版本,添加一个 group 过滤
            stream = stream.filter(dto -> dto.getGroup().equals("connected") // 2.10 正式版
                                          || dto.getGroup().equals("connected_peers") // 2.11 Beta 1-2
                                          || dto.getGroup().equals("peers_connected")); // 2.11 Beta 3
        }

        return stream.map(peer -> new PeerImpl(parseAddress(peer.getIp(), peer.getRemotePort(), peer.getListenPort()),
                peer.getIp(),
                new String(ByteUtil.hexToByteArray(peer.getPeerId()), StandardCharsets.ISO_8859_1),
                peer.getClientType(),
                peer.getDlRate(),
                peer.getDlSize() != null ? peer.getDlSize() : -1, // 兼容 2.10
                peer.getUpRate(),
                peer.getUpSize() != null ? peer.getUpSize() : -1, // 兼容 2.10
                peer.getPermillage() / 1000.0d, null, Collections.emptyList())
        ).collect(Collectors.toList());
    }

额外的话题:

BitComet 的 WebUI 的列表是不是不支持分页?在 qBittorrent 那里有个问题就是当种子数量过多的时候,因为没有分页,导致种子数量达到一定程度后,过多的 DOM 引发打开 WebUI 时浏览器卡死的问题。BitComet 的 WebUI 刚刚起步,现在改是个不错的时机。