Larger .torrent files

[honeyko]

=== A plea for larger .torrent files ===

I would like to propose that some of the original demands for small .torrent file sizes are no longer ongoing technological concerns. Today's bandwidth, processor power and drive capacity are all an order of magnitude greater than when the protocol was introduced a dozen or so years ago. The slight overhead of more pieces in larger .torrent files is now negligible compared to the large gains in swarm trading efficiency with smaller piece size.

An anecdote: my last 100+ gig UL saw 40% of uploading going to leechers (particularly the insidious IPv4/IPv6 sock-couplets who'll rob "initial seeders" blind) who wouldn't retrade. And this with only 4mb piece size.

Multi-season 1080p megatorrents (e.g., 300+gb) are now the vangard -- and it's clear to at least this individual that swarm efficiency already suffers horribly with the larger piece sizes even in considerably smaller collections.

Request: a healthy jump in the permitted size of .torrent files. For example, 10mb file sizes would permit 500gb torrents with 1mb piece size -- ideal for those entire-series torrents with several hundred or more episodes (meaning each individual episode would be broken into several hundred or a couple thousand pieces, exactly as if posted alone -- meaning much easier client appraisal of which peers seeking those particular files in the swarm are playing nice).

[NIK]

Larger torrents are becoming viable.

But they were never blocked because they weren't viable, they were blocked because mass uploads of multi-megabyte torrent files were being used in attempts to choke off uploading to the site.

Given the defensive basis of the block and the fact that they're still, at this point in time, almost always a bad idea anyway, I wouldn't expect any such change for a few more years.

[honeyko]

Is there any possibility of interim exceptions (even if only on a manual basis)? I have a 350 gigger already partially upped at a few lower-traffic places, and would like to share the love here. (.torrent file is 6.7mb; coddled for extreme longevity) I can box up all the relevant NFO details in a PM.

[NIK]

The way the system is at the moment, Mods have no ability to make exceptions.

Winston could modify that but he won't (and I would argue against it). You would simply replace automated attacks on the system with manual attacks on mods. I certainly wouldn't be willing to spend my time downloading hundreds of gigabytes to pre-validate torrents (most of which would never survive anyway).

Give it a couple of years and things might be different.

[honeyko]

Well, the .torrent already exists; so whomever can handle it gets it.

But thanks for listening.

[asterastrip]

An anecdote: my last 100+ gig UL saw 40% of uploading going to leechers (particularly the insidious IPv4/IPv6 sock-couplets who'll rob "initial seeders" blind) who wouldn't retrade. And this with only 4mb piece size.

This has happened to me to. I'm using an older version of utorrent, 2.2.1.
I tried:

bt.allow_same_ip: false
net.disable_incoming_ipv6: true

But it won't work in the version i'm using.
You happen to know if the issue is addressed in the newer versions of utorrent?

[honeyko]

This has happened to me to. I'm using an older version of utorrent, 2.2.1.
I tried:

bt.allow_same_ip: false
net.disable_incoming_ipv6: true

But it won't work in the version i'm using.
You happen to know if the issue is addressed in the newer versions of utorrent?

The disable ipv6 command doesn't work in 3.3.1 (the last good version).

[asterastrip]

The disable ipv6 command doesn't work in 3.3.1 (the last good version).

Cheers

[NIK]

It's off-topic but there are a couple interesting points here:

1. "the insidious IPv4/IPv6 sock-couplets who'll rob "initial seeders" blind"

"insidious" is too menacing a term for an inadvertent and mostly unnoticed consequence of the transitionary period in the migration to IPV6. Few-if-any of the downloaders involved will even realise what's happened, let alone understand it, let alone be deliberately exploiting it.

It would easy for client developers to mitigate (by ignoring "have" messages from peers with the same Peer ID the piece was initially sent to) but, until they do, you shouldn't initial seed on a machine with IPV4 AND IPV6 support (just as you shouldn't initial seed when there is only one peer or when there are multiple seeds). Initial seeding has NEVER BEEN universally appropriate, this is just one more condition under which it fails.

[Incidentally, any harmful effects of this anomaly wouldn't be reduced in the slightest by a reduction in piece size--the dual address peers would simply download more of the smaller pieces.]

2. The larger the torrent, the greater the likelihood of selective downloading. Selective downloading contributes to early torrent death because even a downloader who uploads 200% of their selection is always shown as a leech--making seedeer ratios appear significantly worse than they actually are and discouraging potential downloaders/potential seeds.

Furthermore: initially seeded torrents, with less redundant piece distribution, are hurt more by the completion of selective downloads than traditionally seeded torrents are.

In short: the larger the torrent (or, more specifically, the greater the number of "logical hunks" that might be selectively downloaded), the more ill advised initial seeding becomes. So initial seeding a season 2 pack is more likely to go well than initial seeding a season 1 pack; and initial seeding a series pack is begging for trouble.

[And again, the problem wouldn't be reduced by a reduction in piece size.]

[honeyko]

1. "the insidious IPv4/IPv6 sock-couplets who'll rob "initial seeders" blind"

"insidious" is too menacing a term for an inadvertent and mostly unnoticed consequence of the transitionary period in the migration to IPV6. Few-if-any of the downloaders involved will even realise what's happened, let alone understand it, let alone be deliberately exploiting it.[/quote]*Uploaders* will very definitely notice it.

It would easy for client developers to mitigate (by ignoring "have" messages from peers with the same Peer ID the piece was initially sent to) but, until they do,

I've been waiting years for client-writers to implement "easy" features (all they seem to care about is shoveling spam).

you shouldn't initial seed on a machine with IPV4 AND IPV6 support....

Aside from it being a PITA to kludge windows to toggle IPV6 (requires a restart, unless you know of a utility I haven't heard of), it's a problem greatly alleviated by smaller piece size in torrents.

(Nailing the IPV4 half of a 6/4 leech-pair in Peerblock will kick them both...but this is only a workable solution when the peer count is neither too small or excessive.)

Initial seeding has NEVER BEEN universally appropriate, this is just one more condition under which it fails.

"Appropriate" for whom? As the uploader of the torrent, all *I* is establishing the torrent to viability with the least blow-over.

[Incidentally, any harmful effects of this anomaly wouldn't be reduced in the slightest by a reduction in piece size--the dual address peers would simply download more of the smaller pieces.]

I know from my own experience that that conclusion is not true.

2. The larger the torrent, the greater the likelihood of selective downloading. Selective downloading contributes to early torrent death because even a downloader who uploads 200% of their selection is always shown as a leech--making seedeer ratios appear significantly worse than they actually are and discouraging potential downloaders/potential seeds.

First, the size of a torrent is offset by its "attractiveness" (i.e., higher-qualify 1080p files, etc); secondly, I have no reason to believe that a peer who demonstrates leeching behavior while I'm uploading the torrent is suddenly going to become a swell guy once he reaches 100% -- and it doesn't really matter to me anyway, as all I'm concerned with is whether or not *I* am being leeched while establishing the torrent in the first place.

Furthermore: initially seeded torrents, with less redundant piece distribution, are hurt more by the completion of selective downloads than traditionally seeded torrents are.

In my long experience, any desirable torrent will eventually attract at least one peer with firehose bandwidth (T1 or greater) -- he will literally dual-seed the torrent for me, keeping himself and several others at near maximum available percentage -- with the difference between where they're actually at and total availability (aside from me, the initial seed) representing that which is locked up in leeches.

In short: the larger the torrent (or, more specifically, the greater the number of "logical hunks" that might be selectively downloaded), the more ill advised initial seeding becomes. So initial seeding a season 2 pack is more likely to go well than initial seeding a season 1 pack; and initial seeding a series pack is begging for trouble.

Would you believe me if I told you that I once managed to initially seed a torrent to sustainability with less than 101% upload? -- I routinely manage less than 105% UL with 512k piece size torrents doing things my way.

And again, the problem wouldn't be reduced by a reduction in piece size.]

And again, my experience says otherwise.

-- If I "regular" seed a new archive torrent, I'm going to get robbed blind by nibblers who grab s01e01-e05, then hit STOP for the day. (I would *kill* for a client function which prevented fulling uploading selected files until the entire torrent itself was approaching completion.)

I regular seed up until there's a dozen peers (or as little as two who are really fast), then switch to initial-seed mode. (And if that means some of those hit-and-run guys are stuck at 96% of ep1, too bad. If they then split -- well, at least took off having robbed less.

My goal when uploading a torrent is to finish it, not cater to selective eaters with tiny hard-drives -- I want as great a percentage of the torrent as possible going to T1 firehose peers, because they'll spray everyone else in the torrent.